Accurately forecasting weather conditions is a challenge faced by all utilities. Expensive decisions related to storm preparedness and demand/renewable forecasting are made on a daily basis, but how good are the forecasts utilities rely on? Macrosoft’s 2016 utility survey delves into these issues to answer valuable questions.
Introduction
Macrosoft’s 2016 survey examines utility weather forecasting practices and usages of the data. The online survey was completed during August 2016 by 109 respondents representing primarily investor owned utilities, with a smaller portion from municipals and cooperatives.
Every day, utilities make several important operational and financial decisions based on weather forecasts. But where do these forecasts come from and how good are they?
The survey takes a systematic approach to address these questions with the objectives stated as:
- Understand weather forecasting practices employed by utilities
- Identify perceived strengths and weaknesses in existing practices
- Categorize how weather data is being utilized
- Determine best practices
Weather Forecasting Resources and Services
There are a wide variety of weather forecasting resources available to electric utilities. Many are free public-domain options that include, the National Weather Service, internet sites, and even TV weather reports. Next, there are commercial subscription services that deliver custom forecasts based on specific utility related requirments. At the next level, there is an in-house meteorologist working directly for the utility.
Staff Meteorologist
A staff meteorologist is a dedicated resource working directly for the utility. Having a highly trained resource who understands your specific needs and requirements producing daily forecasts sounds like something that is highly desirable. But, it turns out that only 29 percent of the respondents have a staff meteorologist. That leaves 71 percent without a dedicated, professionally trained resource.
- 79% of the companies that employ an in-house meteorologist have at least 500K customers
- 64% of the companies that employ an in-house meteorologist open their Storm Center at least three times a year
Commercial Weather Forecasting Service
There are a number of commercial companies that offer customized weather forecasts specifically designed for electric utilities. It is not atypical for a utility to utilize more than one of these services. The results state that 73 percent of the responding utilities use at least one of these commercial services, with 11 percent using 3 to 4 different services. This leaves 27 percent of the respondents not using any commercial services.
Public Domain Forecasts
While the ‘public domain’ forecasts are prepared by trained meteorologists, they do not often take into account the weather elements and geographic particulars that are relevant to electric utilities.
With all too important decisions made by utilities based on their weather forecasts, it was interesting to gather statistics on utilities that exclusively rely on public domain forecasts. The results show that 31 percent responded that they solely rely on public domain forecasts. This is statistically similar to the 27 percent that reported on the previous question that they do not use any commercial services. Interestingly, 28 percent of the respondents that said they solely rely on public domain forecasts employ a staff meteorologist. In Macrosoft’s attempt to understand what types of utilities solely rely on public domain forecast sources, several data correlations were examined.
- Smaller companies rely solely on public domain forecasts, with 64% having less than 500K customers.
- The number of Storm Center openings does not affect a utility’s reliance on public domain forecasts. 50% opened less than 2 times and 50% more than three times.
- 29% of the companies that solely rely on public domain forecasts are located outside of the US.
Weather Data Usages
The average person looks at daily weather forecasts to determine if they need a sweater, umbrella or boots. Electric utilities are also watching forecasts but the decision they make are much more consequential with potentially huge financial impact.
Importance of Weather Forecast in Business Operations
Participants ranked on a scale of one to five the importance of using the weather forecasts across four separate business operations. Leading across responses was the use of forecasts for Storm Preparedness with it being ranked over 4.5 out of 5. The three other surveyed uses of weather data is being used for Demand Forecasting 3.85; Outage Prediction Modelling 3.74; and Renewables Forecasting 3.03.
Each of these four areas has a large financial impact to the company, but preparing for storms appears to be the top reason utilities are collecting or preparing forecasts. It is Macrosoft’s opinion that as this valuable forecast data is coming into the company and being utilized for storm preparedness there would be great value with little incremental investment to support other areas of their operations, assuming there is a high degree of confidence in the forecast data.
Outage Prediction Models
About 20 percent of the electric utility companies responding to the survey indicate they do not have a formal outage prediction model. The companies with outage prediction models in place are evenly split between feeding the weather data elements into the outage algorithm.
Of the companies that feed the forecast data into outage prediction models, only 33 percent apply statistical verification of the forecasts they receive. An additional 17 percent indicated though they don’t currently verify accuracy currently they would like to include verification of forecasts.
Macrosoft safely concludes that as the use of outage prediction models increase, reliable weather forecasting becomes even more important.
Of the companies that feed the forecast data into outage prediction models, the top three elements that go into the model include: Max Winds, Ice Accumulation and Snow Accumulations. Rarely used elements included Wet Bulb, Humidity and Barometric Pressure.
Forecast Outlook
When preparing this survey we suspected that the majority of electric utility companies forecasts looked out 7-10 days. We were surprised to learn 60 percent of participants’ forecasts are under one week out. Clarification was not received as to why this length of time was selected, but it is suspected that it is due to accuracy. With companies feeling that decisions should not be made based on forecast data that is greater than one week out as the accuracy is too low to make business impactful conclusions.
Specific Weather Elements
Gaining a clear understanding of which weather elements are forecasted, and electric utilities’ satisfaction with the forecasts they receive is very insightful. The 109 participants were asked to rank twelve weather elements for their importance to decision making and the satisfaction they current have with the forecasts provided.
Most surprising was uncovering that across the twelve elements surveyed the three utilities rate as the most important: maximum winds; ice accumulation and sustained winds were amongst the lowest satisfaction.
When looking at the full table of information, it is clear that Ice Accumulation is the biggest area of concern. It ranks as the second most important weather element but dead last in satisfaction of the forecast. As little as a quarter of an inch difference across lines and trees results in a huge variance in the amount of downed equipment. If weather services could simply get this right, utilities would be much happier.
It is imperative to know which forecasts get ice accumulation correct as it is so important to decision making.
Best Practices
One of the key objectives of the survey is to identify best practices being employed at various utilities. It is helpful to know about what colleagues at other utilities are doing and saying in the area of weather forecasting when it comes to formulating your company’s strategies. This section looks deeper into several areas where utilities are working towards improving the accuracy of their forecasts. These areas include: the geographic specificity of forecasts, the collecting of observational data, and applying statistical verification to compare actual versus forecast. Additionally, we found a significant amount of evidence by asking the open ended question, “In your opinion, what can be done to improve confidence with your weather forecast?”
Geographic Specificity of Forecast
Almost half of the respondents’ report using a Regional (covering a large service territory) forecast.
General purpose regional forecasts that cover a large geographic area might not provide the level of detail to address specific utility related issues. Differences such as elevation, coastal proximity, population density, tree coverage, etc. potentially impact the accuracy of these regional forecasts. At the opposite end of the spectrum are Point Specific (individual location) forecasts, which are reported by 11%. Finer granularity is desirable, especially when feeding forecasts into other systems such as outage prediction models. The remaining third of the respondents report a hybrid approach, ‘somewhere in-between’ a regional and point forecast.
Collection of Observational Weather Data via Field Weather Stations
Another best practice is the collection of observational weather data via field weather stations. This type of service is provided by the National Weather Service, Earth Networks (Weather Bug), and a variety of other providers.
Statistical Verification of Weather Forecasts
Statistical verification of weather forecasts is an interesting best practice which involves comparing the weather forecast to the actual observational results using advanced statistical methods.
Thirty percent of respondents report that they are currently employing statistical verification of their forecasts. Additionally, 23 percent report that they are not currently, but would like to in the future.
Factors Needed to Improve Confidence in Weather Forecasting
The open ended question, “In your opinion, what can be done to improve confidence with your weather forecast?” was asked to all respondents. Since confidence is a critical element in any forecast, this provides a broad collection of evidence that is useful in identifying best practices. The responses to this question are classified in three general categories: technological improvements, forecasting improvements, and general improvements.
Conclusions
Macrosoft elected to focus our annual Utility survey on how weather forecasting is being used by electric utilities. The topic was selected as the weather conditions appear to be more severe with hotter hots, colder colds and windier winds. Utilities are faced with providing continuous power to an unforgiving client base that expects undisrupted delivery no matter what the conditions. Due to the huge financial decisions that are made based on weather forecasts Macrosoft wanted to understand best practice and usage of forecasts to determine how this can be measured and scored.
The survey provided many insightful observations including:
- Over 70% use commercial services but only 29% of the respondents have a staff meteorologist. That leaves 71% without a dedicated, professionally trained resource
- Of the companies that feed the forecast data into outage prediction models, only 33% apply statistical verification of the forecasts they receive.
- The most important weather elements are the toughest to forecast, specifically Ice Accumulation ranks as the second most important weather element but dead last in satisfaction of the forecast.
It is disappointing that only 13 percent of respondents rank the forecasts they receive as a 9 or better on a scale of 1-10. Almost every day somewhere in the United States there is an impactful storm. These storms have a major impact on how electric utilities manage their operations. Waiting until it hits is too late.
Utilities are investing hundreds of millions into hardening their infrastructure to ensure they can withstand the impact of a weather event. It is our conclusion that companies need to be focusing resources on ensuring they received detailed forecasts of weather elements that are impactful to operations and then score these forecasts as to their accuracy so good decisions can be based on good data. Utilities using forecasts must eliminate the garbage in – garbage out paradigm.
Today electric utilities are well past the age when simply “Red Sky at Night… Sailor’s Delight” reflected an accurate forecast. In today’s digital world with real-time major financial decisions in a connected world, it is important to ensure the forecasts you receive are the best they can be and that the decisions you make are based on focuses that score highly on a accuracy scale.
The full 24 page report can be found at www.resourceson-demand.com
About the Authors
Jason Singer has been the Director of Macrosoft’s Utilities Practice since May 2005. Jason manages all aspects of Macrosoft’s utility portfolio including Resources on-Demand, Assessments on-Demand, Outage Central, and Mine-Weather. Jason works closely with dozens of major utility clients to delivery technology solutions that solve emergency restoration challenges. Jason earned a bachelor’s degree from Rutgers University.
Dr. Ronald Mueller is CEO and Founder of Macrosoft, Inc., an enterprise software company in Parsippany, NJ. Macrosoft has proprietary software products including Resource-on-Demand software for electric utilities. Ron has a career-long passion in ultra-large-scale data processing and analysis including: predictive analytics; data mining, and AI. Ron has a Ph.D. in Theoretical Physics from New York University.
John Kullmann is Vice President at Macrosoft. With more than twenty years’ experience, John is a recognized expert in business development efforts for professional services firms. He is responsible for expanding Macrosoft from its traditional roots as a leading software development and system implementation company into an equally accomplished provider of packaged technology products.
