The term “big data,” has emerged to describe the nearly inconceivable amount of data the business world now generates, and the job of analyzing it. Market research firm IDC, for one, sees “big data” becoming the next must-have organizational competency this year.¹ Consider the credit card companies that now alert customers to divergent patterns in their purchasing trends, in an effort to fight identity theft. And then there are the wireless providers that supply customers with reams of insight into not only how many text messages they’ve sent, but to whom and on what day.

Now, it’s the utilities’ turn to leverage their data. Thanks to the investments that many energy providers have made in advanced metering infrastructure (AMI) and meter data management (MDM) software, utilities are now poised to meaningfully analyze data and provide strategic insights to all levels of business users. This directly benefits executives, for strategic planning; engineers, for systems planning; and line personnel, for improving operations.

Analytics: Beyond Meter-to-Cash

In order to unlock the full value of AMI, utilities will need to implement MDM software capable of processes beyond basic meter-to-cash functionality. Meter data analytics provides utilities with the information they need to:

• Generate new customer insights
• Manage and even prevent outages
• Size distribution assets
• Implement preventive maintenance techniques
• Forecast and build predictive models for demand program planning
• Develop new rate plans and services for customers

It is meter data analytics that will enable utilities to tackle the biggest problems they face today, including failing transformers, unbalanced energy generation based on imprecise forecasts, operational inefficiencies and even energy theft. In the end, it is meter data analytics that will pave the way to a more engaged – and profitable – relationship with energy consumers.

But while many utilities are beginning to recognize the need for meter data analytics, they’re often too bogged down with massive amounts of data and aging enterprise systems to even know where to begin. Energy firms will need to invoke IT leadership to help them develop an analytics capability that aligns with business strategy, uses accurate and up-to-date information and is based on a foundation that is extendable to everyone in the organization who needs these insights, with virtually no interruption to operational MDM functions.

MDM: The Source of Truth

Through investments in MDM systems, utilities now collect hundreds of millions of events and readings every day from sources such as the following:

• Meters (status, manufacturer, purchase date, events such as reprogramming notifications and tamper alerts)
• Transformers (ID, circuit section, circuit ID)
• Service points
• Customer accounts (type, status, billing cycle)

Collected data includes interval readings, register readings, meter-related problems, outage information, data quality information, and more. This data is heavily analyzed to ensure accuracy, calculate statistics and isolate any potential issues. This means the MDM is considered the “system of truth” for all data generated by smart meters, as well as the control point from the organizational network, down to customer homes and businesses.

By building on MDM with data-intensive applications, organizations can extract additional value from all this data. Just as credit card companies have fortified customer relationships by learning more about consumer behavior, desires and needs, utilities can improve customer satisfaction, boost revenues and become a more efficient utility as a whole.

Consider the following opportunities that meter data analytics enable:

• Identification of unbilled revenue: Meter events and usage information can help paint an overall picture of what’s happening with a customer’s energy usage over time. This unified view can help detect energy theft, meter tampering or equipment problems that may be affecting service levels. For instance, customers can be identified who have active accounts but no recorded usage, or the converse – energy usage but no active account. In many instances, water, gas and electricity providers are able to detect key indicators of unbilled revenue. For example, many utilities see customers with water usage but no electrical usage over months or even years, indicating a very likely candidate for investigation.
• Outage event analysis and prevention: Today, some utilities are still unable to verify an outage unless personnel physically visit the suspected problem area to confirm. With outage event analysis, however, the utility can know the exact piece of equipment causing a problem, along with the customers directly impacted by it. Utilities can obtain this more granular view of outage information by using outage information that is delivered along with meter readings to identify and track outages, rather than legacy phone-call-based systems where customers often had to notify the utility first when there was an outage. These outage event reports can help utilities understand the overall impact of outages, then drill down to find hot spots and particular problem areas in the distribution network. They can then isolate areas of high impact and work to understand how to address them.
• Meter quality assurance: Focusing on meter reading performance enables utilities to ensure AMI reliability. For instance, when meter readings are expected but not delivered, the system takes note, and calculates overall performance statistics for the AMI system. Utilities are made privy to problems they never would have been able to identify in the past.

Data analytics can open the door to a range of possibilities.
From customer profiling to distribution planning, analytics
unlocks the true value of the data utilities already collect.

Smoothing the Way to Analytics

Many users in the organization are well aware of the opportunities that meter data analytics pose. Because of this, analysts in various business functions are already chomping at the bit to begin analyzing this data. However, before making the data accessible to users, it’s crucial to choose a meter data management platform with an analytics foundation that takes the following into consideration:

• Ensure support for many use cases: Organizations need to choose an analytic foundation that is flexible for as many use cases as possible. With the rich set of possibilities offered by meter data analytics, it would be wasteful and potentially harmful to future competitiveness to pick a system that limits you to just one capability. The energy industry is undergoing massive change, so it’s important that the system is powerful enough to support possibilities that haven’t been considered yet. The challenge is choosing an MDM foundation that is flexible enough to handle many different types of analysis, while also considering the specific needs of your everyday business processes.
• The need for reliable, billing-quality data: A second challenge is the need for reliability and trust in the data that flows into the MDM system. Some utility data that originates at customer sites – particularly in rural areas – is either never delivered or does not pass validation checks upon arrival, for instance, and needs to be vetted for accuracy.
• No impact on meter-to-cash: The third challenge is ensuring your system can use the data in the MDM system (the “system of truth”) without impacting basic meter-to-cash operations in any way. As important as meter data analytics is, this capability cannot interrupt billing and other operational systems in terms of performance, data corruption or functionality. Bottom line: The analytics capability cannot threaten the utility’s ability to collect revenues.
• Near real-time: Lastly, in order to retain its value to executives, engineers and operational staff, data analytics need to be performed in as near real-time as possible.

The ultimate goal is to establish a repeatable data analytics discipline and infrastructure to reduce the time, cost and complexity of each incremental capability, and with the lowest risk possible to the existing MDM functionality.

All data should be fed into two different databases to
ensure the sanctity of basic meter-to-cash functionality.

Building the Analytics Database

To preserve the sanctity of the MDM system and enable the essential capabilities described above, it’s necessary to have a second system that is separate from but cohesive with the data stored in the MDM system. This separate system – we’ll call it the analytics database – should be structured very differently from the MDM database. This is because the MDM database is normalized for “fast writes,” since it needs to quickly record large volumes of real-time meter information. On the other hand, the analytics database needs to be normalized for “fast reads,” since it needs to provide fast access to data for users looking for real-time insights.

To leverage all the work that has already been done to create the MDM system, the analytics database should use a star schema design that classifies the attributes of an event into “facts,” which would include the data itself, and “dimensions” that can give the facts context such as a customer’s name or location. In the case of utilities, “facts” can include data interval values, register reads, billing values, missed reads and meter events. “Dimensions” can include meters, transformers, service points and customer accounts (see diagram below, to the right).

The analytics database should be separate from the operational
data base, and use a star schema design for fast reads of the
data, ensuring real-time accessibility for all users.

Using the star schema design, the analytics database correlates measured data (“facts”) along many “dimensions” (e.g., by postal code, by transformer, etc.) and stages them so that the data can be analyzed in many ways. This enables users to gain more understanding of events, as well as what they mean. For instance, analysts can correlate power outage events (“fact”) with the transformers (“dimension”) to identify faulty or aging infrastructure with a single simple calculation.

These analytics processes are data intensive and will likely need to be performed throughout the business day; yet another crucial reason for this entire system to be separate from the operational MDM database and avoid interruption of business-critical processes.

High-Quality Data

When it comes to populating the analytics database, the data needs to be extracted from the MDM database into the analytics database in a way that arranges the facts and dimensions in the star schema design. This is known as the extract, transform and load process, or ETL. For data that does not change very often (accounts, meters, transformers), the ETL can be scheduled on an hourly or daily basis. However, traditional ETL cannot be used for the billions of transactions that flow from AMI networks into the MDM system. Instead, a “real-time message bus” should be employed. The “message bus” can essentially grab the data before it reaches the meter-to-cash system, thereby avoiding performance disruptions but ensuring real-time data is available for analysis.

To ensure data is of the highest quality when it’s loaded into the analytics database, other processes are involved, including data validation, estimation and editing. It’s important, for instance, to isolate “bad data” (inconsistent or inaccurate data) and identify missing reads for meter readings that don’t come in or arrive late. The latter can be calculated to understand more about AMI performance and service levels.

Once the data is staged in a separate system that’s isolated from the meter-to-cash processes, it’s ready to be made available for online reporting, graphical representations of data and queries. As time goes on, you can continue to expand the system’s value to meet the needs of limitless numbers of use cases. The beauty is, you’ve properly leveraged the investments made in your AMI and MDM systems without disrupting their functionality.

Align With the Business

Along with ensuring the best system design, it’s also crucial that the meter data analysis system is in line with the business architecture; in other words, you first need to determine the business needs, and establish your strategic intent. Once you have strategic alignment, you can begin to understand what types of analysis you want to do and what benefits the analysis will provide.

Addtionally, the analytics system needs to meet the needs of users throughout the organization, from senior executives, to operations workers who interact with customers, manage the grid, plan maintenance and purchase equipment. That’s why utilities need to establish a repeatable discipline and process for their analytics functions. The first time a new capability is rolled out, it might be an involved exercise, but with a sound infrastructure in place, each incremental capability will require less work.

Ultimately, utilities need to choose an MDM platform capable of creating an adaptive, analytically-based, data-driven organization, in which the metrics and data being collected and the analysis performed are all linked with the business’s strategic priorities.

Data Analytics: Key to Survival

To win in business today, data is the name of the game. For utilities, one of the only ways to turn data into valuable information is through MDM software. For energy firms still in the MDM planning stages, it’s key to choose a platform that enables the four essential criteria of meter data analytics: support for multiple use cases, high-quality data, no impact on meter-to-cash functionality, and the ability to operate in near-real-time.

In the end, data analytics will be the key to survival for energy providers. Utilities can only unlock the true value of their data by choosing a flexible and powerful meter data management platform with the proper data analytics foundation. By utilizing the data they already collect, utilities will finally be able to modernize and adapt to the fast-changing world of energy provisioning.

Building the Analytics Database

To preserve the sanctity of the MDM system and enable the essential capabilities described above, it’s necessary to have a second system that is separate from but cohesive with the data stored in the MDM system. This separate system – we’ll call it the analytics database – should be structured very differently from the MDM database. This is because the MDM database is normalized for “fast writes,” since it needs to quickly record large volumes of real-time meter information. On the other hand, the analytics database needs to be normalized for “fast reads,” since it needs to provide fast access to data for users looking for real-time insights.

About the Author

BK Gupta is a Senior Project Manager at eMeter, A Siemens Business, focusing a line of software, hardware, and partnerships that provide essential services for the Smart Grid.

¹ Source: http://www.businesswire.com/news/home/20111201005201/en/IDC-Predicts-2012-Year-Mobile-Cloud-Platform