April 25, 2024

System Solutions to C&I Data Collection

by Shawn Fields, Director of Market Development, Datamatic Energy Systems
Utilities have traditionally collected data from their commercial and industrial (C&I) customers once a month. This data has been collected via remote interrogation over phone lines or manual probing of the meter. Recent market changes and improved communication technologies have driven utility managers, energy service providers and meter data management agents to search for new methods and products that react to these changes while streamlining operations in an effort to improve their bottom line.

These forces have made collecting and delivering the C&I data much more challenging. One of the ways to help manage these obstacles is to consider C&I data retrieval systems that support customer diversities, emerging communication technologies and energy conservation programs. This article is an analysis of market conditions, technology trends and available system solutions.

An Industry in Flux
Deregulation, new additional complexity to service territories and customer switching has created new challenges for energy service providers and data collection agents. They have been forced to become more innovative in their approach to tracking customers, delivering bills and collecting meter data while trying to reduce the cost of doing business.

In many cases, energy costs are the third highest expense an organization must contend with, yet it’s a variable that few end-users understand. While these customers are diverse in their businesses, they share a common need to manage or reduce costs. Energy companies must not only deliver reliable energy and energy services, but also offer value added products and services to teach end-use customers how to better manage their energy bills and consumption.

Market Drivers
Due to changes in the marketplace, customers are beginning to ask energy providers for more timely and accurate energy information including web-based representation of data, value added services and competitive rates. Although it is still important to use multiple meter manufacturers, it’s becoming just as vital to have the ability to support existing and emerging communication technologies that enable faster and cheaper methods of data collection. The following areas identify market drivers and changes that are creating the need for improved C&I data collection solutions:

• Deregulation - Energy shortages through transmission & distribution shortfall and more timely settlement has enhanced the need for understanding customer load on a timely basis. Where deregulation exists, so do energy service providers (ESP’s) or energy marketers that give customers a choice. In most cases, these energy providers span multiple service territories, which can create complexity and frustration to the data collection process. As margins on the energy commodity itself are thin, energy providers need not only accurate and timely data for customer billing and settlement, but also a cost effective means in which to collect it. Thus, they are moving toward the digital data technologies that support GPRS and 1xRTT. This also holds true for the meter data management agents (MDMA’s) and meter service providers (MSP’s) that offer data collection services for energy providers.


• Energy Conservation - Demand Response Programs provide a method to help reduce load and slow the need to build additional generation. Load Response Programs are used by Independent System Operators (ISO’s), utilities, or load-serving entities to ask customers to reduce load during periods of high demand or generation shortfall. There are also price response programs where end-users will react to highenergy prices during certain periods of the day.

In many cases, the data for these programs must be collected in hourly or sub-hourly intervals requiring, near real-time data collection. In order to measure the success of these programs, accurate and timely metering must be in place to measure compliance and help determine settlement to the participants.

Traditional telephone dial up methods are too costly and may exceed the data retrieval capability of most dial-up systems, like MV-90. Utility Commissions not only require 15-minute resolution, but also mandate the interval data from a broader/lower range of demand customers; this increases the number of meters that a utility needs to interrogate. Additionally, utilities are supporting new and more complex rate structures; this increases the need for load profile information for the various classes of customers and to satisfy PUC requirements. The dramatic increase in meter collection may stress or surpass the limits of existing data retrieval methods.


• Emerging Communication Technologies – As communication technologies improve, communication providers are adopting various platforms such as digital voice, digital data and Ethernet – TCP/IP; this gives utilities better choices for many types of data transfers between devices. Wireless data capabilities are becoming more widely available and analog cellular services are being phased out.

The newer data packet technologies offer full two-way communications, higher reliability, faster connections and lower costs than traditional telephone, cellular modems, or two-way paging. As the market continues to drive toward higher resolutions of data, it also drives the need for near always-on connections. C&I data retrieval systems will be expected to validate, aggregate, report and provide all the normal data management functions but in addition, are expected to seamlessly manage the various communication networks that are emerging as technology improves.

Enhanced Customer Service
Whether a utility is in a regulated or deregulated environment, providing top-notch customer service to help improve customer loyalty and customer retention remains an important goal. C&I customers are demanding more reliable and timely access to usage data to help make operational decisions that could result in measurable energy savings. Detailed profile information can also help customers predict their energy budgets or help with rate negotiations. Providing the data can improve customer service by reducing billing errors and help provide quicker response time to customer requests.

Providing value-add products and services such as Web-based energy information services can deliver better information to end-use customers.

Immediate access to the meter data via the Internet will help customers understand usage patterns and provide immediate billing calculations rather than waiting to the end of month.

In addition to billing or profile information, customers are concerned with power quality and reliability. C&I data retrieval systems should have functionality that allows for event notification and information on the quality of the power being delivered.

In summary, the following functionality within a C&I data retrieval system provides end-use customers additional value:

  • Web-based energy information (profile data)

  • Event notification

    • Demand thresholds

    • Response to curtailment programs

    • Outages

  • Power quality

  • Web-based bill analysis

System Flexibility/Architecture
Current data retrieval systems need an open and expandable architecture so that utilities, or their third party agents, can develop real time interfaces for the systems that need access to the data such as billing, load forecasting, outage management, and CIS systems. While most utilities’ processing systems historically “pushed” data at some periodic interval through custom file transfer programs, these systems are now requiring the option of realtime access to the data on-demand or “pulling” it through direct open access via SQL, APIs, or web services. The ideal system provides all these technologies / techniques since different systems have different data access requirements; the best solution for a report generator is not necessarily the most efficient interface for a billing or an event notification system.


Retrieval systems need to support an architecture that provides flexibility to expand the system vertically into multiple tiers or multiple boxes for system throughput, division of labor and the efficient utilization of available computing resources. Additionally, the system must be expandable horizontally at each tier to support multiple users/clients, communication ports, and application processors, as well as provide for functional redundancy to eliminate single points of failure.

Along with these architectural requirements, the retrieval system must provide support for multiple meter protocols, as has been accepted as a standard requirement for C&I data collection. However, they now need to provide the capability to integrate the various wireless public and private communication networks as well as Internet based metering. Just as multi-vendor was the key word for the 1980’s and 1990’s; multi-network for communications is the key requirement for the next decade. Utilities will begin to mandate the economics and ease of use of a single system that is open, expandable, and supports multiple metering protocols and communication networks.

Supporting Multi-Protocols and Communication Networks
New C&I meters are continually being introduced that incorporate higher functionality, as well as the ability to communicate over different platforms. As long as there are diverse meter manufacturers, there will be a continued need for multi-vendor data retrieval support.

The new variables are the communication networks used to collect the data from the C&I meters. The C&I data collection systems currently in use need to provide traditional telephone (POTS), Internet, as well as the new wireless public networks through a single user interface / system.

Cellular carriers are now providing data communication within their next generation d i g i t a l w i re l e s s network s i n c l u d i n g GSM/GPRS and CMDA/1xRTT. These capabilities are offering new technologies that can be utilized for more robust meter communications.

In addition, cellular providers are switching their data communications from analog packet data (CDPD) to the two digital standards mentioned above. With these new standards comes a fundamental architectural change in IP addressing techniques and communications with end point devices (meters). The challenge for the future data retrieval system is in handling the new architecture of dynamically assigned IP addresses for the communication devices (meter modems).

Traditional outbound IP communications use a static IP address, which was assigned to the CDPD modem just like a phone number is assigned to a cell phone. For the new packet switched standards (GPRS and 1xRTT), cellular providers are requiring the communication end point devices to retrieve an IP address from the system whenever they want to communicate. This is acceptable for “inbound” calls since the host system does not need to know the IP address. However, this system creates a problem for “outbound” calls since the host system does not know the IP address ahead of time.

The various communication equipment/ modem manufacturers are seeking solutions to this problem, but each seems to be designing their own techniques. Therefore, data collection systems must be able to support these varied solutions. In some cases, there is a need to implement another protocol layer while in others, the solution includes developing a seamless interface to a manufacturer’s backend communication server application. No matter which technique is implemented, the key requirement is that it remains transparent to the end-user and other interfacing systems that collect data from the meter.

Below is a graphical depiction of what C&I data collection systems must to look like in an effort to support new demands and technologies:

Adapt and Survive
The electric utility market has undertaken its most significant evolution over the past decade since regional power grids were introduced. The effects of deregulation, new customer demands and the explosion of technology has irrevocably changed the landscape. Utilities are learning to work faster, leaner and smarter while providing even more value; they expect their vendors and equipment to match those parameters.

The combination of recent market changes and improved communication technologies have dramatically impacted C&I data collection systems as utility managers, energy service providers and meter data management agents seek new ways to streamline operations.

These forces have made collecting and delivering the C&I data much more challenging. Systems that provide maximum capabilities, multi-vendor flexibility and adaptability are beginning to emerge in the marketplace. Systems that do not meet these criteria will be eventually phased out in the ultimate Darwinian competition. The days when a vendor could dictate the meters, software and collection systems that a utility must use are numbered as utilities regain control of this critical function.

About the Author
Shawn Fields is the Director of Market Development for the Datamatic Energy division. Fields has extensive experience in managing key accounts and business processes. He was previously Director of Business Development at Invensys Energy Management, global manufacturer of meters. He served as Director of Technical Services at Home Town Connections, a subsidiary of American Public Power and was Manager of Implementation & Support Services for Itron/Utility Translation Systems, which is a global manufacturer of Automated Meter Reading Systems. Fields graduated from Roberts Wesleyan College in Rochester, NY, with a Bachelors in Business Management.

Shawn can be reached for comments and questions at sfields@datamatic.com.