In today’s changing regulatory environments, utilities are beginning to see a need for business systems to collect, store, and publish meter data. These business systems must be able to handle the complex data collection requirements of large commercial and industrial customers and be a cost-effective solution for reading residential energy consumption.
For Automated Meter Reading (AMR) systems to be cost-effective business systems, they must be scalable systems. A scalable AMR system is capable of handling several thousand residential meters on the low-end to more than a million residential meters and many hundred thousand commercial and industrial (C&I) accounts on the high-end. Additionally, a scalable AMR system operating as a back office business system must be able to:
The information collection demands placed on utilities in deregulated markets affect both the quantity and quality of metering data a utility company is required to collect, and the speed at which that data is collected and shared. In addition to collecting meter quantities required by traditional utility markets (consumption, load profiles, etc.), a scalable AMR system must be able to retrieve information such as time-of-use (TOU) quantities, alarm events and power quality events directly from meters for it to be a useful business system in deregulated markets.
There are four key system requirements that address scalability in an automated meter reading system:
An AMR system must be properly designed for it to deliver the scalability and flexibility required by utilities to implement a small-scale to large-scale business solution. Key design features to look for when implementing a scalable AMR system are:
What is Scalability?
The general understanding of scalability in IT architectures is that a system is scalable if it can be expanded to meet increasing data handling and workload requirements of an application. Scalability refers to the ability of the system
to process large volumes of transactions (throughput), users, and/or data without a significant degradation in its performance and reliability. It also implies the ability to increase system capacity by deploying additional software and hardware.
Scalability Requirements
To examine the scalability requirements of an AMR system, it is useful to look at them in the context of system functions, the factors that affect scalability requirements for each function, and the affected system components.
Meter Data Collection
Meter data collection factors that affect system scalability:
AMR system components affected by meter data collection:
The primary requirement for any AMR system is to collect data from meters. Hence the first scalability requirement for an AMR system is the ability to collect data from small systems with several thousand residential meters to large systems with thousands of C&I meters and millions of residential meters. AMR systems typically read residential meters (consumption/usage) once a month to meet billing requirements. If data is needed for data analysis or data publishing, the entire residential meter population may require several readings per month. Commercial accounts typically have high-end meters that measure at least two channels of load profile (LP) data and two channels of consumption. These meters are read on a daily basis to meet the data collection requirements of deregulated markets.
Meter Data Storage
Meter data storage factors that affect system scalability:
AMR system components affected by meter data storage:
Data for an entire C&I meter population and part of the residential meter population is inserted into the database on a daily basis. After data is collected, it is stored in the database for various time periods based on regulatory requirements. Typical storage requirements for billing data is 13 months online and three to five years archived. The database storage space required to meet meter data storage requirements typically runs in the terabytes (TB). The volume of data stored in the system also impacts data collection and retrieval rates. Thus the data collection and retrieval mechanisms used by the database also affect scalability.
Meter Data Validation, Editing and Estimation (VEE)
VEE factors that affect system scalability:
AMR system components affected by VEE:
VEE algorithm
VEE application server
database management system
database design and schema
To provide billing-ready data in a deregulated market, an AMR system must be able to validate, edit, and estimate raw data read from the meter before submitting the data for billing. VEE is a computation intensive and I/O intensive process because it reads historical data from the database, runs VEE algorithms on historical and current data, then updates the database with the resulting VEE data. Therefore, processing power, complex algorithms, database size (resulting from online data storage requirements), and database schema all affect system scalability.
Meter Data Publication
Meter data publication factors that affect system scalability:
AMR system components affected by meter data publication:
database management system
file management system
export application server
An AMR system must be able to publish collected and validated data to external entities such as service companies, market participants, etc. The primary publishing requirements include:
Exporting and publishing meter data are tasks that can burden both the system database and file input and output mechanisms. It impacts the system database because many queries are needed to retrieve data from the database. It impacts the file I/O mechanisms because the data is published in multiple files.
Additional factors that affect scalability include data availability deadlines (the time of day an external entity requires the data) and system availability requirements. For example, a system might collect data between midnight and 5:00 a.m. and finish all processing by 8:00 a.m. when system operators begin working online. Also, system users may need to run ad hoc queries and reports for other applications that need meter data while data exports are in process.
Performance Criteria for Scalable AMR Systems
Below are performance criteria and measurements required for sample meter populations. The following factors were considered in the calculations used to generate this data:
The table below shows the relationship between meter population and system performance criteria. This information reflects performance criteria that should be satisfied to deliver system performance that is a linear function of the meter population. This means that once a system is tuned for the performance level required by a utility, increasing the size of the system (meter population) will not compromise system performance.
Definition of Table Column Headings:
Meters Calls/hr: The number of meters called per hour during the daily 6-hour read window.
Parallel Calls: If each meter call requires one minute, this is the number of calls that must be made in parallel to satisfy the required meter calls per hour.
Meter Data Insert Rate (record/min): One record is one 15 minute interval LP data for commercial and industrial (C&I) meters. Therefore, a 2 channel C&I meter reading 15 minute interval LP data uses 192 records/meter daily. Residential meters use 1 record/meter daily. All data read within the 6-hour read window is stored in the database within the same period.
Meters VEE per time unit: The number of meters that must be processed by the VEE subsystem in the specified time period (hour/minute) to complete the process within the two hour window.
Billing data record export per time unit: The number of meters for which data must be exported within specified period (hour/minute) to complete the process within two hours.
Note that large systems may also support 100 to 200 concurrent users who are likely to compete for system resources with batch collection processes, storage, and publishing, may pose additional performance requirements.
Design Features to Look for in Scalable AMR Systems
Distributed Architecture
This design feature provides scalability because existing software services can be replicated on multiple hardware nodes without changing the system functionality. Additional hardware is easily added to the system configuration to increase system capacity to meet growing business needs. This allows individual data processes to be distributed to independent CPUs within the system. This feature also controls deployment costs by eliminating expensive initial deployments and full system upgrades.
Workflow Based Processing
Business processes are modeled as asynchronous task-based workflows. This provides a structured process to implement the high volume, long running tasks in the system with minimal interdependencies between the tasks (and hence minimal blocking of tasks). Workflow based processing supports multiple-users, processes, and tasks simultaneously by using a pipeline approach to stage work. This is analogous to manufacturing assembly lines where individual workstations perform units of specialized work before passing it on to the next stage, thereby maximizing the total throughput.
Industry Standard Technologies
AMR systems should use robust industry standard technologies for distributed system deployment, distributed transaction processing, and for relational database management. The database technology utilized must be a proven solution with support for options like bulk load, data partitioning and instance replication.
Application Programming Interfaces (API)
A system that supports millions of meters requires interfaces that can handle more daily transactions than humans sitting at graphical user interfaces. The volume of daily transactions processed (meter change outs, customer move in/move out, data collection for failed communications, etc.) is large enough to warrant the integration of the AMR system to the enterprise systems and business processes. Typically, this is carried out using enterprise application integration (EAI), which acts as the plumbing between the AMR system and the utility’s enterprise system that coordinates the business process. Such integration requires support of application programming interfaces (API) for business process integration, or at a minimum, file interfaces for bulk setups and modifications by the AMR solution.
Data Collection System Architecture
Scalable data collection processes are a key feature of scalable AMR systems. The collection system should consist of decoupled servers capable of running asynchronously without requiring a single point of control and coordination. Systems with single control points typically create bottlenecks and experience single point failures. The communication network should support two-way communications. A communication network that depends completely on one-way communications initiated by the server, or initiated by the meter is not able to scale to meet the volume requirements of residential meters because it requires deploying a large number of communication servers and communication infrastructure. The communication network infrastructure may need to support a hierarchical topology that allows accumulation of information from multiple end-devices to a single point, thereby scaling down the number of communications initiated from the collection system, and hence infrastructure requirements and associated airtime costs.
Conclusion
An AMR system with the ability to scale and support data requirements from a few thousand C&I points to several million metering points, can be a truly scalable enterprise solution for metering process automation and meter data management. The early stages of market deregulation and evolving business requirements have introduced new strategies for utilities in traditional markets to evaluate and decide on sound, cost-effective meter reading and data management strategies using scalable AMR systems.
About the Author
Dileep Rudran is a Systems Architect in the Software Product Development and Systems Integration Group at ABB Electricity Metering in Raleigh, North Carolina. He is responsible for new architectural initiatives on the EnergyAxis suite of automated meter reading systems and for consulting with customers on systems integration and business process automation. For further information, contact ABB at: www.abb.com/metering
For Automated Meter Reading (AMR) systems to be cost-effective business systems, they must be scalable systems. A scalable AMR system is capable of handling several thousand residential meters on the low-end to more than a million residential meters and many hundred thousand commercial and industrial (C&I) accounts on the high-end. Additionally, a scalable AMR system operating as a back office business system must be able to:
- support the metering data collection requirements of utilities operating in a deregulated market
- support the revenue metering needs of utilities operating in more traditional energy markets
- allow service bureaus to use the system to read and publish data for multiple service level businesses
The information collection demands placed on utilities in deregulated markets affect both the quantity and quality of metering data a utility company is required to collect, and the speed at which that data is collected and shared. In addition to collecting meter quantities required by traditional utility markets (consumption, load profiles, etc.), a scalable AMR system must be able to retrieve information such as time-of-use (TOU) quantities, alarm events and power quality events directly from meters for it to be a useful business system in deregulated markets.
There are four key system requirements that address scalability in an automated meter reading system:
- data collection
- data storage
- data validation, editing, and estimation (VEE)
- data publication to external business entities
An AMR system must be properly designed for it to deliver the scalability and flexibility required by utilities to implement a small-scale to large-scale business solution. Key design features to look for when implementing a scalable AMR system are:
- a distributed architecture
- workflow based processing
- industry standard technologies
- application programming interfaces (API)
- the data collection system architecture
What is Scalability?
The general understanding of scalability in IT architectures is that a system is scalable if it can be expanded to meet increasing data handling and workload requirements of an application. Scalability refers to the ability of the system
to process large volumes of transactions (throughput), users, and/or data without a significant degradation in its performance and reliability. It also implies the ability to increase system capacity by deploying additional software and hardware.
Scalability Requirements
To examine the scalability requirements of an AMR system, it is useful to look at them in the context of system functions, the factors that affect scalability requirements for each function, and the affected system components.
Meter Data Collection
Meter data collection factors that affect system scalability:
- number of meters in the system
- volume of data per meter. For example, number of channels collected, and data type collected (load profile, consumption, etc.)
- frequency of measurement
– monthly, daily, hourly - frequency of data collection
- read window available to maintain low communication costs
AMR system components affected by meter data collection:
- communication server
- communication network
- corporate network for data transfer between communication servers and database servers
The primary requirement for any AMR system is to collect data from meters. Hence the first scalability requirement for an AMR system is the ability to collect data from small systems with several thousand residential meters to large systems with thousands of C&I meters and millions of residential meters. AMR systems typically read residential meters (consumption/usage) once a month to meet billing requirements. If data is needed for data analysis or data publishing, the entire residential meter population may require several readings per month. Commercial accounts typically have high-end meters that measure at least two channels of load profile (LP) data and two channels of consumption. These meters are read on a daily basis to meet the data collection requirements of deregulated markets.
Meter Data Storage
Meter data storage factors that affect system scalability:
- regulatory requirements
- historical data for validation/estimation
- historical data for dispute/audit
- version control for tracking data
AMR system components affected by meter data storage:
- database server (hardware)
- database management system
- data insertion mechanism
- database design and schema
Data for an entire C&I meter population and part of the residential meter population is inserted into the database on a daily basis. After data is collected, it is stored in the database for various time periods based on regulatory requirements. Typical storage requirements for billing data is 13 months online and three to five years archived. The database storage space required to meet meter data storage requirements typically runs in the terabytes (TB). The volume of data stored in the system also impacts data collection and retrieval rates. Thus the data collection and retrieval mechanisms used by the database also affect scalability.
Meter Data Validation, Editing and Estimation (VEE)
VEE factors that affect system scalability:
- complex rules for estimating missing data
- data volume
AMR system components affected by VEE:
To provide billing-ready data in a deregulated market, an AMR system must be able to validate, edit, and estimate raw data read from the meter before submitting the data for billing. VEE is a computation intensive and I/O intensive process because it reads historical data from the database, runs VEE algorithms on historical and current data, then updates the database with the resulting VEE data. Therefore, processing power, complex algorithms, database size (resulting from online data storage requirements), and database schema all affect system scalability.
Meter Data Publication
Meter data publication factors that affect system scalability:
- business process automation
- file-based interface with enterprise systems
- internet-based information system
AMR system components affected by meter data publication:
An AMR system must be able to publish collected and validated data to external entities such as service companies, market participants, etc. The primary publishing requirements include:
- Daily exports of a days worth of data for C&I accounts (interval meters).
- Daily exports of a months worth of data for 1/20th or 1/30th of the residential meter population. This is based on the number of working days or number of days in a month.
- Daily exports of a days worth of data for the entire meter population.
Exporting and publishing meter data are tasks that can burden both the system database and file input and output mechanisms. It impacts the system database because many queries are needed to retrieve data from the database. It impacts the file I/O mechanisms because the data is published in multiple files.
Additional factors that affect scalability include data availability deadlines (the time of day an external entity requires the data) and system availability requirements. For example, a system might collect data between midnight and 5:00 a.m. and finish all processing by 8:00 a.m. when system operators begin working online. Also, system users may need to run ad hoc queries and reports for other applications that need meter data while data exports are in process.
Performance Criteria for Scalable AMR Systems
Below are performance criteria and measurements required for sample meter populations. The following factors were considered in the calculations used to generate this data:
- For systems of 5,000 to 50,000 meters, the entire meter population is composed of C&I meters.
- For systems of 100,000 meters or more, ten percent of the meter population is C&I, with two channels of 15-minute load profile and one channel of consumption.
- A six-hour read window is used every night, with the entire meter population being read once a day.
- VEE is performed and data exported from 1/20th of the meter population on a daily basis.
- The online data retention requirement is 13 months.
- Communications to the end-device average one minute.
The table below shows the relationship between meter population and system performance criteria. This information reflects performance criteria that should be satisfied to deliver system performance that is a linear function of the meter population. This means that once a system is tuned for the performance level required by a utility, increasing the size of the system (meter population) will not compromise system performance.
Definition of Table Column Headings:
Meters Calls/hr: The number of meters called per hour during the daily 6-hour read window.
Parallel Calls: If each meter call requires one minute, this is the number of calls that must be made in parallel to satisfy the required meter calls per hour.
Meter Data Insert Rate (record/min): One record is one 15 minute interval LP data for commercial and industrial (C&I) meters. Therefore, a 2 channel C&I meter reading 15 minute interval LP data uses 192 records/meter daily. Residential meters use 1 record/meter daily. All data read within the 6-hour read window is stored in the database within the same period.
Meters VEE per time unit: The number of meters that must be processed by the VEE subsystem in the specified time period (hour/minute) to complete the process within the two hour window.
Billing data record export per time unit: The number of meters for which data must be exported within specified period (hour/minute) to complete the process within two hours.
Note that large systems may also support 100 to 200 concurrent users who are likely to compete for system resources with batch collection processes, storage, and publishing, may pose additional performance requirements.
Design Features to Look for in Scalable AMR Systems
Distributed Architecture
This design feature provides scalability because existing software services can be replicated on multiple hardware nodes without changing the system functionality. Additional hardware is easily added to the system configuration to increase system capacity to meet growing business needs. This allows individual data processes to be distributed to independent CPUs within the system. This feature also controls deployment costs by eliminating expensive initial deployments and full system upgrades.
Workflow Based Processing
Business processes are modeled as asynchronous task-based workflows. This provides a structured process to implement the high volume, long running tasks in the system with minimal interdependencies between the tasks (and hence minimal blocking of tasks). Workflow based processing supports multiple-users, processes, and tasks simultaneously by using a pipeline approach to stage work. This is analogous to manufacturing assembly lines where individual workstations perform units of specialized work before passing it on to the next stage, thereby maximizing the total throughput.
Industry Standard Technologies
AMR systems should use robust industry standard technologies for distributed system deployment, distributed transaction processing, and for relational database management. The database technology utilized must be a proven solution with support for options like bulk load, data partitioning and instance replication.
Application Programming Interfaces (API)
A system that supports millions of meters requires interfaces that can handle more daily transactions than humans sitting at graphical user interfaces. The volume of daily transactions processed (meter change outs, customer move in/move out, data collection for failed communications, etc.) is large enough to warrant the integration of the AMR system to the enterprise systems and business processes. Typically, this is carried out using enterprise application integration (EAI), which acts as the plumbing between the AMR system and the utility’s enterprise system that coordinates the business process. Such integration requires support of application programming interfaces (API) for business process integration, or at a minimum, file interfaces for bulk setups and modifications by the AMR solution.
Data Collection System Architecture
Scalable data collection processes are a key feature of scalable AMR systems. The collection system should consist of decoupled servers capable of running asynchronously without requiring a single point of control and coordination. Systems with single control points typically create bottlenecks and experience single point failures. The communication network should support two-way communications. A communication network that depends completely on one-way communications initiated by the server, or initiated by the meter is not able to scale to meet the volume requirements of residential meters because it requires deploying a large number of communication servers and communication infrastructure. The communication network infrastructure may need to support a hierarchical topology that allows accumulation of information from multiple end-devices to a single point, thereby scaling down the number of communications initiated from the collection system, and hence infrastructure requirements and associated airtime costs.
Conclusion
An AMR system with the ability to scale and support data requirements from a few thousand C&I points to several million metering points, can be a truly scalable enterprise solution for metering process automation and meter data management. The early stages of market deregulation and evolving business requirements have introduced new strategies for utilities in traditional markets to evaluate and decide on sound, cost-effective meter reading and data management strategies using scalable AMR systems.
About the Author
Dileep Rudran is a Systems Architect in the Software Product Development and Systems Integration Group at ABB Electricity Metering in Raleigh, North Carolina. He is responsible for new architectural initiatives on the EnergyAxis suite of automated meter reading systems and for consulting with customers on systems integration and business process automation. For further information, contact ABB at: www.abb.com/metering
