April 24, 2024

Keeping the Lights on for Communications Systems: The Need for Improved Network Monitoring

by Ashish Singh, Global Product Management, GridMaven
For nearly a century, U.S. utilities have routinely managed complex energy systems to deliver safe, reliable power. In contrast, utility communication networks have been simpler. Even as telephone lines, fiber optics and advanced radio networks were added, they were purpose built and often based on low-bandwidth technologies that delivered relatively modest quantities of communications data. This is changing as utilities move forward with ambitious Smart Grid visions.

Many utility Smart Grid plans share common elements such as advanced metering infrastructure, advanced distribution management systems and distribution automation. The data needs of these different Smart Grid applications vary significantly, so utilities are deploying varied portfolios of two-way communications technologies across their service territories. Utilities are now among the largest users of privately owned and operated communications networks; their needs are increasingly being met by a variety of new and legacy technologies, including fiber optics, power line carrier and various licensed and unlicensed wireless spectrum.

Complexity Has Arrived
Several factors can affect the delivery of data across a network, including latency, packet loss, retransmission delays and throughput. Latency is particularly important for mission-critical applications and latency requirements for utility applications span a wide range: less than 10 ms for teleprotection, 20 ms for synchrophasors, 100-200 ms for SCADA and VoIP and 2-14 seconds for smart meters.

Utilities are finding that hybrid, heterogeneous communications networks are the norm, not the exception. For example, a utility may choose to build out a new RF mesh network for AMI but then tap into public cellular and existing fiber for backhaul purposes. Customers located at the edge of the service territory or in terrain that causes issues for RF mesh may require additional connection via cellular, satellite or another network. The utility may have legacy systems in place for SCADA and also have plans to enhance DA capabilities, which may require additional communications infrastructure.

Utility Smart Grid communication networks are like an orchestra in search of a conductor. Distributed autonomous devices (“nodes” in network parlance) route, gather, and process data in complex ways. Depending on the deployment architecture, each node may be able to communicate through other nodes to send information back to the network operators. Intermediate nodes may aggregate and cache data to reduce bandwidth usage for data transmission back to the utility. Some applications will be data-gathering, requiring network nodes to report data periodically to a collection unit. Some are event driven, meaning nodes only send data when an event of interest (e.g., a fault or exception) occurs. Many applications, particularly in control centers, are a hybrid of these two kinds.

As Smart Grid applications continue to grow, new networks may be added and new uses of existing networks will increase network complexity, bandwidth demands and performance sensitivity. To avoid possible disruptions of the grid system, a highly reliable, scalable, secure, robust and cost-effective integrated communications infrastructure is needed. Utilities must establish Quality of Service (QoS) metrics with their vendors and be able to monitor and successfully manage these requirements across devices from all networks in order to fully exploit the power of a unified network infrastructure.

The Multi-Network Monitoring Dilemma
Multi-vendor, multi-protocol, varied latency requirements-based Smart Grid communications systems are becoming common. This environment creates a host of management challenges. While any given utility will have specific requirements, there are basic components in managing any type of network.

  1. Network Planning: Determining the optimal network set-up and planning for growth based on the underlying business case for which the Smart Grid network is being deployed and the requirements of the services that will use it.
  2. Network Security: Ensuring data and applications are only accessed by authorized users and often ensuring standards compliance.
     
  3. Network Analysis: Monitoring and reporting of key performance indicators in a network to give an overall view of network health, to document standards compliance and to aid in troubleshooting.
     
  4. Device Management: Monitoring, updating or controlling specific devices on the network, or components of the network itself.

Managing multiple networks together as one system is not a simple task. Needing to manage networks across a variety of platforms can significantly increase operational expenses, not least of which is the cumulative cost of training personnel. Element management tools may be provided to manage each individual network but operators are still left with multiple silos that are not integrated. Some networks may even require management at the device level. Locating failures across multiple network management platforms is extremely difficult without both end-to-end provisioning and a unified performance monitoring mechanism. Troubleshooting across multiple networks is also complicated and requires high backend integration costs.

For example, a utility with a million meters may see up to one percent of those generating events and alarms daily. At 10,000 alarms per day, that is overwhelming even for a sophisticated Network Operation Center. However, many utilities lack alarm correlation techniques to determine the root cause of faults across different network platforms so fault restoration is often less than optimal.

Reactive problem detection can lead to longer communication outages, increasing the possibility of affecting power service. Insufficient information on how the network is used and uncertainty about how planned changes will affect the network and grid applications make network planning difficult and sometimes incomplete. Limited monitoring of network performance metrics also creates difficulty for operators to understand if vendor Service Level Agreements are being upheld. The need to maintain multiple maintenance teams of engineers hampers quick fault detection and has an adverse effect on the operator’s OPEX costs.

The key challenges for utility network operators with multi-network management can be summarized as the following:

  • High cost of training personnel across multiple management platforms and performing device management
     
  • Reduced efficiency in identifying root cause of network faults and exceptions with siloed management systems
     
  • Manually monitoring and analyzing large quantities of network health data becomes unsustainable
     
  • Reactive-only maintenance misses opportunities to prevent interruptions and improve asset life

As intelligent devices are deployed en mass across a utility service territory, the communication infrastructure becomes more dynamic and complex. The need for a Smart Grid communications-focused network management platform is now even more critical than before. Utilities are renowned for their abilities to monitor and manage the power grid; this rigor and sophistication should be applied to utility communications as well.

Tapping Telecom’s Transformation to Meet Unique Utility Needs
The telecommunication industry has undergone a similar dramatic transformation over the past decade and has leveraged the Telecommunications Management Network (TMN) protocol to deal with the challenges. This model has four layers: business management, service management, network management and element management, which in turn, cover a wide variety of management areas including planning, installation, operations, administration, maintenance, and provisioning of telecommunications networks and services. Utilities can benefit from this framework and extend the paradigm for the unique needs of managing Smart Grid networks. However, the challenge is to apply an inherently hierarchical TMN architecture to a wired and wireless multi-vendor, multi-protocol, varied latency requirements-based Smart Grid communications system.

Figure 1 captures the process flow for Smart Grid specific network management.


Figure 1

 
Utilities have unique communications needs and require management tools that address:

  • Performance and fault management recommendations specific to Smart Grid networks. For example, for an RF mesh network, relevant statistics might include hops per router and range per node.
     
  • Interworking functions between various segmented networks managing different latency-based service requirements. For example, incorporating data from an AMI network into an OMS system.
     
  • Performance objectives and procedures for utility communication networks. Utilities’ objectives may be impacted by regulators and consumer advocacy groups.
     
  • An end-to-end view of entire system performance to better understand reasons for business service impacts. For example, if an operator does not receive confirmation that a control signal was delivered to backhaul network device, s/he will want to pinpoint where faults occurred throughout the entire system that may have caused an interruption.
     
  • Management of differentiated QoS guarantees for different network uses. The same network may have more than one use, for example, backhaul for AMI and DMS. Operators will want to monitor and manage different QoS metrics for each application.
     
  • Combining legacy systems (e.g., SCADA), which may be decades old but with newly deployed networks. Fully integrated management of a utility’s communications infrastructure requires visibility into all networks.
     

The Next Wave of Management Tools
Traditional element and network management solutions often do not allow network administrators to keep pace with the size and complexity of Smart Grid networks. The latest tools for utility communications take a centralized approach to monitoring. This new method goes beyond managing just one network or one type of network technology, such as wireless networks, to view the entire communications enterprise via a “single pane of glass”. In some ways, this new approach mirrors the way utilities have always monitored the flow of electrons – real-time monitoring of multiple, disparate applications and systems.

GridMaven’s Network Manager, developed by the utility division of SK Telecom Americas, centrally monitors RF Mesh, cellular, PLC, fiber, satellite or any other network a utility may use. This approach, developed though years of operator experience, embodies the new breed of tools which help utilities to not only maintain networks but to fully leverage their communications infrastructure to capture increased business benefits.


Figure 2


A centralized Smart Grid ‘Manager-of-Manager’ platform provides the necessary tools for rigorous and sophisticated network management by extending key TNM concepts and adapting them for Smart Grid networks. The platform can provide a complete end-to-end view of the system health and fault and performance data from different network elements.

Such a platform will not replace individual tools provided by vendors who build out utility communications systems but rather augment their usefulness. It will sit atop the communications ecosystem and gather data from element management systems and directly from network devices, if necessary. The platform should also incorporate policy, or rules-based, management functions. Operators set a course of action to be followed for a specific network issue or combination of issues across the system. This ensures standard procedures are followed and decreases likelihood of service disruption.

In general, utility network operators will see the following business benefits from using a centralized management platform:

  • Meet QoS expectations through end-to-end service visibility
     
  • Optimize network resources to improve performance and quality
     
  • Cut network operating costs
     
  • Support network planning process to roll out more Smart Grid network services
     
  • Expedite Smart Grid network diagnostics


Operationally, the key benefits for network operators can be grouped into two categories, as shown in Figure 3.


Figure 3


Whether network operators want to undertake a quick network diagnostic, develop a detailed backhaul network upgrade or validate an existing network SLA and QoS, such a technology will support the business in clarifying its Smart Grid goals and objectives.

Fulfilling the Promise of a Smarter Grid
One of the visions of the Smart Grid is to optimize asset utilization and increase grid operation efficiency. Tools that can help bridge the gap between IT and telecom operations are crucial to helping utilities achieve their plans. The availability of centralized grid communications intelligence will give management, planners and engineers the knowledge to build what is needed when it is needed, extend the life of legacy assets, repair equipment before it fails unexpectedly and more effectively manage the communication system that is becoming the intelligence of the Smart Grid.

About the Author

Ashish Singh is Global Product Manager for GridMaven and has 19 years of diverse experience in the communications technology industry that includes in-depth market, product and technical insight into mobile carrier (MNOs) networks and Cloud architecture. He is deeply experienced in wireless Wi-Fi, VoIP, WLAN, 802.11, 4G mobile networks, 3GPP, Smart Grid and M2M networks, DPI and data traffic management. He can be reached at asingh@gridmaven.com.