Accessing, transporting and managing the massive amounts of data being collected in today’s power utilities requires sophisticated technology. In addition, in an increasingly insecure world, Ethernet provides a rich set of security protocols and applications that can help keep substations secure. “Zones of protection” is one strategy that can be used to provide the level of security demanded today.

Protective relay engineers keep utility grids and equipment safe from faults and system unbalances by dividing the grid into zones, each with a unique protection scheme. Overlapping zones provide backup protection. This same strategy can be used to provide communications zones of protection to address the rich information that is collected in the course of operation today. Ethernet is the best protocol for supporting that strategy.

Figure 1 is a one-line of a typical substation depicting the zones of protection within the substation. Notice that zones overlap each other to provide backup protection should a primary zone fail.

Figure 1: Zones of protection for a substation
 

As the figure above shows, there is a zone of protection covering from each feeder breaker (Blue dotted lines), a zone covering each low-side bus, a zone covering each transformer (Green dotted lines), a zone covering each high-side bus and a zone covering each incoming transmission Line (Yellow Dotted lines). The Transformer Zone acts as a primary zone for faults internal to the transformer and a backup zone to faults on the low-side bus and feeders.

IEDs – The Information Enabler

Intelligent Electronic Devices (IEDs) collect valuable information such as fault location, relay targets, and customer usage in increasingly fine granularity to support Smart Grid. Protective relays, meters, remote terminal units, LTC/ regulator controllers, and predictive maintenance equipment are becoming rich sources of data that can be made readily available to remote users. This new information requires increased communications bandwidth and a secure strategy for transporting the information to its destination points throughout the utility, which can be best provided by an Ethernet-based architecture.

Ethernet – The Information Transport

Typical concerns with moving to Ethernet include cost of replacement of existing equipment, and, more to the point of this article, the challenge of providing security over Ethernet.

Ethernet is a more open protocol than the traditional closed serial strategies, but as StuxNet proved in 2010, even unconnected systems can fall victim to the good old “Adidas network” as employees intentionally or unintentionally expose systems to malicious attacks. Ethernet provides the flexibility and rich supporting protocols and applications that enable a utility to create powerful zones of protection for both physical and cyber assets.

Ethernet offers three major advantages for substations: bandwidth, simultaneous access, and support for multiple protocols.

Bandwidth:

With current IEDs supporting Ethernet technologies, 100MB is a typical bandwidth, and it is several orders of magnitude faster than serial ports. Speeds of 1GB and 10GB are on the horizon. Ethernet allows the large amounts of data, such as setting files and fault records, to flow across the network without slowing down time critical information such as SCADA data. Increased bandwidth also enables new applications such as inter-relay protection schemes over communications networks.

Simultaneous access:

Ethernet allows multiple users to access the same IED at the same time; e.g., a user can access an IED for viewing and changing of settings or uploading historical data at the same time SCADA is polling the IED for real-time data. Serial protocols have a master/ slave relationship, with one master that can poll only one slave device at a time. When multiple masters need to access the same slave, it is a complex process. More importantly, with Ethernet, it is possible for users to access this data remotely, saving additional time and expense.

Multiple protocols:

Ethernet allows multiple protocols to run on the same network at the same time; the user above can be using vendor-supplied software and protocol at the same time SCADA is accessing the same IED with a different protocol. Serial links cannot support multiple protocols simultaneously; each IED responds only to one pre-selected protocol. For instance, an IED may use DNP 3.0 for SCADA information but may use Modbus to change settings or download historical data.

Security in the Ethernet Era

When multiple users simultaneously access devices from outside the substation, the network is open to security challenges that would not be a consideration with serial communications. However, as mentioned previously, demand for rich and timely data, and demand to reduce costs through activities such as remote access, is driving Ethernet adoption, and security solutions are available.

Figure 2: Contrasting a serial substation with a hybrid Ethernet/ serial substation architecture
 

In the above diagram, a pure serial substation and a hybrid Ethernet/serial substation architecture are contrasted. A pure serial substation has certain levels of protection in the use of dial up modems, while a hybrid substation architecture simplifies communication and increases bandwidth among various parts of the utility. As this section demonstrates, the security capabilities of Ethernet reduce the risk while providing increased benefits in terms of simplicity and increased bandwidth.

In a hybrid or full Ethernet architecture, multiple levels of protection are required: protection against unauthorized users (authentication); protection against authorized network users accessing devices or control for which they are not authorized (authorization); and protection against snooping and hacking.

Authentication

Firewalls, typically contained in routers, are a first zone of protection. Their job is to keep unauthorized outsiders from accessing the networking within the substation where the IEDs reside. Authorized addresses are managed from a common firewall file and pushed down to all routers and switches. Remote users must be properly authenticated when logging into a router or switch using IEEE-supported security protocols such as RADIUS and TACACS+. Using a secure, encrypted tunnel to process authentication (passwords), settings and configuration files, is critical, and replacing Telnet with SSL will support that process.

In addition, routers and switches can limit physical connections to the network by blocking ports that are not assigned or disconnected without authorization, and by verifying IP addresses and MAC addresses authorized to access a port. If an unauthorized user is detected, the router or switch can turn off the port and send an alarm to the network administrator.

Authorization

A password protection zone keeps users from accessing devices where they have no privileges. NERC CIPs require IED passwords to be a minimum of eight characters, with a combination of numbers, letters and other characters.

In a system with thousands of IEDs and multiple levels of passwords within each IED, it is impractical to manage passwords manually. Technology exists that allows a server on the network to manage all passwords. An extra level of protection is available when the server not only authenticates a user, but also performs logging into authorized IEDs. In this way, the users never need to know their actual passwords. Subnet Solutions and other vendors provide password management systems that allow, servers to manage and change passwords on a periodic basis and also to disallow access to personnel who have changed functions or left the utility.

Snooping and Hacking

VLANs and VPNs offer virtual protection zones that suppress snooping and hacking. Virtual networks (VLANs) segregate IEDs connected to the same physical switches from each other for protection. To pass data between VLANs, users must goes through a Layer 3 device such as a router. These Layer 3 devices contain the firewalls to limit access from one VLAN to another. Using separate VLANs for remote access to protective relays, and placing a firewall between the remote user and the VLAN that recognizes only specific device addresses adds to security.

If a user is allowed to access certain devices on the network (e.g., regulator controls but not protective relays), these devices should be segregated onto separate VLANs. Layer 4 Ports (used by UDP or TCP) can also limit user activity. For example, a user could poll a protective relay for SCADA data using DNP 3.0, but not be able to Telnet into the protective relay to change settings.

Virtual Private Networks (VPNs) can be set up between remote users and field devices so that the user is authenticated and all data is encrypted. Sensitive data such as passwords, settings and fault data are encrypted, removing the possibility that it could be deciphered by another user on the network (snooping).

Ethernet and Physical Zones of Protection

Physical security within the substation provides an additional zone of protection through devices such as card readers installed on control house doors and outdoor cabinets such as transformer cabinets. Security cameras monitor substation access and watch over sensitive areas to guard against unauthorized penetration, copper theft and other attacks. Security data may be logged onto a substation server and also sent through the network to a central monitoring area for viewing in real-time or for archiving. When configuring communications transport for physical security data, it is important that Quality of Service (QoS) levels are set that ensure that data will be received in a timely manner.

Getting to a Secure Modern Substation

The secure modern substation requires a different approach to substation communications architecture. The demand for both security and for timely data will necessitate a series of interrelated, but not necessarily interconnected LANs, in an overall interconnection strategy connecting both downstream devices and upstream data centers, some of which have been described in this article. This architecture addresses both security and network integrity, and it must provide a migration path from serial networks and from lower bandwidth interconnection technologies.

Because many of the IEDs that support only serial communications are relatively newly installed and can be expected to perform their primary functions for an additional decade or more, utilities, of course, want to maximize their investments. They are not looking to replace these devices prematurely. Nonetheless, these devices have several limitations that must be addressed.

Integrating Serial Connectivity into the Zones of Protection

When integrating serial IEDs into an Ethernet-based substation communications architecture, it is important to consider password security and secure connectivity as integral to the overall network.

First, many of the older IEDs supported only simple 4-character passwords. The new password management systems, with their tiered levels of password support, can map the serial IED’s password into a more complex 8-character password that will meet CIP requirements.

Second, there are two fundamental approaches to integrating serial devices securely onto the overall network: gateways and terminal servers.

A gateway can be inserted between the serial IEDs and the network that will poll the IEDs via serial communications and collect the data into a common database that can be accessed over the Ethernet network. Gateways are typically hardened computers that act as small SCADA masters, polling IEDs using various protocols and converting and combining the data into a common database that can be accessed from the main SCADA system or by remote users.

Figure 3: Typical Hybrid Substation Automation LAN with connection to WAN
 

Gateways can be expensive and time consuming as each gateway must have a separate database that needs to be programmed and maintained.

Terminal servers, on the other hand, perform a basic Ethernet-to-serial conversion so the end users can access the IEDs as if there were connected directly to the network, and thus eliminate the need for additional databases. Some terminal servers also support secure communications such as SSL and RADIUS to maintain the ESP.

Summary

The performance, flexibility and data management capabilities of Ethernet-based make a strong argument for transitioning to Ethernet-only solutions. However, practically speaking, hybrid transitional architectures will be standard operating procedure for the foreseeable future.

While security concerns are real, they can be easily overcome by deploying proper zones of protection. These zones start in the network equipment and access to upstream control centers and users, and also propagate through password servers to the downstream IEDs.

The costs of fiber-based connectivity have decreased, making fiber an affordable alternative to telephone lines for connectivity with the central office. Security protocols and strategies have advanced to address concerns associated with cyber attacks. Today, networks, engineered properly through the use of today’s security and connectivity technologies, can provide the required functionality and security without disregarding the existing investments in serial devices.

About the Author

Robert (Bob) McFetridge, Strategic Account Manager for GarrettCom, Inc., has worked in the utility industry for more than 25 years, including seven years at two large investor-owned utilities in the Southeast. His experience includes protective relays, SCADA, substation automation, distribution automation and Volt/ VAR SmartGrid projects.