Today Ethernet is the dominant networking technology used in office and home environments. Because Ethernet networks are inexpensive and fairly well understood, their use is quickly becoming popular for industrial and utility applications including substation automation networks.
Ethernet networks were not developed specifically for operation in substations and other harsh environments. So, why is there so much interest in applying Ethernet networks in these locations? The answer is similar to why personal computers are now used in many industrial and power system applications. Ethernet is so popular in other applications that it is simpler to employ and enhance Ethernet than to create something new.
Only 15 years ago, most Human Machine Interfaces (HMIs) operated on dedicated mainframe computers with terminals rather than the legion of personal computers that is used today. Early personal computer HMIs used custom operating systems dedicated to HMI operation. While dedicated systems are more stable and reliable, today’s systems often cost from 10 to 1 percent of the expense of dedicated singlepurpose systems.
Both industrial and utility networking experts are moving forward accepting the limitations of Ethernet networks and solving the problems associated with Ethernet networks. Advances in computing power and network technology allow us to take advantage of the popularity and availability of Ethernet networking equipment and solutions.
Ethernet Physical and Data Link Layers
Each standard physical layer and corresponding data link layer has a designator (e.g., 10BASE-T) that identifies the layer specifications. The most popular physical and data link layer combinations for Local Area Networks (LANs) within a single building are fiber optics (10BASE-FL and 100BASE-FX) and twisted-pair metallic (10BASE-T and 100BASE-TX). For general use networks, 10 Mbps and 100 Mbps are the most popular data transmission speeds. As data demands on networks have increased, 100 Mbps networks have become more popular. The actual network loading for a given offered load is about 10 times lower on 100 Mbps networks than on 10 Mbps networks, reducing collisions and network latencies.
The first generation of Ethernet networks used coaxial cable wired in multidrop topologies that were expensive and difficult to modify after installation. The present generation of Ethernet standards uses twisted-pair wiring or fiber optic cables in a star network topology. A star network topology employs a central node (hub or switch) to connect the individual network segments. The star network is more robust and less expensive to install and modify than multidrop network topologies.
Improvements on the hub including switches and routers enhance network performance. Twisted-pair network standards allow unshielded twisted-pair wiring (UTP) similar to that required for telephones. Sites can be prewired using multiple cables of the same type to each anticipated node location. The cables can then be used for telephone or network operation as required, increasing the flexibility and decreasing the cost of twisted-pair networks.
Unfortunately, while UTP cables are inexpensive and simple to apply in an office environment, they can be a problem in other environments. For example, offices typically do not contain strong sources of radio frequency interference (RFI) because of the shielding provided by office buildings and the lack of strong RFI sources within the building. Utility and industrial installations, however, often contain strong internal sources (hand-held radios, variable frequency drives, welders, etc.) and may not be well shielded from external sources including radio transmitter towers. Substations also include control wiring that typically is not shielded and can induce voltages in adjacent wiring.
Also, it is critical to note that you should never run any type of metallic communications cable from the substation control house to equipment in the substation yard. The ground potential differences experienced during a fault in the substation can subject equipment to damaging voltages and currents, especially in cables with shields that are grounded at both ends.
In a substation control house, many of these risks are decreased, but the noise from control and instrumentation circuits and circuit breakers in metal-enclosed switchgear can disrupt networks. Even with shielding and physical separation from other cables and wiring, metallic cables with shields provide paths for current to flow from ground potential differences, DC faults, and other stray electricity. The only sure protection from these problems is a cable system that is unaffected by electrical and electromagnetic interference.
Fiber-Optic Cable
Fiber-optic cable systems provide two principal benefits. First, the signals within fiber-optic cables are immune to RFI and electrostatic interference that can disrupt communication on metallic cables. Second, fiber-optic cables can have an all-dielectric (nonconducting) construction. This means that you can run fiber-optic cables outside of the control house to provide robust and reliable communication without the threat of damaging critical equipment at the ends of the communications circuit.
Where twisted-pair wiring uses two pairs, one for transmit and one for receive, fiber-optic cable systems use a pair of fibers, one for transmit and one for receive. Fiber-optic cable systems also require a central node or hub that combines the point-to-point fiber-optic cable segments into one logical network.
The most popular fiber-optic Ethernet network standards are 10BASE-FL and 100BASE-FX, 10 Mbps and 100 Mbps, respectively. Fiber-optic cable is more expensive than metallic cable. However, shielding measures for twisted-pair cables and installation labor are significant construction costs. You must consider the installed cost of the network to properly evaluate the impact of your choice of network physical medium.
Industrial Protocols
Several of the most popular industrial integration protocols are either running over Ethernet networks or are being prepared for operation over Ethernet networks. For example, Modbus TCP is Modbus for use over TCP/IP networks. Other industrial protocols, including ControlNet, Profibus, and Foundation Fieldbus, are migrating to Ethernet networks. It is important to consider this work when you plan to use Ethernet in power system applications. The office environment has made technologies inexpensive and available for industrial applications and in cases where officegrade equipment is not adequately robust or rugged. Office-grade equipment provides a foundation for the understanding and development of appropriate industrial Ethernet network components.
Ethernet Network Topology
It is very popular to characterize Ethernet networks (and other substation automation networks) as a magic bus that connects all devices and solves all problems. However, the magic bus concept is only partially correct, even in a logical sense. With an Ethernet network that contains a hub, the network is actually wired as a star that functions logically as a bus. If a switch is the central node, each segment between a switch and a node operates independently with the switch buffering and directing traffic to reduce collisions and decrease message transmission delays.
You may want to connect your network as a simple star topology with a switch or hub, but there are several additional considerations. External access from an engineering network requires an entry point to the substation LAN, typically through a router. You should consider whether engineering access and potentially mission-critical data should travel over the same network segment.
Engineering networks are also often connected to corporate networks and ultimately to the Internet. If you connect a mission-critical LAN to the engineering network, you have provided a path (if a hacker defeats security measures) from the Internet to your mission-critical substation LAN. A simple denial-of-service attack within your corporate LAN could jeopardize the substation LAN.
Environmental Robustness
Substation control houses typically are not environmentally controlled spaces. There is often a minimum of heating (perhaps to 50°F) and no cooling. There is also the possibility that the control house may be without power. During this time, the station battery maintains protection operation and other essential functions, but does not provide a backup electrical source for heating and cooling.
Office-grade network equipment including transceivers, hubs, and switches are often unsuited to environments without adequate heating and cooling. The mission-critical nature of protective relays has led to several environmental requirements including RFI, ESD, operating temperature, and vibration. As such, you should carefully evaluate whether Ethernet is a mission-critical component of your substation and select equipment accordingly.
Engineering Access
One of the places where Ethernet networks can be the most helpful is for engineering access to station IEDs. There are three primary reasons that engineers communicate with relays:
It is possible to connect Ethernet networks so that relays in the substation become accessible from desktop engineering workstations in the central office. This type of architecture must be implemented with care, as there are numerous system administration and security issues that require attention.
However, it is vital that you address network security issues in designing any system that allows access to station IEDs from outside the substation; otherwise, this could become a path that would allow either inside (an employee) or outside (from the Internet) access directly to mission-critical protective devices.
SEL produces protective relays and instrumentation products for utility and industry that monitor power systems, collect equipment and system operation data, and initiate system protection operations. SEL also produces innovative communications products for use with SEL equipment as well as communications interfaces for many SEL products. SEL Ethernet products include communications cards for protective relays and Communications Processors, a serial-to-Ethernet converter, and integral Ethernet interfaces in several products under development. For more information on SEL products and services, including Ethernet solutions, visit www.selinc.com.
Biography
Darold Woodward has a B.S. in Electrical Engineering from Washington State University. He is a member of the Instrument Society of America (ISA). He joined Schweitzer Engineering Laboratories in 1998 in the position of System Integration Engineer. He was with the consulting firm HDR Inc., for six years where he participated in design and commissioning projects for electrical, automation, and instrumentation systems in water, wastewater, and hydroelectric facilities. Before joining HDR Inc., he was with R. W. Beck and Associates assisting with the design of electrical and instrumentation systems for substations, wastewater, and hydroelectric facilities.
Ethernet networks were not developed specifically for operation in substations and other harsh environments. So, why is there so much interest in applying Ethernet networks in these locations? The answer is similar to why personal computers are now used in many industrial and power system applications. Ethernet is so popular in other applications that it is simpler to employ and enhance Ethernet than to create something new.
Only 15 years ago, most Human Machine Interfaces (HMIs) operated on dedicated mainframe computers with terminals rather than the legion of personal computers that is used today. Early personal computer HMIs used custom operating systems dedicated to HMI operation. While dedicated systems are more stable and reliable, today’s systems often cost from 10 to 1 percent of the expense of dedicated singlepurpose systems.
Both industrial and utility networking experts are moving forward accepting the limitations of Ethernet networks and solving the problems associated with Ethernet networks. Advances in computing power and network technology allow us to take advantage of the popularity and availability of Ethernet networking equipment and solutions.
Ethernet Physical and Data Link Layers
Each standard physical layer and corresponding data link layer has a designator (e.g., 10BASE-T) that identifies the layer specifications. The most popular physical and data link layer combinations for Local Area Networks (LANs) within a single building are fiber optics (10BASE-FL and 100BASE-FX) and twisted-pair metallic (10BASE-T and 100BASE-TX). For general use networks, 10 Mbps and 100 Mbps are the most popular data transmission speeds. As data demands on networks have increased, 100 Mbps networks have become more popular. The actual network loading for a given offered load is about 10 times lower on 100 Mbps networks than on 10 Mbps networks, reducing collisions and network latencies.
The first generation of Ethernet networks used coaxial cable wired in multidrop topologies that were expensive and difficult to modify after installation. The present generation of Ethernet standards uses twisted-pair wiring or fiber optic cables in a star network topology. A star network topology employs a central node (hub or switch) to connect the individual network segments. The star network is more robust and less expensive to install and modify than multidrop network topologies.
Improvements on the hub including switches and routers enhance network performance. Twisted-pair network standards allow unshielded twisted-pair wiring (UTP) similar to that required for telephones. Sites can be prewired using multiple cables of the same type to each anticipated node location. The cables can then be used for telephone or network operation as required, increasing the flexibility and decreasing the cost of twisted-pair networks.
Unfortunately, while UTP cables are inexpensive and simple to apply in an office environment, they can be a problem in other environments. For example, offices typically do not contain strong sources of radio frequency interference (RFI) because of the shielding provided by office buildings and the lack of strong RFI sources within the building. Utility and industrial installations, however, often contain strong internal sources (hand-held radios, variable frequency drives, welders, etc.) and may not be well shielded from external sources including radio transmitter towers. Substations also include control wiring that typically is not shielded and can induce voltages in adjacent wiring.
Also, it is critical to note that you should never run any type of metallic communications cable from the substation control house to equipment in the substation yard. The ground potential differences experienced during a fault in the substation can subject equipment to damaging voltages and currents, especially in cables with shields that are grounded at both ends.
In a substation control house, many of these risks are decreased, but the noise from control and instrumentation circuits and circuit breakers in metal-enclosed switchgear can disrupt networks. Even with shielding and physical separation from other cables and wiring, metallic cables with shields provide paths for current to flow from ground potential differences, DC faults, and other stray electricity. The only sure protection from these problems is a cable system that is unaffected by electrical and electromagnetic interference.
Fiber-Optic Cable
Fiber-optic cable systems provide two principal benefits. First, the signals within fiber-optic cables are immune to RFI and electrostatic interference that can disrupt communication on metallic cables. Second, fiber-optic cables can have an all-dielectric (nonconducting) construction. This means that you can run fiber-optic cables outside of the control house to provide robust and reliable communication without the threat of damaging critical equipment at the ends of the communications circuit.
Where twisted-pair wiring uses two pairs, one for transmit and one for receive, fiber-optic cable systems use a pair of fibers, one for transmit and one for receive. Fiber-optic cable systems also require a central node or hub that combines the point-to-point fiber-optic cable segments into one logical network.
The most popular fiber-optic Ethernet network standards are 10BASE-FL and 100BASE-FX, 10 Mbps and 100 Mbps, respectively. Fiber-optic cable is more expensive than metallic cable. However, shielding measures for twisted-pair cables and installation labor are significant construction costs. You must consider the installed cost of the network to properly evaluate the impact of your choice of network physical medium.
Industrial Protocols
Several of the most popular industrial integration protocols are either running over Ethernet networks or are being prepared for operation over Ethernet networks. For example, Modbus TCP is Modbus for use over TCP/IP networks. Other industrial protocols, including ControlNet, Profibus, and Foundation Fieldbus, are migrating to Ethernet networks. It is important to consider this work when you plan to use Ethernet in power system applications. The office environment has made technologies inexpensive and available for industrial applications and in cases where officegrade equipment is not adequately robust or rugged. Office-grade equipment provides a foundation for the understanding and development of appropriate industrial Ethernet network components.
Ethernet Network Topology
It is very popular to characterize Ethernet networks (and other substation automation networks) as a magic bus that connects all devices and solves all problems. However, the magic bus concept is only partially correct, even in a logical sense. With an Ethernet network that contains a hub, the network is actually wired as a star that functions logically as a bus. If a switch is the central node, each segment between a switch and a node operates independently with the switch buffering and directing traffic to reduce collisions and decrease message transmission delays.
You may want to connect your network as a simple star topology with a switch or hub, but there are several additional considerations. External access from an engineering network requires an entry point to the substation LAN, typically through a router. You should consider whether engineering access and potentially mission-critical data should travel over the same network segment.
Engineering networks are also often connected to corporate networks and ultimately to the Internet. If you connect a mission-critical LAN to the engineering network, you have provided a path (if a hacker defeats security measures) from the Internet to your mission-critical substation LAN. A simple denial-of-service attack within your corporate LAN could jeopardize the substation LAN.
Environmental Robustness
Substation control houses typically are not environmentally controlled spaces. There is often a minimum of heating (perhaps to 50°F) and no cooling. There is also the possibility that the control house may be without power. During this time, the station battery maintains protection operation and other essential functions, but does not provide a backup electrical source for heating and cooling.
Office-grade network equipment including transceivers, hubs, and switches are often unsuited to environments without adequate heating and cooling. The mission-critical nature of protective relays has led to several environmental requirements including RFI, ESD, operating temperature, and vibration. As such, you should carefully evaluate whether Ethernet is a mission-critical component of your substation and select equipment accordingly.
Engineering Access
One of the places where Ethernet networks can be the most helpful is for engineering access to station IEDs. There are three primary reasons that engineers communicate with relays:
- Communicate directly for diagnostics and status information;
- Retrieve file-based data including oscillography and SER reports;
- Manage and manipulate relay settings.
It is possible to connect Ethernet networks so that relays in the substation become accessible from desktop engineering workstations in the central office. This type of architecture must be implemented with care, as there are numerous system administration and security issues that require attention.
However, it is vital that you address network security issues in designing any system that allows access to station IEDs from outside the substation; otherwise, this could become a path that would allow either inside (an employee) or outside (from the Internet) access directly to mission-critical protective devices.
SEL produces protective relays and instrumentation products for utility and industry that monitor power systems, collect equipment and system operation data, and initiate system protection operations. SEL also produces innovative communications products for use with SEL equipment as well as communications interfaces for many SEL products. SEL Ethernet products include communications cards for protective relays and Communications Processors, a serial-to-Ethernet converter, and integral Ethernet interfaces in several products under development. For more information on SEL products and services, including Ethernet solutions, visit www.selinc.com.
Biography
Darold Woodward has a B.S. in Electrical Engineering from Washington State University. He is a member of the Instrument Society of America (ISA). He joined Schweitzer Engineering Laboratories in 1998 in the position of System Integration Engineer. He was with the consulting firm HDR Inc., for six years where he participated in design and commissioning projects for electrical, automation, and instrumentation systems in water, wastewater, and hydroelectric facilities. Before joining HDR Inc., he was with R. W. Beck and Associates assisting with the design of electrical and instrumentation systems for substations, wastewater, and hydroelectric facilities.
