Building on part 1 of this article in the previous issue, this second installment illustrates the issues and impacts of implementing substation integration in harmony with the utility’s organization and operations using a standards-based approach. The engineering process implemented by Black & Veatch at a Northeastern U.S. utility is presented here in case study fashion. This project began in 2005 with the addition of a new substation.
The utility wanted to move their substation design from a mix of electromechanical relays and other IEDs to an integrated substation that supports automation. However, they were not starting with a blank sheet of paper. They were already extending their corporate WAN to their substations, so they knew they wanted Ethernet; but how do you install Ethernet in substations? They knew what microprocessor relays they wanted, but what protocol was the best choice of the three supported protocols (DNP3, Modbus, and IEC 61850)?
Recognizing the need from previous experience, the utility began with substation integration training to introduce personnel from across the enterprise to the equipment, concepts and issues, reasons, costs, risks, benefits, and process of substation integration. Once the initial training was completed, functional requirements for performance were defined as outlined in the first article, and the process was under way.
Input and Outputs
Substation inputs and outputs (I/O) include measurements, status, and control. By defining an I/O scheme that shows the substation data sources (See Figure 1), several underlying issues were uncovered and resolved.
Figure 1 – Example I/O Scheme for 15 kV Breakers
On the distribution side, metering data from relays was considered accurate enough. Revenue meters were already required on two of three transmission lines and both transformers. For the remaining transmission line, it was discovered that metering values from the primary relay were accurate enough. While metering values could have been obtained from primary and backup devices, the utility decided that multiple sources added unnecessary complexity to system design and operation.
Besides standard metering quantities, other examples of analog quantities gathered from other substation IEDs are tap position, battery DC voltage, control house temperature, transformer dissolved hydrogen level, and transformer winding temperature.
For wiring the approximately 270 hard-wired status points, brief consideration was given to wiring them to relays. Significant concern was raised because many status points are not associated with relays and the status of equipment still in service would be lost when an unassociated relay was taken out of service. By hardwiring breaker status and other status points to an IED dedicated to that purpose, the source of status and controls is readily known and easily isolated. For these reasons, a distributed I/O device was selected for status inputs.
Initial discussions surrounding controls had the SCADA controls assigned to the primary relays and local HMI controls assigned to the backup relay. In addition, the utility wanted a manual way to perform controls outside of the relays and to also manually disable controls. Performing controls through relays resulted in the difficult association of some control points with relays and the loss of control when a relay is out of service. By using dedicated distributed I/O, the utility was able to directly trip the breaker without the use of interposing relays, but careful consideration of contact output ratings was required (See Figure 2).
Figure 2 – Comparison of Distributed I/O Control Output Ratings
Protection and Time Synchronization
IEC 61850 was viewed by the utility as important for the future, but not required for immediate implementation. Although it was decided that protection functions would continue to be accomplished through traditional hard-wiring to relays initially, it was also decided that a network architecture capable of easy migration to IEC 61850 in the future was required.
Time synchronization was also considered important. IEDs are synchronized using the most accurate method supported by each IED: IRIG-B, NTP/SNTP, and DNP. All three time synchronization methods were implemented in the system, depending upon how each IED vendor supports time synchronization. Some IEDs did not support any of these standards, however.
IRIG-B calculations were performed during the initial design to ensure proper signal levels. It was discovered that the load from the backup relays and the distributed I/O devices required the IRIG-B network to be split up, and a high-drive output was needed on the satellite clock. The high-drive output requirement made it impossible, however, to include a timeserver on the satellite clock.
The substation computer was connected to IRIG-B and configured as a network timeserver that is ± 250 milliseconds, relative to the IRIG-B source. All data received in the HMI on the substation computer is either time stamped at the IEDs or at the data concentrator to ± 1 millisecond accuracy.
Bringing IRIG-B to the substation yard using copper cables impacts performance and may not be possible because the copper cable runs are too long for reliable operation. Distributed I/O devices located in the substation yard wer e connected using a fiber optic port achieving much better isolation and eliminating the distance constraints of copper cables. The same fiber optic cable used for the LAN connection to the IEDs in the substation yard was also used for distributing IRIG-B to the switchyard.
Programmed Logic and Ancillary Services
Programmed logic is the basis for substation automation. Supporting system-wide programmed logic functionality requires a high-speed peer-to-peer communication network. While IEC 61850 supports this requirement using high-speed messages with guaranteed performance requirements, it is also possible to use DNP3 over Ethernet for non-protective functions.
This is accomplished by IEDs broadcasting data to multiple masters or multiple masters polling the same slaves. In this application, this type of DNP3 functionality was a differentiator in the selection of distributed I/O devices and the RTU/data concentrator. How programmed logic is implemented in an IED can also vary. While most data concentrator vendors support an embedded IEC 61131-3 PLC engine, others only support a text-based or object-based programming language.
IED Selection
IED selection is always a challenging and time-consuming process because there are many IEDs available with various issues impacting the final selection. As with many utilities, this utility had already selected most of the IEDs before system design even began. Most importantly, they did not want any risk of “vaporware” in unproven products.
The utility was planning on using their standard SCADA RTU, but significant performance drawbacks were identified with this RTU related to the integrated system. This resulted in considering other RTUs and data concentrators from various suppliers. A data concentrator was ultimately selected based on how well the IED met the evaluation criteria.
Because a traditional RTU was not provided and the I/O capability of the relays was not being used, a distributed I/O device was required in the design. A distributed I/O device was selected based upon superior flexibility. However, a media converter was required for the Ethernet port to make the device fully compatible with the broader design approach.
The utility initially wanted to use their SCADA master software in the substation as the HMI. One reason was that the utility wanted the tags placed at either end replicated at the other end. After some detailed technical review with the vendor, the utility was not able to use the existing SCADA master software as the substation HMI. Thus, an evaluation of several HMI packages from various industries was undertaken to find a suitable alternative. Software from a well-established source was eventually selected that offered a number of significant performance advantages.
Substation Computer
The utility knew that they wanted a substation-hardened computer that met IEEE Standard 1613. This was a contributing factor to the SCADA master HMI not being used in the substation because of memory and operating system requirements. One useful feature was that the substation computer would easily accept an IRIG-B input and could act as an NTP/SNTP time-server.
Security
Like many utilities, this utility needed to address NERC CIP (Critical Infrastructure Protection) requirements that went beyond the new substation but were not addressed by this project. Even so, the physical perimeter is monitored. Cyber security issues are addressed by providing substation LAN equipment that includes features that help address NERC CIP requirements and implement best practices that provide a “defense-in-depth.” The general approach was to physically separate the substation network from the corporate network. The substation data concentrator could also be used for access control to the substation IEDs and monitoring and logging all IED connections.
Architecture Selection
The architecture selection was based on Ethernet for four primary reasons. First, the utility had already extended their private corporate WAN and SCADA WAN to the substations, and they wanted to better utilize these networks. Second, several Ethernet-based relays were installed previously, and the utility wanted to continue using them due to their ease of use and network support. Third, the client wanted more than just SCADA data out of their investment in IEDs, including remote access and non-operational data. All of this data and functionality has value, but it requires a medium that can support significant bandwidth requirements and multiple simultaneous connections. Finally, a system architecture that could easily support IEC 61850 was desired.
While some selected IEDs did not support Ethernet, the decision was made to preserve the architecture so that serial devices were connected to substation hardened serial device servers. This worked well because the serial devices were usually located near each other.
Redundancy was not selected because the substation LAN is not initially supporting protection functions and IEEE 1613 compliant switches and routers are used along with fiber optic cables. Issues with redundant LANs include the increased cost for a LAN that is almost completely isolated inside the control house and segmenting the architecture since some IEDs do not support redundant LANs.
Protocol Selection
There was a strong desire to use DNP3 in the substation. IEC 61850 was discussed, but it was decided that using IEC 61850 might be too much to take on at the present time. In addition, the only devices that would have supported IEC 61850 directly were the relays, data concentrator, and HMI. By contrast, the one protocol almost all selected IEDs supported was DNP3.
Performance
System availability was addressed by using substation hardened equipment in all aspects of system design. All Ethernet equipment meets IEEE 1613, including the substation computer. Any IEDs that did not meet IEEE 1613 (e.g., meters, distributed I/O) were connected to a media converter. In addition, fiber optic cables are used to completely isolate the system cables from any interference. The only equipment not meeting IEEE 1613 requirements was the monitor for the substation computer, which was isolated behind a DC-DC converter designed to meet IEEE C37.90.
System changeability includes ease of expansion, provision of spare capacity, ease of replacement, and ease of maintenance. The system architecture was based on Ethernet and is inherently modular and easily scalable for expansion. By using distributed I/O, spare capacity was easily provided and can be easily added, although there is already a rather large upper limit in the number of I/O points and devices the data concentrator can effectively handle.
Because of the modular nature of the system, replacement of portions of the system can be accomplished without removing the whole system from service. System maintenance considerations included the ability to change operational parameters as well as the configuration. With a modular system being supported by multiple vendors, changes to some IEDs are easier than others, depending upon the evaluation criteria. Generally speaking, however, all selected devices support good user interfaces that are easy to work with to implement system changes.
Today
In early 2007, the conceptual design was finished and detailed design was nearing completion. This was followed by a period of validation, during which the system components were programmed and tested in a laboratory setting. Today, the project is nearing completion and will be energized soon. The internal team has received approval to build an automation test center to be used for training, testing, and provide a development environment for future automation programs. Notably, this same system design process is already being applied to another new substation.
About the Author
Craig Preuss is the Engineering Manager for Utility Automation at Black & Veatch Corporation where he is involved in virtually all facets of substation integration and automation. Craig earned his bachelor’s degree in electrical engineering from Valparaiso University in Valparaiso, IN and a master’s degree in power systems from the Illinois Institute of Technology and is a registered professional engineer in the states of Illinois and Washington.
During his 18-year career in the utility engineering and automation field he has authored several papers, presentations, and articles on topics dealing with substation integration and automation. Craig is an active member of the IEEE where he serves as the new working group chair for IEEE Standard C37.1. He was also involved in the writing of IEEE Standard 1615 and IEEE Standard 1686 as well as participating in other IEEE working groups. He is also a member of the ISA (Instrumentation, Systems, and Automation Society).
