Welcome to this installment of Security Sessions, a regular feature focused on security-related issues, policies and technologies. In my work I deal extensively with the NERC-CIP standards. One of the things that has always bemused me is that although the focus of those standards is on protecting industrial automation systems (i.e., EMS/SCADA and now, generating plant DCS systems), the standards – despite now being in their third major revision – still draw almost exclusively from IT-focused methodologies. When the standards initially aimed their protection at EMS/SCADA systems, the deficiencies weren’t so noticeable, but now that generating plants and transmission substations are part of the mix, this limitation is quite problematic. In this column we’ll discuss some of the Industrial Automation issues that don’t (yet) seem to be very well addressed in the present standards – Tim
William T. (Tim) Shaw
PhD, CISSP
When the initial Urgent Action 1200 standard was issued by NERC following the 9-11 attacks, it was primarily focused on protecting Energy Control Centers (ECCs) and their associated SCADA/EMS systems. With the issuance of Standard 1300 (where it was inferential) and the subsequent conversion to the individual NERC-CIP standards (where it is explicit), the standards coverage was expanded to include transmission substations and generating facilities. But nothing was specifically added to address the differences between the automation technologies employed for those facilities versus those of the ECCs. The fact is that for the most part, generating plants employ Distributed Control Systems (DCS) and Safety Shutdown Systems rather than EMS/SCADA systems. Not to say that SCADA technology can’t be found in generating facilities, but the primary systems needing protection in the generation environment tend to be DCS systems.
DCS systems have been around since the late 1970s when Honeywell (and then EMC Controls) introduced the first generation. In the intervening years, many vendors entered (and exited) the market and extensive technological changes have been made to DCS systems during the decades since. The most recent DCS systems utilize high-speed, fiber-optic Ethernet LANs and IP communications, in place of the earlier, proprietary data highways. But a lot of those vintage proprietary systems still exist. Current systems use PCs, running Microsoft Windows, as operator workstations. But there are still a lot of old systems using Sun workstations and DEC VAXstations (now H-P), as their operator consoles and engineering workstations.
Most DCS systems also support “gateways” that allow instrumentation networks to be interfaced and integrated. These include standards such as Foundation Fieldbus, Profibus, HART, etc. and even certain PLC networks. There are also a lot of “smart” instruments that communicate using serial EIA RS-232 interfaces and MODBUS (or similar) protocols. Most DCS systems support serial ports and drivers for connecting to such instruments.
The possibility of attacking or compromising a DCS system, and taking down a plant, through an attack on an instrument bus or serial interface is a topic of debate, but it should be remembered that a control system reacts to the inputs it receives. If inputs can be manipulated to make it seem as if an unsafe condition existed, it might be possible to ‘trick’ the control system into triggering a unit trip. But the NERC CIP standards do not speak to the issue of the instrument bus or non-TCP/ IP networks. In a way this is understandable since these networks don’t exist in the IT world – or in the SCADA/EMS world where these standards originated.
In order to address the corporate desire to have vertical integration of information (presumably so that some executive “dashboard” filled with KPIs can be ‘automagically’ updated from plant data), most DCS suppliers have incorporated separate Ethernet/IP LANs – isolated from the controller networks – and added more gateways. These gateways may offer OPC connectivity to business and engineering applications and feed them plant information in a (one hopes) controlled manner. In large facilities, where it makes sense to have several independent DCS systems, it is still common to create a plant-wide LAN, with a connection to each DCS system, so as to enable having “supervisory” workstations that display plant-wide information.
Often this same plant LAN will interface with numerous independent subsystems such as continuous emissions monitoring, fuel metering skids, water treatment subsystems, turbine monitoring subsystems and many others. There might even be a plant-wide data historian attached to that upper level LAN. And of course, at some point the plant LAN will be interfaced to the corporate WAN and eventually, to the Internet. This multi-layered architecture and inter-layer gateway design leads to complications when trying to establish an electronic security perimeter (ESP). You can end up with either lots of ‘access points’ into that ESP or lots of equipment within the ESP that is subject to the various requirements of the CIPs. ISA Working Committee SP-99 has been looking at this issue and has been devising security strategies based on segmentation and layering of plant systems into “zones” with “conduits” established to control inter-zone communications. None of this work seems to have caught the attention of the people at NERC yet, however.
Today, we are also seeing a growing interest in the use of wireless technologies due to safety issues and the cost of running wire in an operating plant environment. Wireless often includes new wireless versions of instrument networks (e.g., wireless HART, vendor-specific wireless instruments and the new ISA-100 wireless standard). Plants are also installing wireless Ethernet mesh networks to support integrated voice, video and data communications. Some large plant facilities may also use point-to-point radio or microwave links to connect with outlying facilities. Securing wireless communications is a tough problem, especially with all of the incompatible wireless technologies available; most of which have no intrinsic security features or functions. So far, however, the NERC-CIP standards don’t specifically address the issues of wireless technology.
The point I am making with all of this is that large generating plant facilities do not easily conform to the simplistic notion of a single, centralized computer system that can be easily enclosed in a physical and electronic security perimeter. Large plant facilities also use a myriad of wired and wireless communications that may not be Ethernet and/or IP based and thus, not readily manageable or easily monitored using conventional IT strategies.
I mention the large installed base of legacy systems because of a general industry problem: staffing and expertise. The fact is, most electric utilities have reduced their engineering and support staffs to minimum levels over the past decade or so. A generating plant may have a couple of people on staff who can make simple repairs to a DCS system and even configure a new operator display or control loop. But most utilities are depending on their suppliers to provide support for things like implementing virus scanning or adding an intrusion detection package to their systems. On the other hand, the vendor community is also struggling because, in many cases, they too have made substantial staff reductions in recent years. Many of the older DCS systems are no longer under active support by their original suppliers, meaning that no further updates, patches, enhancements – or even bug fixes – will be forthcoming.
Some of those DCS vendors would have relied on their original equipment suppliers (e.g., Sun, DEC, H-P or others) for much of that support, and those suppliers no longer exist. Under these circumstances it is unlikely that a plant will be able to fully implement all of the requirements established in the CIP standards; at least not until the time when they can replace those legacy systems with newer and more current technologies. Until then, many will likely resort to tampering with these legacy systems, and because of the lack of the necessary technical expertise, may inadvertently cause unforeseen system failures and threaten plant safety and operational availability. So maybe it’s time to gather up the old-timers that originally built those legacy systems and assemble a task force to address those issues. (Hmm, better be careful what I wish for… that could be me!) – Tim
About the Author
William T. “Tim” Shaw (PhD, CISSP) has been active in industrial automation for more than 30 years and is the author of Computer Control of BATCH Processes and CYBERSECURITY for SCADA Systems. Tim has contributed to several other books and is a prolific writer and presenter on a range of technical topics. He is currently a senior security consultant for SecuriCon, an information security solutions firm, based in Alexandria, Virginia. Tim has been directly involved in the development of several DCS and SCADA system products and regularly teaches courses for ISA (International Society of Automation) on various topics. Inquiries or comments about this column may be directed to Tim at Tim@electricenergyonline.com.
