When Bob Dylan sang “For the times they are a-changin'” in the 1960s, it was an ode to recognizing and accepting change. This same frame of mind is required when we think of securing electric utilities from modern cyber threats.
Up until the last decade most disruptions in power grids were not caused by cyberattacks. For example, the Northeast blackout of 2003 was caused by a series of inter-related events. As we all know, a generating plant near Cleveland, Ohio went offline during a time of high demand, putting strain on high-voltage power lines, which led to a cascading effect. Other factors such as overgrown trees and a bug in the software used to manage the energy system for one utility contributed to the blackout. The result was that one hundred power plants were shut down and about 55 million people in the U.S. and Canada went without power for 8 to 16 hours.
A notable blackout that was likely caused by a cyberattack occurred in Brazil in 2007. It resulted in outages that left 60 million people across multiple cities without power. Not many details are known about the attack, likely because Brazil, as well as other nations, did not want the hacking community to know that their electric grid is vulnerable to such attacks.
Despite the Brazilian incident, overall the risk of an external malicious cyberattack on a power utility was considered minimal. However, over the last decade utilities have been moving to smart grid infrastructure, which introduces many points of cyber vulnerability, at the same time as worldwide cyberattacks on energy systems have increased.
Hackers and nation-states focus cyberattacks on energy providers
Focused control system cyberattacks, like the Brazilian one, reached a zenith in July 2010 when the Stuxnet malware was discovered destroying the centrifuges used for uranium enrichment at Iran’s Natanz nuclear facility, proving that industrial sabotage by malware is possible.
Stuxnet’s fame made it all but impossible for governments or other entities to keep the vulnerability of electric grids off hackers’ radar. Once the media started publishing Stuxnet stories (media as varied as ‘60 Minutes’ and ‘Vanity Fair’ have covered it), security researchers started focusing on hacking industrial systems and devices, particularly those in the energy sector.
Stuxnet’s design also provided the hacking community with a toolkit for developing a type of sophisticated malware known as APTs (advanced persistent threats). Like earlier malware, APTs are designed to be very hard to detect, so they can do their dirty work over a period for years.
Flood-gate at a hydro dam in South Africa
The combined impact of Stuxnet and other APTs was called out in a 2011 study done by McAfee, the computer security firm, and the Center for Strategy and International Studies in Washington. It surveyed utility industry executives from 14 nations and noted:
“One of the more startling results of our research is the discovery of the constant probing and assault faced by these crucial utility networks. Some electric companies report thousands of probes every month. Our survey data lend support to anecdotal reporting that militaries in several countries have done reconnaissance and planning for cyberattacks on other nations’ power grids, mapping the underlying network infrastructure and locating vulnerabilities for future attack.”
Vulnerability deluge
Further evidence that a tipping point in cyber landscape for power systems occurred in 2011 when more ICS (industrial control system) vulnerabilities were made public (many with exploit code available on the internet), than in the entire previous decade, and the trend has continued well into 2012. Power utilities had exposure in relation to many of the problems reported.
Fig 1. ICS Specific Vulnerabilities in the Public 2001-2011 (by quarter). Chart used with permission of creator, Sean McBride.
For example, recently Telvent, the smart grid giant owned by Schneider Electric, reported that hackers breached its network, left behind malicious software and accessed project files for its OASyS SCADA system. This system is used by Telvent to control power grids, gas pipelines, and industrial controls around the world.
This attack is another milestone in escalating cyber threats, as it was not just an example coded by a security researcher but a real attack with serious implications. The hackers, which clues indicate are a Chinese group, might be able to learn how the Telvent OASyS system works, and use that information for further attacks.
Another development is that detailed descriptions of SCADA system vulnerabilities have started to be widely published. A prime example of this was the January 2012 release of cyberattack tools for the GE D20 RTU. The ICS-CERT reports that 40 percent of the publicly disclosed vulnerabilities include working attack software.
Shamoon: Low-grade malware with high-grade impact
Another example of the changed cyber threat landscape occurred in August 2012 when an insider, possibly with Iranian backing, successfully attacked Saudi Aramco, and Qatar’s RasGas with the Shamoon malware. At Saudi Aramco, it destroyed the data on a huge number of computers (30,000 – 55,000) at their world headquarters facility.
Leon Panetta, the U.S. Defense Secretary recently described Shamoon as the most destructive attack the business sector has seen to date and a ‘significant escalation of the cyber threat.’
A disturbing aspect of Shamoon is that it lowers the bar for effective disruption of a business or agency. A programming effort that involved a small group of people with skills slightly better than amateurs was able to penetrate a well-protected network and destroy massive amounts of data (albeit with insider access). In addition, they did it at a scale and speed that is unprecedented.
For utilities that are linking up legacy systems to IT technology and have to contend with millions of endpoints such as smart meters and grid controls, the challenge of securing the system has never been greater. Imagine the damage that could be done if any group of people with an axe to grind against your utility activates a similar attack against you. The success of Shamoon is sure to attract copycats.
By Eric Byres, Tofino Security at Belden Inc.
A large utility had an agreement to buy and sell power with several industrial customers. This required real-time communications between the utility substations and 3rd party facilities for power quality metering. The utility owned the metering equipment, but it sat in another company’s premises, making the definition of an ESP fuzzy.
If inside the ESP, demands for physical isolation and auditing would be unrealistic. If outside, the traffic needed to be routed through the ESP firewall, adding complexity and risk. There was also the issue of how to manage any connectivity supported by 3rd party network elements. Fine-grained control of SCADA traffic between the customer and the utility cannot be assured by a standard IT firewall, which can only deny the SCADA protocol in use, and not filter it for specific content. This potentially allows dangerous functions, such as programming commands or malformed SCADA messages, to be injected from the client site into the utility’s control systems as part of the permitted data stream.
The solution was to define the meters in a separate zone that had low security capability. The conduits between the 3rd parties’ facilities and the utility needed three attributes in order to provide adequate security— encryption of messages, authentication of the end points and fine-grained inspection and filtering of all SCADA messages.
Utilities need to secure all communications with other entities in the grid to ensure protection.
A firewall on each conduit, with deep packet inspection (DPI) and VPN encryption modules, allowed SCADA data read commands (and corresponding reply messages) and blocked all other traffic. The VPN authenticated all connections and encrypted all traffic. The security devices also provided alerts for blocked SCADA messages or suspicious connection attempts.
Bottom Line: Prepare for the attacks
Whether it is a highly sophisticated, targeted APT attack or a low grade Shamoon-like attack, and whether it is motivated by a political agenda or activist groups, the cyber threat landscape is in a new era. It’s time to renew your defenses to protect your power infrastructure.
In October 2012 the U.S. Defense Secretary Leon Panetta declared:
“….foreign cyber-actors are probing America’s critical infrastructure networks. They are targeting the computer control systems that operate chemical, electricity and water plants and transportation systems.”
He said the government knows of ‘specific instances where intruders have successfully gained access to these control systems, and that the intruders are trying to create advanced tools to attack the systems to cause panic, destruction and death.
Defense approach 1: NERC CIP compliance
Systems that transmit and distribute electricity have not been designed with cyber security as a priority. Instead, reliable continuous delivery of electricity has been the focus, and it was assumed that utility systems would stand apart, in locked industrial sites and control centers. Nowadays, as utilities implement the smart grid, many of the SCADA systems in use, particularly ones that have been in operation for 10 or 20 years or more, are vulnerable to exploitation.
The North American Electric Reliability Corporation (NERC) has been focused on Critical Infrastructure Protection (CIP) for a number of years, and the fifth version of their security compliance standards has recently passed the first stage of approvals. By law, bulk electric producers need to comply with NERC CIP regulations. In doing so you might assume that you are protecting your systems from the new wave of cyber attacks. Unfortunately that is not necessarily the case.
There are number of limitations of NERC CIP.
a. Focus on compliance versus effective security
When considering the effectiveness of a standard, a top concern is whether or not it reflects a well-thought-out design for security. In the case of NERC CIP, particularly the versions preceding version five, there is a high focus on compliance and less of a focus on effective security. The penalties imposed are about compliance (for example, not providing comprehensive audit logs) rather than low security effectiveness.
b. NERC CIP exemptions
Any asset that can be claimed as not critical to the Bulk Electric System (BES) is exempt in NERC CIP. Also, small generation facilities (defined as 500 MW in version three and 1,500 MW in version five) are exempt. The difficulty with this is that a security breach may allow the initiator to impact the main network once access is achieved at the ‘non-critical’ facility. As well, an attack on a few non-critical assets could be as equally as damaging to a company as an attack on a single critical asset.
c. Electronic Security Perimeter (ESP)
NERC CIP defines security in terms of having a strong boundary, i.e. the ESP, as the only layer of defense. This is a model known as the Bastion model and it has failed in military campaigns and in IT security. It will fail for electrical systems.
This doesn’t mean that boundary protection is not important: it is, but there needs to be additional layers of defense to back it up – what is known as Defense in Depth (DiD).
Let’s look at a couple of examples of boundary issues. People are one of them: they cross the boundary all the time and they are able to introduce malware inside of it.
Another example is cogeneration (cogen) facilities. If you define cogen as being outside the NERC CIP BES criteria, then it is outside the ESP. This then begs the question of how to secure the interaction between the cogen and the primary systems.
A different perimeter problem occurs when utilities communicate with their large customers. When information about usages needs to be exchanged, how do both sides insure that only the proper communications are exchanged?
Defense approach 2: Defense in Depth and best practices
If NERC CIP does not ensure security, what does? The answer is to use a number of best practices that contribute to providing DiD.
ISA IEC 62443
Instead of only relying on an ESP, production networks should be segmented according to ISA IEC 62443 standards (formerly ISA-99 standards). Each zone of devices with similar security requirements should be protected with its own industrial firewall. Such a firewall should be optimized to secure SCADA protocols such as DNP-3 and IEC 61850, rather than email or web traffic. Web and email messages simply have no place in the electrical system; products that inspect these protocols simply add cost and complexity to the security solution. Industrial firewalls also have an advantage over IT firewalls because they can be deployed into live networks by field staff with no risk to operations.
Seven steps to ICS and SCADA security
We have developed, in partnership with exida, our own best practices for ensuring good security, called the ‘7 Steps to ICS and SCADA Security.’ They apply to both industrial automation and power utility operations. We condense numerous industry standards and best practice documents and combine them with the experience of our staff. The result is an easy-to-follow process:
- Assess existing systems: Understand risk and prioritize vulnerabilities
- Document policies and procedures: Determine position regarding ICS and develop company-specific policies
- Train personnel and contractors: Develop and institute policy awareness and training programs
- Segment the control system network: Create distinct network segments and isolate critical parts of the system
- Control access to the system: Provide physical and logistical access controls
- Harden the components of the system: Lock down the functionality of components
- Monitor and maintain the system: Update antivirus signatures, install patches and monitor for suspicious activity
A white paper detailing this process is available at: http://web.tofinosecurity.com/download-7-steps/.
Focused efforts
Enterprise IT teams focus their efforts on their more important assets. Every control system has one or more assets that would seriously impact production, safety or the environment if successfully attacked. Often these are not the same as what the NERC-CIP regulation define as ‘critical.’ Your engineers know what really matters to the operation. If those assets are aggressively protected, the chance of a truly serious cyber incident is massively reduced.
Another area for focus is detection. The industrial automation world is poor at detecting anything unusual on controls networks, and this adds a level of vulnerability to power utility operations as well. Make sure your firewalls and other security devices have good reporting capabilities, and ideally are integrated into an industrial management system. Your production engineers and operators should be immediately alerted if a read-only remote operator station suddenly tries to program a remote terminal unit (RTU). Waiting for the IT team to analyze the event the next morning is too late.
For the times they are a-changin’
The golden years when power utilities needed minimal physical protection and SCADA systems were considered to be under the radar of the hacker community are gone. The question is when, not if, a power utility will come under cyberattack. The recent escalation in cyberattacks against energy companies and the warnings of the U.S. Defense Secretary indicate that now is the time to renew the cyber defense initiatives for your utility.
As Bob Dylan sang:
The order is
Rapidly fadin’
And the first one now
Will later be last
For the times they are a-changin’.
About the Author
Eric Byres, P. Eng. and ISA Fellow, is recognized as one of the world’s leading experts in the field of SCADA security. His technical knowledge combined with his background as a process controls engineer has been indispensable in his role as CTO and VP Engineering for Tofino Security at Belden Inc. He has written extensively about the malware Stuxnet. Eric founded the British Columbia Institute of Technology (BCIT) Critical Infrastructure Security Centre and shaped it into one of North America’s leading academic facilities in the area of SCADA cyber-security. As a result, Eric was awarded a SANS Institute security Leadership Award in 2006.
He is a notable contributor to industry standards and is Chair of the ISA99 Security Technologies Working Group and Chair of the ISA99 Cyber Threat Gap Analysis Task Group. Eric is Canadian representative for IEC TC65/WG13, a standards effort focussing on an international framework for the protection of process facilities from cyberattack.
