US Govt warns Russia attacking US energy sector, renewable industry responds with cybersecurity working group

Today the FBI and the US-CERT (United States Cybersecurity Emergency Readiness Team) issued a warning that Russian Government hacking groups are not only attacking US energy sector facilities, but that they are INSIDE the systems. That is, the attackers began by infiltrating networks of 3rd party organizations who had lax security, and then scouted around to find a means to laterally enter higher-profile networks including at utility companies. The result is that it appears these attackers can now shut down segments of the US electricity grid at will, and they’re simply waiting for the political command to do so.

Complicating matters is the fact that the US electrical grid, specifically the control systems, have lax security. The utility companies use communications protocols designed in the 1980’s, with weak security systems, and rely on security-through-obscurity (namely, the use of private networks) to give a degree of security. But, security-through-obscurity never works, and in the Internet age private networks simply are not private enough to be trusted.

I can say with certainty – because I’m peripherally involved – that at least the Renewable Energy sector is taking action to beef up cybersecurity of renewable energy equipment. I’ll get to that in a minute, but first a little bit of context setting.

What will the Trump Administration do?

Over the last couple weeks senior US counter-intelligence officials have announced they were resigning because the Trump Administration has not allowed them to counter Russian attacks on the USA. At the time the reports were of attacks on the election system, and the Social Media Warfare attacks using twitter botnets and the like. At the time this warning about attacks on energy sector facilities had not been made public, but those officials certainly will have known of this imminent warning.

Bottom line – the Trump Administration and the Republican Party have been soft-pedaling all allegations of dangers from Russia, despite the clear evidence that Russia is attacking not only the USA but a wide swath of countries in Europe. These attacks are not the traditional military-on-military action one might expect, but are being conducted over the Internet, and are attacking truth itself.

It’s vitally important for US Citizens to be considering whether the Trump Administration has been compromised by a foreign power, and whether they are foreign agents. That is no longer a matter of wild half-assed conspiracy theory nonsense – that question is being seriously bandied about in the news, and there is plenty of evidence suggesting that might be the case. One way to measure that allegation will be the response by the Administration to this threat on the energy infrastructure.

The FBI/CERT Warning

This attack on the energy sector is a different beast, however. The CERT/FBI warning spells out a long list of attack types.

This campaign comprises two distinct categories of victims: staging and intended targets. The initial victims are peripheral organizations such as trusted third-party suppliers with less secure networks, referred to as “staging targets” throughout this alert. The threat actors used the staging targets’ networks as pivot points and malware repositories when targeting their final intended victims. NCCIC and FBI judge the ultimate objective of the actors is to compromise organizational networks, also referred to as the “intended target.”

As I said above, the attackers are finding weaknesses in 3rd party supplier networks. Having established a foothold, they then launch attacks into other networks of higher value.

Some techniques being employed are:

spear-phishing emails (from compromised legitimate account),
watering-hole domains,
credential gathering,
open-source and network reconnaissance,
host-based exploitation, and
targeting industrial control system (ICS) infrastructure.

The alert document then goes on to describe a multi-stage infiltration plan that results in getting to the core systems.

At this point we need to segue into a little background information.

Utility company control systems

The last two years I’ve gone back to writing software for my living, and have worked in the Solar industry. The first job I worked on a system for reporting solar array performance. The software was about writing database queries about kilowatt-hours of solar production. That was extremely cool to be able to do that, and then the company assigned me to research the plans for integrating renewable energy assets into the electric grid in a big way. That led me to learn (at a high level) about the control systems at utility companies.

These control systems are really old technology. Utility companies began implementing remote control systems in the 1950’s or earlier. You might be aware that was well before any kind of computer capabilities existed, and therefore the earliest control systems would have involved analog systems.

What’s generally used now is what’s called a supervisory control and data acquisition (SCADA) system. SCADA is a general term for systems that can be implemented by a variety of technologies. The two most common (as far as I know) are:

MODBUS – This was developed in the 1970’s, and is similar to old-school serial communication lines. Using MODBUS one can query or set the value of so-called registers. MODBUS registers correspond to measurements emitted by equipment, or else act to control the behavior of equipment.
DNP3 – Distributed Network Protocol 3, a.k.a. IEEE 1815, is a higher level communications protocol for communicating control commands and data between remote terminals and control terminals.

DNP3 is generally used for wide area control systems, while MODBUS is only suitable for local control. For example a DNP3 box would be installed at a site, and then be connected to a variety of equipment using MODBUS. The box will send/receive DNP3 commands to monitor or control that equipment.

The key thing here is both these protocols were designed in the 1970’s and 1980’s. That’s long before the word cybersecurity had been invented, and certainly long before anyone thought much about computer security. As a result both of these protocols have essentially zero security. MODBUS has no security at all – meaning that anyone who can get software access to the MODBUS ports on devices can read any value, and make any configuration change. For DNP3 there is a modicum of security …

The utility companies currently run their SCADA systems over so-called “private networks”. Supposedly a private network cannot be accessed, and therefore it’s safe to run an unsafe protocol on a network that cannot be accessed.

The problem is as the Internet connects to everything it’s increasingly rare to have a fully private network.

This diagram was published by the FBI/CERT as part of the proof that Russian attackers had infiltrated core systems at electricity utilities. So … what is this diagram?

It’s a screen shot of what’s known as a Human-Machine-Interface (HMI) for a SCADA system. You can see it’s a diagram of a turbine/generator system presumably at an electric power plant. There is a bunch of status information about the components, not just the turbine and the generator but cooling systems.

Such a screen is typically touch-sensitive and operators will control the system from this HMI screen. Or else, that there will be knobs and other controls on the control panel next to this display.

The most important thing is – access to this display means having access to the control system on the supposedly private network.

Cybersecurity and the Renewable Energy Grid

Now we can finally get to why this story belongs on LongTailPipe.

Renewable electricity production in the USA (and elsewhere) is growing rapidly. That’s especially true in California and Hawaii. I’ve heard California has half the installed solar capacity in the USA. In the case of Hawaii, they’ve traditionally powered their electricity grid using imported gasoline or diesel, and desperately want to move quickly to be fully solar and wind powered.

This means that – starting in California and Hawaii – renewable energy assets must transition from being play-things at the side of the electricity grid, to becoming a full partner in the electricity grid. The phrase is that they must become “dispatchable assets”, which I think means that utility companies must be able to dispatch commands at any time to renewable energy assets.

What’s happening is that stakeholders – the utility companies, the regulatory agencies, the equipment suppliers, and the companies offering various services – are designing a distributed control system for the electricity grid. What must happen is a transition from a centrally controlled electricity grid, to one where assets from a variety of companies are working together to support the electricity grid, using a variety of assets.

There are a variety of such assets — collectively known as Distributed Energy Resources, or DER — including solar arrays, wind turbines, energy storage units, smart thermostats, hot water heaters, pool pumps, electric car charging stations, as well as traditional assets like central generation plants, hydroelectric facilities, and pumped energy storage systems.

Various working groups convened by these stakeholders are working on new control systems for distributed control of these assets. Since the government does not have the power to dictate specific communication technologies – there is an inevitable period of questioning which is the best communication protocol and so forth.

One firm result is that California and Hawaii are mandating that solar inverters and related equipment must have “communication” capabilities, which means the ability to remotely monitor and control the equipment.

It’s also expected that the control system will operate over the Internet rather than over private communication networks. In order to have a distributed system made from multiple competing companies, the control system needs to run over neutral territory rather than over a private network.

Which – GULP – means – in the face of attacks on the electricity grid systems – how can we run these systems over the public Internet without compromising security? Neither MODBUS nor DNP3 will cut it.

The Cybersecurity Working Group

What I know of, because I am directly involved (at the periphery), is the SunSpec Cybersecurity Working Group.

SunSpec is a non-profit industry consortium which has acted to speed up technology development in the Solar industry. That is, they’ve spent years working with companies in the solar industry to get common communications and data standards. That resulted in a standardized MODBUS map, and at a higher level SunSpec was able to influence various national (IEEE and UL) and international (IEC) standards to develop a common data model for solar inverters, energy storage units and other equipment.

The Cybersecurity Working Group is following a similar model — gathering industry experts to hash out issues — while focusing on the issues of cybersecurity best practices for distributed energy resource control systems.

The current work is gathering requirements and best practices distilled from related industries. These sorts of cybersecurity considerations are already well known, because plenty of companies are running secure systems over the Internet at a large scale. They’re working fairly quickly on producing documents with the gathered information.

The next step is to bring the findings to standards organizations and influence the development of standards for interconnecting equipment on the electricity grid, and distributed control of the equipment.

These same people have already been working on similar efforts over the last few years. IEEE 2030.5 is a fairly new communication protocol for controlling distributed energy resource equipment over the Internet. It includes a number of advanced Internet-centric practices like using a public-key-infrastructure for SSL certificates that are used to authenticate DER equipment, and using HTTPS with high-grade encryption. The protocol was approved about 2 years ago, and had lots of input from not just SunSpec but the California Public Utilities Commission and its Smart Inverter Working Group.

This is good work that’s underway. I’ve participated in the working group calls for a couple months, and am impressed with the level of detail they’re going through.

When will the solution arrive?

BUT – this particular effort is a long ways from making a significant impact on electrical grid cybersecurity. There is a clear and present danger active RIGHT NOW.

The SunSpec Cybersecurity Working Group process is at the beginning stages of developing technology requirements that will eventually roll out as best practices for the electricity grid. The process can move quickly, but in this case “quickly” means 3-5 years or more.

An issue is that the electricity companies have an incumbent system that’s paid for by ratepayers. Typical rules under which utility companies operate mean ratepayer-paid-assets cannot be replaced before the utilities commissions allow them to do so. The reason has to do with how electricity rates are computed, and the amortization of utility company assets into electricity rates. In other words the existing SCADA systems with all their flaws are being amortized through electricity rates, and will eventually be replaced, but not until they companies are financially allowed to do so.

So, GULP, in the face of a clear and present danger, where geopolitical adversaries have compromised critical systems that are inherently insecure and cannot be replaced right away, …. what to do? Hoping they never pull the trigger won’t cut it.

About the Author
Latest Posts

About David Herron

David Herron is a writer and software engineer living in Silicon Valley. He primarily writes about electric vehicles, clean energy systems, climate change, peak oil and related issues. When not writing he indulges in software projects and is sometimes employed as a software engineer. David has written for sites like PlugInCars and TorqueNews, and worked for companies like Sun Microsystems and Yahoo.