12 September 2019 | by Andy Greenberg | Wired.com
A fresh look at the 2016 blackout in Ukraine suggests that the cyberattack behind it was intended to cause far more damage.
For nearly three years, the December 2016 cyberattack on the Ukrainian power grid has presented a menacing puzzle. Two days before Christmas that year, Russian hackers planted a unique specimen of malware in the network of Ukraine’s national grid operator, Ukrenergo. Just before midnight, they used it to open every circuit breaker in a transmission station north of Kyiv. The result was one of the most dramatic attacks in Russia’s years-long cyberwar against its western neighbor, an unprecedented, automated blackout across a broad swath of Ukraine’s capital.
But an hour later, Ukrenergo’s operators were able to simply switch the power back on again. Which raised the question: Why would Russia’s hackers build a sophisticated cyberweapon and plant it in the heart of a nation’s power grid only to trigger a one-hour blackout?
A new theory offers a potential answer. Researchers at the industrial-control system cybersecurity firm Dragos have reconstructed a timeline of the 2016 blackout attackbased on a reexamination of the malware’s code and network logs pulled from Ukrenergo’s systems. They say that hackers intended not merely to cause a short-lived disruption of the Ukrainian grid but to inflict lasting damage that could have led to power outages for weeks or even months. That distinction would make the blackout malware one of only three pieces of code ever spotted in the wild aimed at not just disrupting physical equipment but destroying it, as Stuxnet did in Iran in 2009 and 2010 and the malware Triton was designed to do in a Saudi Arabian oil refinery in 2017.
In an insidious twist in the Ukrenergo case, Russia’s hackers apparently intended to trigger that destruction not at the time of the blackout itself but when grid operators turned the power back on, using the utility’s own recovery efforts against them.
“While this ended up being a direct disruptive event, the tools deployed and the sequence in which they were used strongly indicate that the attacker was looking to do more than turn the lights off for a few hours,” says Joe Slowik, a Dragos analyst who formerly led the Computer Security and Incident Response Team at the Department of Energy’s Los Alamos National Laboratory. “They were trying to create conditions that would cause physical damage to the transmission station that was targeted.”
Setting a Trap
The Ukraine-targeted blackout malware, known alternately as Industroyer or Crash Override, grabbed the attention of the cybersecurity community when the Slovakian cybersecurity firm ESET first revealed it in June 2017. It featured a unique ability to directly interact with an electric utility’s equipment, including features that could send automated, rapid-fire commands in four different protocols used in various power utilities to open their circuit breakers and trigger mass power outages.
But the new Dragos findings relate instead to an often-forgotten component of the 2016 malware, described in ESET’s original analysis but not fully understood at the time. That obscure component of the malware, ESET pointed out, looked like it was designed to take advantage of a known vulnerability in a piece of Siemens equipment known as a Siprotec protective relay. Protective relays act as electric grid fail-safes, monitoring for dangerous power frequencies or levels of current in electric equipment, relaying that information to operators and automatically opening circuit breakers if they detect dangerous conditions that could damage transformers, melt power lines, or in rare cases even electrocute workers. A security flaw in Siemens protective relays—for which the company had released a software fix in 2015 but which remained unpatched in many utilities—meant that any hackers who could send a single data packet to that device could essentially put it in a sleep state intended for firmware updates, rendering it useless until manually rebooted.
In 2017, ESET had noted the disturbing implications of that malware component; it hinted that Industroyer’s creators might be bent on physical damage. But it was far from clear how the Siprotec-hacking feature could have actually caused more lasting damage. After all, the hackers had merely turned off the power at Ukrenergo, not caused the sort of dangerous power surge that disabling a protective relay might exacerbate.
The Dragos analysis may provide that missing piece of the Ukrenergo puzzle. The company says it obtained the Ukrainian utility’s network logs from a government entity—it declined to name which one—and for the first time was able to reconstruct the order of the hackers’ operations. First, the attackers opened every circuit breaker in the transmission station, triggering the power outage. An hour later, they launched a wiper component that disabled the transmission station’s computers, preventing the utility’s staff from monitoring any of the station’s digital systems. Only then did the attackers use the malware’s Siprotec hacking feature against four of the station’s protective relays, intending to silently disable those fail-safe devices with almost no way for the utility’s operators to detect the missing safeguards.1
The intention, Dragos analysts now believe, was for the Ukrenergo engineers to respond to the blackout by hurriedly re-energizing the station’s equipment. By doing so manually, without the protective relay fail-safes, they could have triggered a dangerous overload of current in a transformer or power line. The potentially catastrophic damage would have caused far longer disruptions to the plant’s energy transmission than mere hours. It could also have harmed utility workers.
That plan ultimately failed. For reasons Dragos can’t quite explain—likely a networking configuration mistake the hackers made—the malicious data packets intended for Ukrenergo’s protective relays were sent to the wrong IP addresses. The Ukrenergo operators may have turned the power back on faster than the hackers expected, outracing the protective relay sabotage. And even if the Siprotec attacks had hit their marks, backup protective relays in the station might have prevented a disaster—though Dragos’s analysts say that without a full picture of Ukrenergo’s safety systems, they can’t entirely game out the potential consequences.
But Dragos director of threat intelligence Sergio Caltagirone argues that regardless, the sequence of events represents a disturbing tactic that wasn’t recognized at the time. The hackers predicted the power utility operator’s reaction and tried to use it to amplify the cyberattack’s damage. “Their fingers are not over the button,” Caltagirone says of the blackout hackers. “They’ve pre-engineered attacks that harm the facility in a destructive and potentially life-threatening way when you respond to the incident. It’s the response that ultimately harms you.”
Appetite for Destruction
The specter of physical destruction attacks on electric utilities has haunted grid cybersecurity engineers for more than a decade, since Idaho National Labs demonstrated in 2007 that it was possible to destroy a massive, 27-ton diesel generator simply by sending digital commands to the protective relay connected to it. The engineer who led those tests, Mike Assante, told WIRED in 2017 that the presence of a protective relay attack in the Ukrenergo malware, though not yet fully understood at the time, hinted that those destructive attacks might finally be becoming a reality. “This is definitely a big deal,” warned Assante, who passed away earlier this year. “If you ever see a transformer fire, they’re massive. Big black smoke that all of a sudden turns into a fireball.”
If the new Dragos theory of the 2016 blackout holds true, it would make the incident only one of three times when in-the-wild malware has been designed to trigger destructive physical sabotage. The first was Stuxnet, the US and Israeli malware that destroyed a thousand Iranian nuclear enrichment centrifuges roughly a decade ago. And then a year after the Ukrainian blackout, in late 2017, another piece of malware known as Triton or Trisis, discovered in the network of Saudi oil refinery Petro Rabigh, was revealed to have sabotaged so-called safety-instrumented systems, the devices that monitor for dangerous conditions in industrial facilities. That last cyberattack, since linked to Moscow’s Central Scientific Research Institute of Chemistry and Mechanics, merely shut down the Saudi plant. But it could have led to far worse outcomes, including deadly accidents like an explosion or gas leak.
What worries Caltagirone the most is how much time has passed since those events and what the world’s industrial-control-system hackers might have developed over those three years. “Between this and Trisis, we now have two data points showing a pretty significant disregard for human life,” Caltagirone says. “But it’s what we’re not seeing that’s the most dangerous thing out there.”
1Updated 9/13/2019 11:40AM EST with more information about how Dragos obtained the Ukrenergo logs.
“Security Analysis of an Advanced Metering Infrastructure,” by Aaron Hansen, et al., International Journal of Critical Infrastructure Protection (2017), at http://www.sciencedirect.com/science/article/pii/S1874548217300495
“An advanced metering infrastructure is an integrated system of smart meters, communications networks and data management systems designed to support the safe, efficient and reliable distribution of electricity while providing advanced functionality to energy customers. Unfortunately, sophisticated cyber attacks on advanced metering infrastructures are a clear and present danger.“