Faced with the global CrowdStrike outage, University of Adelaide experts led by Associate Professor Hung Nguyen from the School of Computer and Mathematical Sciences rapidly mobilised to find a solution and implement it.
The experts consulted with staff from CrowdStrike, the Australian Signals Directorate (ASD) and the Australian Cyber Collaboration Centre (AUS3C) to validate and roll out a solution for the University’s ITDS infrastructure.
CrowdStrike Falcon sensor outage causes widespread BSOD issues
“Using our long track record of research into Windows security we, like many others in the global IT community, quickly brought our expertise to bear to examine the problem that was causing chaos around the world,” said Associate Professor Nguyen.
“The official fix from CrowdStrike required computers to be rebooted into safe mode, but this proved to be a challenge for many IT administrators.
“The issue was compounded by the fact that many computers were protected by Windows BitLocker, which requires a recovery key to reboot into safe mode.
“Many IT administrators did not have access to these recovery keys, leaving them unable to recover from the CrowdStrike outage.
“In some cases, the only option was to wipe the data and perform a fresh install, a drastic measure that most administrators would prefer to avoid.”
The CrowdStrike platform is purpose-built to stop breaches via a unified set of cloud-delivered technologies that prevent all types of attacks – including malware.
The global software error in the CrowdStrike Falcon sensor software that occurred on Friday afternoon Australian time, caused widespread blue screen of death (BSOD) issues on many Windows computers and impacted airlines, retail businesses and media outlets as well as universities.
“Using our long track record of research into Windows security we, like many others in the global IT community, quickly brought our expertise to bear to examine the problem that was causing chaos around the world.”Associate Professor Nguyen.
The problem with safe mode – a solution from the University of Adelaide
“Fortunately, our team at the University of Adelaide discovered a quirk in the way BitLocker protects the boot sequence and developed a method that allows safe booting without a recovery key,” said Associate Professor Hung.
How it works
“The key to our solution lay in the Boot Configuration Data (BCD) database, which stores boot-related information on Windows computers. BitLocker verifies that the security-sensitive BCD settings haven’t changed since BitLocker was last enabled, resumed, or recovered,” said Associate Professor Hung.
“However, BitLocker leaves a long that it does not check by default.
“Our method booted computers from a USB key and rewrote the BCD to the minimal boot configuration, taking advantage of these unprotected areas. This allowed computers to be booted into safe mode without requiring the recovery key and then the update from CrowdStrike automatically was applied.
“The method allowed computers to boot into safe mode only and did not break the data protection provided by BitLocker. All data encrypted by BitLocker remained encrypted.”
Adoption of the University of Adelaide fix
“On 20 July our solution was shared by A3C on their LinkedIn page so that it could be used by the wider cybersecurity community along with many other solutions being deployed globally, to solve the problems caused by the CrowdStrike outage,” said Associate Professor Hung.
“The post received widespread attention, with comments from researchers who successfully used our method. Some commenters confirm that by using our method they managed to fix ‘dozens’ of their computers.”
