A massive IT outage has caused multiple services to be taken offline, while planes and trains were grounded as a result.
This story first appeared at SC Media. See the original story here.Potentially linked to
an update from CrowdStrike to
Microsoft users, the issue is apparently due to a misconfigured update which is causing users globally to hit a "blue screen of death" (BSOD).
In a
Reddit update, a post — apparently by a member of
the CrowdStrike team — said they are aware of "widespread reports of BSODs on Windows hosts, occurring on multiple sensor versions. Investigating cause."
Read our full coverage across the CRA network:Defect in single content update
CrowdStrike CEO George Kurtz issued a statement saying the company is actively working with impacted customers, and the issue was caused by a defect found in a single content update for
Windows hosts.
Confirming the outage was not “a security incident or cyberattack,” Kurtz said: "The issue has been identified, isolated and a fix has been deployed. We refer customers to the support portal for the latest updates and will continue to provide complete and continuous updates on our website.
“We further recommend organizations ensure they’re communicating with CrowdStrike representatives through official channels. Our team is fully mobilized to ensure the security and stability of CrowdStrike customers."
Reports of crashes
The disruptive file was identified as an
update for CrowdStrike's Falcon sensor, which was released on July 9. A workaround was
published, with CrowdStrike saying it is "aware of reports of crashes on Windows hosts related to
the Falcon sensor."
CrowdStrike has
confirmed that it is no longer pushing the update, “so you only have to fix the machines that were already stuck in a BSOD loop: anything that isn't impacted now shouldn't be impacted.”
Writing on X, CrowdStrike's chief threat hunter Brody Nisbet offered a
workaround, recommending users:
1. Boot Windows into Safe Mode or WRE.
2. Go to C:WindowsSystem32driversCrowdStrike
3. Locate and delete file matching "C-00000291*.sys"
4. Boot normally.
Cybersecurity researcher Kevin Beaumont
wrote on X that the global IT outage "is CrowdStrike as cause, not Microsoft" and that two different outages were linked together. While the Microsoft one was solved a while ago, it may be the CrowdStrike update that is causing the issue.
Infinite loop
Ilkka Turunen, field CTO at Sonatype said in an email to SC UK that the update caused a BSOD loop on any Windows machine, essentially making it boot and crash on an infinite loop.
“Making it worse is the fact that there are a significant number of Windows machines that the update was auto-installed on overnight,” he said. “There are workarounds that customers of theirs will apply, but it seems to be very manual.
"It’s definitely a supply chain style incident — what it shows is that one popular vendor botching an update can have a huge impact on its customers and how far a single well-orchestrated update can spread in a single night."
