A faulty CrowdStrike update file took down Windows-based PCs worldwide for some hours last night disrupting airlines and other businesses. The impact of this global event was major, to say the least as airlines from around the world had to ground planes. The faulty CrowdStrike update file rolled out to Windows hosts did not affect Mac or Linux machines. Meanwhile, as reported by the BBC who checked Downdetector for outages, it was discovered that many major services were struggling to function, for a short time. BBC noted that outage report spikes have dropped off significantly but because Downdetector relies on users to self-report there could be many, many more not getting documented.
Per BBC:
“But the drop-off from that spike shows that very few people are now reporting issues with many services – of course, that’s just a snapshot of some of the companies which seem to be affected.”
The fallout from this event continues to spread and while some airlines are recovering quickly others are still stuck on the ground. Reports from Australia indicate cancelations for the rest of the day have happened. Microsoft continues to provide Azure status updates advising users running the CrowStrike Falcon agent to restart their VM machines. Some users have reported that it can take as many as 15 restarts to get their machines back up and running.
Per Microsoft:
“We’ve received feedback from customers that several reboots (as many as 15 have been reported) may be required, but overall feedback is that reboots are an effective troubleshooting step at this stage.”
CrowdStrike’s Falcon security software was reportedly causing BSODs on Windows machines essentially rendering them unusable but a workaround involving a Safeboot to delete the faulty file was quickly discovered. However, since then an updated file has been deployed and CrowdStrike has issued a statement regarding the matter.
Official Statement regarding Sensor Update:
“CrowdStrike is actively working with customers impacted by a defect found in a single content update for Windows hosts. Mac and Linux hosts are not impacted. This is not a security incident or cyberattack. The issue has been identified, isolated and a fix has been deployed. We refer customers to the support portal for the latest updates and will continue to provide complete and continuous updates on our website. We further recommend organizations ensure they’re communicating with CrowdStrike representatives through official channels. Our team is fully mobilized to ensure the security and stability of CrowdStrike customers.”