By now, most people are aware of â or have been personally affected by â the largest IT outage the world have ever witnessed, courtesy of a defective update for Crowdstrike Falcon Sensors that threw Windows hosts into a blue-screen-of-death (BSOD) loop.
âWe currently estimate that CrowdStrikeâs update affected 8.5 million Windows devices, or less than one percent of all Windows machines. While the percentage was small, the broad economic and societal impacts reflect the use of CrowdStrike by enterprises that run many critical services,â David Weston, Microsoftâs VP of Enterprise and OS Security, stated on Saturday.
CrowdStrike claimed earlier today that âa significant numberâ of affected systems are back online and operational.
âTogether with customers, we tested a new technique to accelerate impacted system remediation. Weâre in the process of operationalizing an opt-in to this technique,â they noted on their remediation and guidance hub. âCustomers are encouraged to follow the Tech Alerts for latest updates as they happen and they will be notified when action is needed.â
Microsoft collaborates with Crowdstrike, provides recovery tool
Microsoft is, understandably, doing everything it can to speed up worldwide recovery from the issue, has deployed hundreds of Microsoft engineers and experts to work with customers to restore services, and is collaborating with CrowdStrike.
âCrowdStrike has helped us develop a scalable solution that will help Microsoftâs Azure infrastructure accelerate a fix for CrowdStrikeâs faulty update. We have also worked with both AWS and GCP to collaborate on the most effective approaches,â Weston explained.
Microsoft has also released a recovery tool that can be downloaded and used by IT admins to make the repair process less time-consuming.
The tool provides two repair options.
The first one â Recover from WinPE (Preinstallation Environment) â does not require local admin privileges, but requires the person to manually enter the BitLocker recovery key (if BitLocker is used on the device).
The second one â Recover from safe mode â may allow recovery without entering the BitLocker recovery keys.
âFor this option, you must have access to an account with local administrator rights on the device. Use this approach for devices using TPM-only protectors, devices that are not encrypted, or situations where the BitLocker recovery key is unknown,â the Intune Support Team noted.
They also included detailed recovery steps for Windows clients, servers, and OSes hosted on Hyper-V.
Microsoft has previously confirmed that the buggy CrowdStrike update affected Windows 365 Cloud PCs and that users âmay restore their Windows 365 Cloud PC to a known good state prior to the release of the update (July 19, 2024)â. The company has also provided guidance for restoring affected Azure virtual machines.
Cloud security company Orca has released a script that automates the remediation of Windows virtual machines hosted on AWS.
Threat actor exploiting the situation
As expected, scammers and threat actors have immediately started taking advantage of the chaos that resulted from the faulty update.
Trend Micro researchers provided examples of tech support scams doing the rounds, and even legal scams.
CrowdStrike warned about:
- Attackers offering a fake utility for automating recovery that loads the Remcos remote access tool
- Phishers and vishers impersonating CrowdStrike support and contacting customers
- Scammers posing as independent researchers, claiming to have evidence the technical issue is linked to a cyberattack and offering remediation insights
âCrowdStrike Intelligence recommends that organizations ensure they are communicating with CrowdStrike representatives through official channels,â the company said.
UPDATE (July 23, 2024, 05:15 a.m. ET):
CrowdStrike has provided a way for remediating affected systems more quickly. Customers must opt in to use the technique via the support portal. (A Reddit user has explained the process involved.)
The company has also released a video explaining how users can self-remediate affected remote Windows laptops.
Fake CrowdStrike repair manual pushes new infostealer malware
“Resiliency in the digital age isnât just about preventing outages; itâs about being prepared to respond effectively when they happen.”
InfoSec services | InfoSec books | Follow our blog | DISC llc is listed on The vCISO Directory | ISO 27k Chat bot