IT professionals across the globe have worked long hours to secure websites against two critical vulnerabilities in PHP amid urgent warnings, unverified reports of active exploitation and worries about service disruptions adding to the pressure.
“This is the type of incident we haven’t seen in a long time,” Microsoft chief executive Satya Nadella said. He described the response as a demanding period for IT and security teams.
For administrators responsible for keeping websites online, the challenge has moved beyond just installing security updates. They’ve also had to assess risks across different systems, deal with anxious customers and figure out what warnings are evidence-based.
"It's been relentless," said Adam Mudd, a UK-based system administrator, who said he'd been getting an unusually high volume of messages from clients worried about their websites.
“We have been considering requests to pull sites offline against the need to apply fixes across a broad array of systems,” Mudd said. He said, "Sometimes it took longer to take down a website than it did to fix the underlying vulnerability.”
He said he worked long hours, sometimes starting early and working until 10:30 p.m. British time, as he sought to secure customers’ systems.
PHP is a commonly used programming language for web servers that support websites and applications. Security vulnerabilities in the software can therefore impact a large number of online services.
October 21, 2020 – The PHP project released security updates to fix two critical vulnerabilities that could allow attackers to execute commands on vulnerable servers. Cybersecurity authorities recommended that organisations quickly patch affected systems or isolate them if they could not be patched.
Response impeded by legacy systems
“We also had to work under intense pressure knowing how widely PHP was used across our software environment,” said IT administrator Loganaden Velvindron in Mauritius.
Some systems were running versions of PHP that were at the end of life and not receiving routine security updates. In those cases, Velvindron said the work could go beyond just installing a patch and could include rebuilding web applications or modifying code.
The short notice of the vulnerabilities also left administrators with little time to prepare, he said.
Jon Jones, the technical director at a digital advertising firm, stated that receiving the security warning was something new. He said there were some exciting moments when the quick response was applied, but there were also risks to applying updates, as changes could cause compatibility issues or disrupt existing services.
“Everybody was rushing to upgrade their systems when the PHP project released fixes. Jones remembered the rush when the patch was released at 3 p.m. British time on Wednesday, October 21.
For organisations running complex websites, installing a patch is only part of the story. Teams also need to ensure systems are still working fine after an update.
Pressure mounts as reports remain unverified.
The response became more complicated as early information circulated about the vulnerabilities and reports suggested that cyber criminals could already be exploiting them.
His team has been inundated with intelligence claims that attackers have working proof-of-concept methods -- demonstrations of how a vulnerability could be exploited, said UK cybersecurity expert Daniel Card.
Some of those reports were not accurate, Card said. He agreed that the vulnerabilities were serious but said that inaccurate or unsubstantiated claims had compounded the pressure on security teams.
This distinction is important because a vulnerability that can be exploited to launch an attack, a proof of concept exploit, and evidence of a successful attack are not the same thing.
As details of the PHP flaws became public, cybersecurity firms reported a sharp rise in attempted exploits. For instance, Cloudflare reported that the number of attempts it had seen increased from thousands before the disclosure to millions afterwards.
But attempted attacks don’t actually prove that attackers successfully compromised systems or stole data. Any successful penetrations must be separately verified as to size and effects.
Alert fatigue concerns
The incident also raises questions about the communication of security advisories.
Government agencies issued urgent advice, and some private-sector warnings about the availability of working exploits were later challenged or withdrawn. Mixed reports made it harder for administrators to distinguish between proven threats and speculation.
That uncertainty could have long-term effects. If an organisation is repeatedly hit with alarming warnings that turn out to be false or overblown, it may become less responsive to genuinely urgent threats.
As artificial intelligence makes it easier to discover and investigate software vulnerabilities, the cybersecurity industry needs to calibrate its messaging, warns Mark Maunder, founder of WordPress security company Wordfence.
Automated analysis tools that use artificial intelligence helped researchers discover the PHP flaws, highlighting how artificial intelligence can be used to uncover weaknesses in widely used software. But these same advances could also help bad actors develop attack methods more quickly.
The incident has heightened fears that security teams will face an increasing number of vulnerabilities that need urgent patching as AI-powered research improves.
However, for IT professionals whose job is to keep websites up and running, the immediate work is practical: identify affected systems, apply the correct fixes, test services and look for signs of compromise.
The bigger challenge is to make sure that urgent warnings convey the seriousness of a threat, but that speculation doesn’t drown out the evidence.
