Why AI could threaten much of the world’s digital infrastructure 

Earlier this month, authorities worldwide urged website administrators to execute emergency updates after a nonprofit programming foundation flagged critical security flaws that could allow hackers to breach systems.

Government and corporate websites briefly went offline to patch the vulnerabilities, nicknamed PHP2Shell, which were discovered by independent cybersecurity experts using artificial intelligence (AI).

The vulnerabilities were embedded in the code of a widely used open-source web programming language, PHP, for more than a decade. They had gone unnoticed, however, because it would have been too complicated for humans to exploit them without the use of advanced AI.

But cybersecurity experts, as well as those maintaining important software, say PHP2Shell could be the first in a deluge of security flaws exposed by emerging AI. They say much of the world’s most critical digital infrastructure is maintained by volunteers, who often lack the resources and time to respond to newly discovered threats.

“There’s been a lot of people in the AI safety community that said this is one of the first areas where we are going to see the harmful impact of AI,” said Peter Slattery, a research scientist at MIT’s FutureTech, who studies how cybersecurity experts are dealing with the new technology and tracks real-world incidents involving AI.

A digital world built on volunteer work

PHP got its start in 1995, when Danish-Canadian programmer Rasmus Lerdorf sought to track how many people visited his online resume. Over time, others around the world improved Lerdorf’s software, and it grew in popularity.

As with other open-source coding projects, anyone could download, modify and share improvements. Websites like Facebook and Wikipedia were built using PHP, which runs about 70 percent of the world’s websites.

In 2021, a dedicated nonprofit, the PHP Foundation, was launched to coordinate development. Like many other foundations maintaining open-source software, it operates on a meagre budget.

Most of the digital infrastructure consumers use is powered by open source software. The Mozilla Firefox browser is open source, as are languages like Perl, Python, and JavaScript. The software that lets a computer communicate with others on the internet is often open source, with many servers using products like Apache HTTP Server, maintained by the Apache Software Foundation.

“The world today, including proprietary software vendors, relies on open source software for its digital infrastructure,” said Amanda Brock, CEO of OpenUK, a nonprofit that supports open source software.

The world’s largest corporations, like Meta, Google, OpenAI and Anthropic, depend on that volunteer-built, open-source infrastructure. Now, those companies’ own AI tools are uncovering security flaws in the infrastructure at a pace that needs far more resources for groups like the PHP Foundation to keep up.

“It’s simply not possible with the resources in place to fix all of the known vulnerabilities,” Brock said. “Foundations are attempting to manage these challenges, but the smaller ones don’t have hundreds of millions in funding, and managing this at the pace AI requires is proving impossible.”

A deluge of new bugs to fix

The PHP2Shell vulnerabilities were found through several advanced AI models, which can employ far more complicated attempts to hack past security barriers than humans could.

Developers of other software are also discovering far more security flaws than in the past because AI is being used to probe defences.

Google said it uncovered more than 1,000 security bugs in the last update for its Chrome browser, more than in the last two dozen updates combined. Mozilla, the foundation behind the Firefox browser, found more than 400 security vulnerabilities in April, a significant jump from the previous record of 76 bugs discovered just one month earlier.

“We have maybe 10 times more issues than usual because we are using AI to find them,” said Jarek Potiuk, part of the volunteer security team at the Apache Software Foundation. (In keeping with the foundation’s non-hierarchical ethos, Potiuk said he has no real title).

Potiuk said there is an “industry-wide movement” to conduct scans using AI models and identify security vulnerabilities, much like was done to find the bugs in PHP. But the scans themselves cost money – used to pay for the tokens for AI models – and afterwards, humans must pore over them to weed out false positives.

“Every project, every foundation is doing this right now – spending enormous amounts of time to find these issues,” Potiuk said. “AI helps to find them, but you have to verify things, you have to review them.”

Earlier this year, Anthropic launched Project Glasswing, offering $100m in credits to support its advanced Claude Mythos AI model to search for security vulnerabilities in the world’s critical software. In one month, more than 23,000 vulnerabilities were found across more than 1,000 open source projects.

The Apache Software Foundation has taken advantage of another scheme, the Alpha-Omega project, to scan for security bugs across the 290 software packages it maintains. The project, backed by companies like Anthropic, Amazon, Google, Microsoft and OpenAI, spends about $7m a year helping open-source developers find critical security bugs.

But those working on finding and fixing the security flaws say the corporate world could be doing a lot more, given the trillion-dollar valuation of companies whose products would not work without free software like the packages the Apache Foundation provides.

“They are helping a lot, but we need more active players in this area,” Potiuk said, “because it’s highly disproportional – the impact it has on the corporate world versus the open source world”.

When a major security flaw like PHP2Shell is uncovered, it often comes down to a handful of volunteers to fix it.

Keeping the internet’s infrastructure as decentralised and open-source as possible is crucial, Potiuk said. Instead of buying out something like PHP, major players like Google or Microsoft could, for instance, pay staff to dedicate some of their work hours to maintaining open-source projects they use.

Real-world targets

The 2021 Log4Shell vulnerability, which allowed hackers to remotely run code on millions of devices through a widely used library maintained by the Apache Software Foundation, underscored the critical role open-source software plays in system security. That bug was discovered without the use of AI.

Suddenly, cars, microwaves and a host of electronic devices connected to the internet were vulnerable, Potiuk said.

“That was the moment when the whole ecosystem realised how important it is to take care of open source,” he said.

A bug like that, if found by a malicious hacker and exploited before there is a chance to patch devices, could have caused a major disruption.

Slattery said the database of incidents he and his colleagues maintain shows that an increasing number of vulnerabilities are being detected, and it is a matter of time before some are exploited in the real world.

“We are starting to see both a lot more vulnerabilities detected and addressed in time, but also attacks on a wide range of systems,” Slattery said. “We have been moving in this direction for a long time. There are more and more things now you can attack.”