AI flaws & supply-chain risks top new pentesting report

Cobalt has published its annual State of Pentesting Report, highlighting rising concern over AI flaws, nation-state threats and third-party software risk.

The report draws on penetration testing data from more than 2,700 organisations and a survey of 450 security professionals. It found that 75% of organisations rank third-party software as a top risk, yet 86% deploy vendor tools without proof of security testing.

AI-related weaknesses featured prominently. Some 32% of all AI and large language model findings were rated high risk, compared with 12% across the wider dataset.

One in five organisations said they had experienced an LLM-related security incident in the past year. Another 18% were unsure, while 19% preferred not to answer.

The findings suggest security teams are finding AI systems harder to secure than other applications. LLMs had the lowest resolution rate of all application types tested, with only 38% of high-risk issues fixed.

Confidence among security teams has also fallen. Last year, 64% said they felt able to keep up with the security implications of AI adoption. That figure has now dropped to 51%.

At the same time, 61% of security professionals said they would like a strategic pause to calibrate defences against AI-driven threats. Yet 97% are adding AI features to software and services, underlining the tension between deployment pressure and security readiness.

Threat shift

Beyond AI, the report points to a broader change in the threat environment. Twenty per cent of respondents ranked nation-state threats as a top risk, rising to 40% in financial services.

It also found that 93% of respondents had observed attackers using AI to make their methods more sophisticated. Cobalt linked that pattern to concerns about software supply chains and the use of trusted third-party tools in attacks.

The data showed wide differences in how quickly organisations fix serious security issues. Top-performing organisations reached a high-risk finding half-life of 10 days, while those in the bottom tier took 249 days.

That gap was mirrored by a divide between senior management and frontline teams. Some 57% of C-suite executives believe their organisation consistently meets remediation service-level agreement targets, but only 15% of security practitioners agree.

Resolution rates were also largely flat. The typical organisation resolves 86% of its high-risk findings, but only 52% of all findings are remediated within five years.

Testing models

Cobalt used the report to argue that frequent testing is linked to faster remediation. Organisations using a continuous, programmatic approach to penetration testing were 4.5 times more likely to resolve critical findings within three days than those relying on compliance-led or ad hoc testing.

Security spending also increased across the organisations surveyed. Nearly a third, 33%, reported significant budget growth in the past year, while 50% saw incremental increases.

That rise comes as organisations face mounting pressure to validate the security of supplier software and AI tools. The report argues that annual assessments and trust based on certifications alone are insufficient against fast-moving attacks.

Gunter Ollmann, chief technology officer at Cobalt, said the difficulty of addressing AI flaws often stems from weaknesses in the underlying models rather than in customer-controlled software.

"The poor resolution rate of AI is largely attributable to issues within LLM models themselves, which security professionals often cannot fix directly. Instead of waiting on vendors, organizations must take on the initiative through continuous pentesting to proactively enhance security," Ollmann said.

He added that organisations should not assume suppliers will move quickly enough when flaws are found.

"By taking an offensive security approach, companies can identify vulnerabilities before vendors do and mitigate risk by blocking their access to data. Last year's data showed us how exposed supply chains are to attack; the message is simple: take matters into your own hands, because vendor fixes often come too late," he said.