AI adoption races ahead of governance in businesses

Data and operational governance, and security frameworks are focal points of cybersecurity. However, AI has changed the game. With vulnerabilities being discovered at an unprecedented pace and exploits being created at break-neck speed, last year's governance frameworks are no longer adequate.

The security controls that sit around AI governance are not keeping up with the pace of technological change. Research from Avec points to a significant governance gap.

Jack Jorgensen, the General Manager of Data, AI and Innovation at Avec, said, "There's been a six-fold increase in embedded strategy adoption. When we look at the actual usage of that AI and workflows within organisation, that went from 13% to 53%."

Despite the massive adoption over the last 12 months the research found that just one in six people are getting enough training with only 8% having clear KPIs to measure AI's benefits.

"The stat that really took me back was that around a quarter of people have never had any AI policy training," added Jorgensen. "When you've had this giant increase in usage of a tool that we know doesn't always give you the right answer, one would think that you would have policies around how we should use that within the business."

A significant factor contributing to that gap is over-trusting the technology. Jorgensen said many people use AI as a research tool but often lack the knowledge to evaluate whether a response is correct. This is often fuelled by what he calls the "sycophancy of the tools" as they are often designed to provide the response people want or expect rather than a truly correct answer.

Often, the goal of AI is to save time and automate repetitive tasks – something Jorgensen said came out clearly in the research. However, he asked how do you measure the business impact.

"I see a lot of businesses that lack ROI governance over their AI adoption. Some organisations are taking their eye off what makes them great and, instead, just asking 'Have we put AI in yet?'," he said.

AI's uptake has broken almost every record there is when it comes to technology adoption. That means governance has often failed to keep up. Jorgensen said there are some steps he takes when working with clients to close the governance gap.

"The first thing I ask for is their policies and for the last time they reviewed and refreshed them. I talk with their IT team and work out what tools are approved in the organisation and if they have that approval stage gate. If they don't, I recommend they ring fence what is permitted and putting those tools through a review process."

The next stage is accurately assessing the material impact and benefits of AI on the organisation and accurately assessing the costs.

"We see a lot of what I would call rhetoric benefit. People say AI makes them more efficient, but they fail to assess the time they take to rectify errors or omissions. AI may make them feel more efficient but there may not be an actual outcome benefit."

Training is also important Jorgensen added.

With so many different AI tools available, the risk of shadow AI is significant. He said providing a clear pathway for people to suggest AI applications they like to use can help develop rigour around how AI tools can be used and applied around a specific business.

While AI is a fast-evolving technology, Jorgensen emphasised the need to ensure traditional controls are not forgotten. Ensuring PII leakage, through people entering sensitive information in public AI tools, is minimised remains critical. And maintain robust data hygiene is important as poorly curated data can lead internal AI models to hallucinate or deliver spurious results.

Organisations also need to protect themselves against prompt injection attacks. Threat actors can inject invisible or hard-to-detect prompts into documents or other content that dupe AI tools into doing anything from prioritising a job application to bypassing controls to provide access to restricted data.

The rapid spread of AI has outpaced existing governance structures leaving many organisations with inadequate policies, scant training and unclear ROI metrics. Organisations need to adopt a disciplined approach. They must refresh or create clear usage policies, vet all tools through an approval gate, quantify benefits and costs before deployment, and maintain traditional safeguards against PII leakage and prompt‑injection attacks. Organisations cannot compromise on governance and security if they want to reap the efficiencies promised by AI.