Story image

When AI goes rogue - a look into its possible futures

28 May 2018

What happens when artificial intelligence (AI) goes bad? According to the Electronic Frontier Foundation, AI and machine learning will bring benefits in diverse areas such as transport, health, art and science, but we’ve already seen things go horribly wrong.

Today’s computers are inherently insecure so they’re a poor choice for high-stakes machine learning systems and AI – and according to the Electronic Frontier Foundation, we need to consider the implications these new technologies may have for computer security.

Earlier this year the Electronic Frontier Foundation was one of six institutions that released a report called The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation.

Also involved in the report was the Future of Humanity Institute, the University of Oxford, the Centre for the Study of Existential Risk, the University of Cambridge, the Center for a New American Security, and OpenAI.

The report looks at AI’s potential impact on digital security, physical security, and political security.

It says there are specific security-relevant properties of AI, including its dual use for civilian and military purposes; its scalability; the ability for its algorithms to be rapidly distributed; and its ability to exceed human capabilities.

They can also expand existing threats, introduce new threats, and alter the typical character of threats, allowing attacks to be more versatile, effective, and targeted.

In terms of digital security, AI could influence email attacks such as spear phishing to become more automated – and it could even eliminate the need for the attacker to speak the same language as the target.

“Many important IT systems have evolved over time to be sprawling behemoths, cobbled together from multiple different systems, under-maintained and — as a consequence — insecure. Because cybersecurity today is largely labour-constrained,” the report notes.

AI could also target malware’s behaviour that it becomes impossible for humans to control in a manual way. The Stuxnet malware is a clear example of how a malware cannot receive commands once it infects computers.

In addition to automation of social engineering attacks, AI could also automatically discover vulnerabilities, automate hacking processes by evading detection and responding to behavioural changes from the target; it could mimic human-like denial-of-service attacks, and exploit legitimate AI itself.

Althrough offensive use of AI has only publicly been disclosed through experiments by white hat hackers, the report says it’s only a matter of time before it is used for malicious consequences – if it is not already happening.

AI could disrupt physical security by repurposing commercial AI systems for terrorism – for example using autonomous vehicles to cause crashes. It could enable distributed swarming attacks for surveillance, and it could increase the scale of attacks.

AI could also affect political security by allowing states to automate surveillance platforms.

“State surveillance powers of nations are extended by automating image and audio processing, permitting the collection, processing, and exploitation of intelligence information at massive scales for myriad purposes, including the suppression of debate,” the report says.

It could also achieve highly realistic videos to support fake news reports, manipulate information availability; automate influencing campaigns; and hyper-personalise disinformation campaigns.

The report recommends four approaches to responsible AI use:

1. Policymakers should collaborate closely with technical researchers to investigate, prevent, and mitigate potential malicious uses of AI.

2. Researchers and engineers in artificial intelligence should take the dual-use nature of their work seriously, allowing misuse-related considerations to influence research priorities and norms, and proactively reaching out to relevant actors when harmful applications are foreseeable.

3. Best practices should be identified in research areas with more mature methods for addressing dual-use concerns, such as computer security, and imported where applicable to the case of AI.

4. Actively seek to expand the range of stakeholders and domain experts involved in discussions of these challenges.

New threat rears its head in new malware report
Check Point’s researchers view Speakup as a significant threat, as it can be used to download and spread any malware.
Oracle updates enterprise blockchain platform
Oracle’s enterprise blockchain has been updated to include more capabilities to enhance development, integration, and deployment of customers’ new blockchain applications.
Used device market held back by lack of data security regulations
Mobile device users are sceptical about trading in their old device because they are concerned that data on those devices may be accessed or compromised after they hand it over.
Gartner names ExtraHop leader in network performance monitoring
ExtraHop provides enterprise cyber analytics that deliver security and performance from the inside out.
Symantec acquires zero trust innovator Luminate Security
Luminate’s Secure Access Cloud is supposedly natively constructed for a cloud-oriented, perimeter-less world.
Palo Alto releases new, feature-rich firewall
Palo Alto is calling it the ‘fastest-ever next-generation firewall’ with integrated cloud-based DNS Security service to stop attacks.
The right to be forgotten online could soon be forgotten
Despite bolstering free speech and access to information, the internet can be a double-edged sword, because that access to information goes both ways.
Opinion: 4 Ransomware trends to watch in 2019
Recorded Future's Allan Liska looks at the past big ransomware attacks thus far to predict what's coming this year.