SecurityBrief New Zealand logo
New Zealand's leading source of cybersecurity and cyber-attack news
Story image

When AI goes rogue - a look into its possible futures

By Sara Barker
Mon 28 May 2018
FYI, this story is more than a year old

What happens when artificial intelligence (AI) goes bad? According to the Electronic Frontier Foundation, AI and machine learning will bring benefits in diverse areas such as transport, health, art and science, but we’ve already seen things go horribly wrong.

Today’s computers are inherently insecure so they’re a poor choice for high-stakes machine learning systems and AI – and according to the Electronic Frontier Foundation, we need to consider the implications these new technologies may have for computer security.

Earlier this year the Electronic Frontier Foundation was one of six institutions that released a report called The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation.

Also involved in the report was the Future of Humanity Institute, the University of Oxford, the Centre for the Study of Existential Risk, the University of Cambridge, the Center for a New American Security, and OpenAI.

The report looks at AI’s potential impact on digital security, physical security, and political security.

It says there are specific security-relevant properties of AI, including its dual use for civilian and military purposes; its scalability; the ability for its algorithms to be rapidly distributed; and its ability to exceed human capabilities.

They can also expand existing threats, introduce new threats, and alter the typical character of threats, allowing attacks to be more versatile, effective, and targeted.

In terms of digital security, AI could influence email attacks such as spear phishing to become more automated – and it could even eliminate the need for the attacker to speak the same language as the target.

“Many important IT systems have evolved over time to be sprawling behemoths, cobbled together from multiple different systems, under-maintained and — as a consequence — insecure. Because cybersecurity today is largely labour-constrained,” the report notes.

AI could also target malware’s behaviour that it becomes impossible for humans to control in a manual way. The Stuxnet malware is a clear example of how a malware cannot receive commands once it infects computers.

In addition to automation of social engineering attacks, AI could also automatically discover vulnerabilities, automate hacking processes by evading detection and responding to behavioural changes from the target; it could mimic human-like denial-of-service attacks, and exploit legitimate AI itself.

Althrough offensive use of AI has only publicly been disclosed through experiments by white hat hackers, the report says it’s only a matter of time before it is used for malicious consequences – if it is not already happening.

AI could disrupt physical security by repurposing commercial AI systems for terrorism – for example using autonomous vehicles to cause crashes. It could enable distributed swarming attacks for surveillance, and it could increase the scale of attacks.

AI could also affect political security by allowing states to automate surveillance platforms.

“State surveillance powers of nations are extended by automating image and audio processing, permitting the collection, processing, and exploitation of intelligence information at massive scales for myriad purposes, including the suppression of debate,” the report says.

It could also achieve highly realistic videos to support fake news reports, manipulate information availability; automate influencing campaigns; and hyper-personalise disinformation campaigns.

The report recommends four approaches to responsible AI use:

1. Policymakers should collaborate closely with technical researchers to investigate, prevent, and mitigate potential malicious uses of AI.

2. Researchers and engineers in artificial intelligence should take the dual-use nature of their work seriously, allowing misuse-related considerations to influence research priorities and norms, and proactively reaching out to relevant actors when harmful applications are foreseeable.

3. Best practices should be identified in research areas with more mature methods for addressing dual-use concerns, such as computer security, and imported where applicable to the case of AI.

4. Actively seek to expand the range of stakeholders and domain experts involved in discussions of these challenges.

Related stories
Top stories
Story image
Tech job moves
Tech job moves - ActiveCampaign, Arcserve, LogRhythm & Qlik
We round up all job appointments from June 17-22, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Phishing
Online identity theft is rising in NZ - here’s what to do about it
It may start with a few stolen details online, but it could end with thousands of dollars missing or worse, a reputation down the drain.
Story image
Internet of Things
Domino's Pizza: A blueprint for secure enterprise IoT deployment
Increasingly, organisations are embracing smart technologies to underpin innovations that can enhance safety and productivity in every part of our lives, from industrial systems, utilities, and building management to various forms of business enablement.
Story image
Artificial Intelligence
Abnormal Security finds financial supply chain under threat
New research by Abnormal Security has found a rising trend in financial supply chain compromise as threat actors increasingly impersonate vendors.
Story image
Zscaler
Securonix partners with Snowflake, Zscaler in joint venture
Securonix is embarking on a joint technology integration with Snowflake and Zscaler to speed up threat detection and response at cloud scale.
Story image
Data Protection
Thales solution supports DevSecOps teams with data protection
Thales' CipherTrust Platform Community Edition enables DevSecOps teams to deploy data protection controls into multi-cloud applications faster.
Story image
Yubico
New research shows global drive for passwordless authentication
A new study has shown there has been a significant shift towards wanting a passwordless future, but adoption is still in its infancy.
Story image
Amazon Web Services / AWS
Zscaler, AWS accelerate onramp to the cloud with zero trust
Zscaler has announced an extension to its relationship with Amazon Web Services, as well as innovations built on Zscaler's Zero Trust architecture.
Story image
Cybersecurity
The link between cybersecurity, extremist threat and misinformation online in Aotearoa
Long story short, it's often the case that misinformation, threat and extremism link closely to cybersecurity issues and cyber harm.
Story image
Cybersecurity
ConnectWise reveals cybersecurity updates and partnerships
ConnectWise has unveiled new updates to its services and highlighted the importance of cyber insurance at its IT Nation Secure conference.
Story image
Cybersecurity
New survey uncovers critical OT security challenges
While industrial control environments continue to be a target for cyber criminals, there are widespread gaps in industrial security.
Story image
Collaboration
Why the success of client collaboration projects depends on addressing these five warning signs
New tools, applications, and software have enabled project collaboration to continue remotely, both between employees within an organisation and with its clients.
Story image
Cybersecurity
Greater API usage raises concerns for protection - report
Radware has released its 2022 State of API Security report, which shows a rise in APIs, with 92% of the organisations surveyed significantly or somewhat increasing their usage.
Story image
Manufacturing
Sternum joins NXP, collaborates on IoT security and observability
Sternum has announced it has joined the software partner community of NXP Semiconductors, a manufacturer of and large marketplace for embedded controllers.
Story image
APAC
Aqua Security launches cloud native security SaaS in APAC
Aqua Security has announced the general availability of cloud native security SaaS in Singapore, serving the broader APAC region.
Story image
Microsoft
SMX partnership with Microsoft leads to NTT recognition
SMX has captured the attention of NTT after receiving positive reviews from businesses across Australasia and beyond for its email security.
Story image
Cybersecurity
Significant security concerns resulting from open source software ubiquity
"The risk is real, and the industry must work closely together in order to move away from poor open source or software supply chain security practices."
Story image
Dark web
Cybercrime in Aotearoa: How does New Zealand law define it?
‘Cybercrime’ is a term we hear all the time, but what exactly is it, and how does New Zealand define it in legal terms?
Story image
Cybersecurity
Aqua Security, CIS create software supply chain security guide
Aqua Securityand the Center for Internet Security have together released the industry’s first formal guidelines for software supply chain security.
Story image
Cybersecurity
Trend Micro unveils dedicated security for electric vehicles
The cybersecurity company has announced VicOne - dedicated security for the electric vehicles and connected cars of today and tomorrow.
Story image
Wavelink
How Wavelink fosters creativity and careers in the channel
Wavelink is a 100% channel business, with all sales flowing through its authorised reseller partners. It has been a B2B technology distributor for almost 24 years and offers a broad range of software across cybersecurity, mobility, networking, and healthcare.
Story image
Cloud
SonicWall recognises partners and distributors at FY2022 partner awards
SonicWall has recognised its distributors and partners for their efforts in producing the company’s most successful year to date.
Story image
Identity and Access Management
Ping Identity launches corporate venture capital fund
Ping Identity has launched a corporate venture capital fund to foster innovative offerings for the identity security market.
Story image
Network Security
Netskope announces zero trust network access updates
Customers can now apply zero trust principles across a range of hybrid work security needs, including SaaS, IaaS, private applications, and endpoint devices.
Story image
Cloud
QuSecure partners with DataBridge Sites to showcase platform
QuSecure has partnered with DataBridge Sites to showcase its Quantum-as-a-Service (QaaS) orchestration platform, QuProtect.
Story image
Cybersecurity
Malwarebytes expands Nebula platform with DNS module
Malwarebytes has expanded its Nebula platform with a new DNS Filtering module designed to provide a quick, flexible, and comprehensive Zero Trust offering for Nebula users.
Story image
Apple
LastPass announces new capability for iPhones and iPads
LastPass has announced its new save and fill experience, allowing customers to fill in, create and save their credentials directly within the site's form field.
Story image
Compliance
OCEG survey shows demand for connected GRC systems
The survey also revealed that many organisations lack visibility and connected processes to manage the increased velocity and volume of risks. 
Story image
Collaboration
IT and security team collaboration crucial to data security
Many IT and security decision makers are not collaborating as effectively as possible to address growing cyber threats.
Story image
Cybersecurity
Threat actors ramp up their social engineering attacks
As people get better at identifying potential threats in their inbox, threat actors must evolve their methods. Their new M.O? Social engineering.
Story image
Ransomware
Rapid7 report examines use of double extortion ransomware attacks
New insight into how attackers think when carrying out cyber attacks, along with further analysis of the disclosure layer of double extortion ransomware attacks, has come to light.
Story image
DDoS
Q1 DDoS and application attack activity reveals surprise result
The cybersecurity threat landscape in the first quarter of 2022 represented a mixed bag of old enemies and new foes. New actors dominated the DDoS threat landscape while application security faced tried-and-true attack vectors.
Story image
Data resilience
Digital resilience in 2022 - A10 Networks releases new study
Of the 250 corporate organisations surveyed, as many as 95% showed high levels of concern for all aspects of enterprise digital resilience.
Story image
Digital Transformation
Cybersecurity priorities for digital leaders navigating digital transformation
In recent years, Asia-Pacific has especially been a hotspot for cyberattacks, and as we continue into 2022, it’s evident that the problem is becoming more significant.
Story image
PagerDuty
Ready for anything with the PagerDuty Operations Cloud
In a world of digital everything, teams face increasing complexity. Ever-growing dependencies across systems and processes put customer and employee experience, not to mention revenue, at risk.
Story image
Cybersecurity
Ingram Micro launches vendor-backed security program
Ingram Micro has unveiled a new program intended to give resellers the effective offerings their customers need to stay safe in the evolving threat landscape.
Story image
10 Minute IT Jams
Video: 10 Minute IT Jams - An update from Rimini Street
Today we welcome back Daniel Benad, who is the GVP & regional GM for Oceania at Rimini Street.
Story image
Cybersecurity
Why is NZ lagging behind the world in cybersecurity?
A recent report by TUANZ has revealed that we are ranked 56th in the world when it comes to cybersecurity - a look into why we're so behind and what needs to be done.
Story image
Secure access service edge / SASE
Cloudflare adds new capabilities to zero trust SASE platform
New features for Cloudflare One include email security protection, data loss prevention tools, cloud access security broker, and private network discovery.
Story image
Ransomware
More than 90% of cyber attacks made possible by human error
The data are clear, with cyberattacks on the rise in recent years and the cybersecurity situation increasingly complex. 
Story image
Cybersecurity
Palo Alto Networks named Google Cloud technology partner of the year for security
Palo Alto Networks was recognised for helping organisations rapidly transform security operations for future success.
Story image
INTERPOL
Hundreds arrested, millions seized in global INTERPOL investigation
A two-month-long investigation by INTERPOL this year involved 76 countries and clamped down on organised crime groups behind telecommunications and social engineering scams.
Story image
API
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
Cybersecurity
Schneider Electric and Claroty launch building security solution
Schneider Electric has announced the launch of Cybersecurity Solutions for Buildings, a solution designed to help buildings customers secure BMS.