“Vaccines” to protect neural networks against hacking

  Last updated September 3, 2019 at 11:59 am


A new programming technique will see machine learning systems ‘vaccinated’ against malicious cyber-attacks.

machine learning systems_hacking_hacking prevention

A programming technique that works on the same principle as disease-preventing vaccinations could safeguard machine learning systems from malicious cyber-attacks.

The technique was developed by the digital specialist arm of the CSIRO, and presented recently at an international conference on machine learning, held in Long Beach, California, US.

Machine learning systems are vulnerable to hacking attacks.

Machine learning systems, or neural networks, are becoming increasingly prevalent in modern society, where they are pressed into service across a wide range of areas, including traffic management, medical diagnosis, and agriculture. They are also critical components in autonomous vehicles.

They operate from an initial training phase, in which they are fed tens of thousands of possible iterations of a given task. The resulting algorithm then has the capability to learn – to add to its own repertoire of possibilities and act accordingly, without the need for further human input.

As efficient as they are, however, machine leaning systems – like any computer-driven mechanism – remain vulnerable to hacking attacks. The primary way in which this is happens involves the introduction of “noise” – additional data-points that interfere with and distort input signals – such that external elements are misclassified.

The approach is known in the jargon of the business as introducing “adversarial examples” into the system. By adding noises (and often, not very much noise) a machine learning algorithm can be misled into classifying an image of a panda as an image of a gibbon.

More pertinently, given the rise of autonomous vehicles, they can be hacked and persuaded to classify a stop sign as a green traffic light.

Latest approach takes its cue from public health

Making machine-learning systems thus hack-resistant is a lively research field, spurred by recent research that found that real-world examples could be easily fooled with the use of nothing more sophisticated than a smartphone camera.

The latest approach, by researchers led by Richard Nock, takes its cue from public health.

In medicine, the practice of vaccination rests on the sound idea that exposing the body’s immune system to weak or dead versions of a pathogen – the ones that cause influenza or polio, for instance – prompts the development of specific antibodies. The immune system then “remembers” the pathogen so that the next time it encounters it – at full strength and for real – it will recognise it and eliminate it straight away.

Nock and colleagues approached their task in the same manner.

“Our new techniques prevent adversarial attacks using a process similar to vaccination,” he explains.

“We implement a weak version of an adversary, such as small modifications or distortion to a collection of images, to create a more ‘difficult’ training data set. When the algorithm is trained on data exposed to a small dose of distortion, the resulting model is more robust and immune to adversarial attacks.”

The approach is still at an early stage and has yet to be tested in a real-world situation against genuine malicious incursion attempts, but the results are promising. They are described in detail in a preprint paper available here.

And although more testing is clearly required, there is at this stage no evidence that vaccination may turn computers autistic.


Machine learning – the revolution has arrived

What is machine learning?

Using machine learning to predict medical outcomes

About the Author

Andrew Masterson
Andrew Masterson is former editor of Cosmos.

Published By

Cosmos is a quarterly science magazine. We aim to inspire curiosity in ‘The Science of Everything’ and make the world of science accessible to everyone.

At Cosmos, we deliver the latest in science with beautiful pictures, clear explanations of the latest discoveries and breakthroughs and great writing.

Winner of 47 awards for high-quality journalism and design, Cosmos is a print magazine, online digital edition updated daily, a daily and weekly e-Newsletter and educational resource with custom, curriculum-mapped lessons for years 7 to 10.

Featured Videos

Fitting natural water treatment processes back into the landscape
Protecting the Great Barrier Reef at the National Sea Simulator