Adversarial AI Attacks, Mitigations, and Defense Strategies

Name: Adversarial AI Attacks, Mitigations, and Defense Strategies | A cybersecurity professional's guide to AI attacks, threat modeling, and securing AI with MLSecOps
Brand: Packt Publishing
Price: 35.99 EUR
Availability: OnlineOnly

A cybersecurity professional's guide to AI attacks, threat modeling, and securing AI with MLSecOps

John Sotiropoulos(Autor*in)

Packt Publishing

1. Auflage

Erschienen am 26. Juli 2024

602 Seiten

E-Book

ePUB mit Adobe-DRM

Systemvoraussetzungen

978-1-83508-867-8 (ISBN)

35,99 €inkl. 7% MwSt.

Systemvoraussetzungen

für ePUB mit Adobe-DRM

E-Book Einzellizenz

Als Download verfügbar

Beschreibung

Weitere Details

Weitere Ausgaben

Inhalt

Cover
Title Page
Copyright
Dedication
Contributors
Disclaimer
Table of Contents
Preface
Part 1: Introduction to Adversarial AI
Chapter 1: Getting Started with AI
Getting the most out of this book - get to know your free benefits
Understanding AI and ML
Types of ML and the ML life cycle
Key algorithms in ML
Neural networks and deep learning
ML development tools
Summary
Further reading
Chapter 2: Building Our Adversarial Playground
Technical requirements
Setting up your development environment
Python installation
Creating your virtual environment
Installing packages
Registering your virtual environment with Jupyter notebooks
Verifying your installation
Hands-on basic baseline ML
Simple NNs
Developing our target AI service with CNNs
Setup and data collection
Data exploration
Data preprocessing
Algorithm selection and building the model
Model training
Model evaluation
Model deployment
Inference service
ML development at scale
Google Colab
AWS SageMaker
Azure Machine Learning services
Lambda Labs Cloud
Summary
Chapter 3: Security and Adversarial AI
Technical requirements
Security fundamentals
Threat modeling
Risks and mitigations
DevSecOps
Securing our adversarial playground
Host security
Network protection
Authentication
Data protection
Access control
Securing code and artifacts
Secure code
Securing dependencies with vulnerability scanning
Secret scanning
Securing Jupyter Notebooks
Securing models from malicious code
Integrating with DevSecOps and MLOps pipelines
Bypassing security with adversarial AI
Our first adversarial AI attack
Traditional cybersecurity and adversarial AI
Adversarial AI landscape
Summary
Part 2: Model Development Attacks
Chapter 4: Poisoning Attacks
Basics of poisoning attacks
Definition and examples
Types of poisoning attacks
Poisoning attack examples
Why it matters
Staging a simple poisoning attack
Creating poisoned samples
Backdoor poisoning attacks
Creating backdoor triggers with ART
Poisoning data with ART
Hidden-trigger backdoor attacks
Clean-label attacks
Advanced poisoning attacks
Mitigations and defenses
Cybercity defenses with MLOps
Anomaly detection
Robustness tests against poisoning
Advanced poisoning defenses with ART
Adversarial training
Creating a defense strategy
Summary
Chapter 5: Model Tampering with Trojan Horses and Model Reprogramming
Injecting backdoors using pickle serialization
Attack scenario
Defenses and mitigations
Injecting Trojan horses with Keras Lambda layers
Attack scenario
Defenses and mitigations
Trojan horses with custom layers
Attack scenario
Defenses and mitigations
Neural payload injection
Attack scenario
Defenses and mitigations
Attacking edge AI
Attack scenario
Defenses and mitigations
Model hijacking
Trojan horse code injection
Model reprogramming
Summary
Chapter 6: Supply Chain Attacks and Adversarial AI
Traditional supply chain risks and AI
Risks from outdated and vulnerable components
Risks from AI's dependency on live data
Securing AI from vulnerable components
Enhanced security - allow approved-only packages
Client configuration for private PyPI repositories
Additional private PyPI security
Use of SBOMs
AI supply chain risks
The double-edged sword of transfer learning
Model poisoning
Model tampering
Secure model provenance and governance for pre-trained models
MLOps and private model repositories
Data poisoning
Using data poisoning to affect sentiment analysis
Defenses and mitigations
AI/ML SBOMs
Summary
Part 3: Attacks on Deployed AI
Chapter 7: Evasion Attacks against Deployed AI
Fundamentals of evasion attacks
Importance of understanding evasion attacks
Reconnaissance techniques for evasion attacks
Perturbations and image evasion attack techniques
Evasion attack scenarios
One-step perturbation with FGSM
Basic Iterative Method (BIM)
Jacobian-based Saliency Map Attack (JSMA)
Carlini and Wagner (C&W) attack
Projected Gradient Descent (PGD)
Adversarial patches - bridging digital and physical evasion techniques
NLP evasion attacks with BERT using TextAttack
Attack scenario - sentiment analysis
Attack example
Attack scenario - natural language inference
Attack example
Universal Adversarial Perturbations (UAPs)
Attack scenario
Attack example
Black-box attacks with transferability
Attack scenario
Attack example
Defending against evasion attacks
Mitigation strategies overview
Adversarial training
Input preprocessing
Model hardening techniques
Model ensembles
Certified defenses
Summary
Chapter 8: Privacy Attacks - Stealing Models
Understanding privacy attacks
Stealing models with model extraction attacks
Functionally equivalent extraction
Learning-based model extraction attacks
Generative student-teacher extraction (distillation) attacks
Attack example against our CIFAR-10 CNN
Defenses and mitigations
Prevention measures
Detection measures
Model ownership identification and recovery
Summary
Chapter 9: Privacy Attacks - Stealing Data
Understanding model inversion attacks
Types of model inversion attacks
Exploitation of model confidence scores
GAN-assisted model inversion
Example model inversion attack
Understanding inference attacks
Attribute inference attacks
Meta-classifiers
Poisoning-assisted inference
Attack scenarios
Mitigations
Example attribute inference attack
Membership inference attacks
Statistical thresholds for ML leaks
Label-only data transferring attack
Blind membership inference attacks
White box attacks
Attack scenarios
Mitigations
Example membership inference attack using the ART
Summary
Chapter 10: Privacy-Preserving AI
Privacy-preserving ML and AI
Simple data anonymization
Advanced anonymization
K-anonymity
Anonymization and geolocation data
Anonymizing rich media
Differential privacy (DP)
Federated learning (FL)
Split learning
Advanced encryption options for privacy-preserving ML
Secure multi-party computation (secure MPC)
Homomorphic encryption
Advanced ML encryption techniques in practice
Applying privacy-preserving ML techniques
Summary
Part 4: Generative AI and Adversarial Attacks
Chapter 11: Generative AI - A New Frontier
A brief introduction to generative AI
A brief history of the evolution of generative AI
Generative AI technologies
Using GANs
Developing a GAN from scratch
WGANs and custom loss functions
Using pre-trained GANs
Pix2Pix
CycleGAN
Pix2PixHD
Progressive Growing of GANs (PGGAN)
BigGAN
StarGAN v2
StyleGAN series
Summary
Chapter 12: Weaponizing GANs for Deepfakes and Adversarial Attacks
Use of GANs for deepfakes and deepfake detection
Using StyleGAN to generate convincing fake images
Creating simple deepfakes with GANs using existing images
Making direct changes to an existing image
Using Pix2PixHD to synthesize images
Fake videos and animations
Other AI deepfake technologies
Voice deepfakes
Deepfake detection
Using GANs in cyberattacks and offensive security
Evading face verification
Compromising biometric authentication
Password cracking with GANs
Malware detection evasion
GANs in cryptography and stenography
Generating web attack payloads with GANs
Generating adversarial attack payloads
Defenses and mitigations
Securing GANs
GAN-assisted adversarial attacks
Deepfakes, malicious content, and misinformation
Summary
Chapter 13: LLM Foundations for Adversarial AI
A brief introduction to LLMs
Developing AI applications with LLMs
Hello LLM with Python
Hello LLM with LangChain
Bringing your own data
How LLMs change Adversarial AI
Summary
Chapter 14: Adversarial Attacks with Prompts
Adversarial inputs and prompt injection
Direct prompt injection
Prompt override
Style injection
Role-playing
Impersonation
Other jailbreaking techniques
Automated gradient-based prompt injection
Risks from bringing your own data
Indirect prompt injection
Data exfiltration with prompt injection
Privilege escalation with prompt injection
RCE with prompt injection
Defenses and mitigations
LLM platform defenses
Application-level defenses
Summary
Chapter 15: Poisoning Attacks and LLMs
Poisoning embeddings in RAG
Attack scenarios
Poisoning during embedding generation
Direct embeddings poisoning
Advanced embeddings poisoning
Query embeddings manipulation
Defenses and mitigations
Poisoning attacks on fine-tuning LLMs
Introduction to fine-tuning LLMs
Fine-tuning poisoning attack scenarios
Fine-tuning attack vectors
Poisoning ChatGPT 3.5 with fine-tuning
Defenses and mitigations against poisoning attacks in fine-tuning
Summary
Chapter 16: Advanced Generative AI Scenarios
Supply-chain attacks in LLMs
Publishing a poisoned LLM on Hugging Face
Publishing a tampered LLM on Hugging Face
Other supply-chain risks for LLMs
Supply-chain defenses and mitigations
Privacy attacks and LLMs
Model inversion and training data extraction attacks on LLMs
Inference attacks on LLMs
Model cloning with LLMs using a secondary model
Defenses and mitigations for privacy attacks
Summary
Part 5: Secure-by-Design AI and MLSecOps
Chapter 17: Secure by Design and Trustworthy AI
Secure by design AI
Building our threat library
Traditional cyber security threats
Adversarial AI attacks
Adversarial AI attacks specific to Generative AI
Supply chain attacks
Industry AI threat taxonomies
AI threat taxonomy mapping
NIST AI taxonomy mapping
AI Exchange mapping
MITRE ATLAS mapping
Threat modeling for AI
Threat modelling in action
Example AI solution
Enhanced FoodieAI threat model
Risk assessment and prioritization
Applying risk assessment to Enhanced FoodieAI
Security design and implementation
Testing and verification
Shifting left - embedding security into the AI life cycle
Live operations
Beyond security - Trustworthy AI
Summary
Chapter 18: AI Security with MLSecOps
The MLSecOps imperative
Toward an MLSecOps 2.0 framework
MLSecOps orchestration options
MLSecOps patterns
Building a primary MLSecOPs platform
MLSecOps in action
Model sourcing and validation
Integrating MLSecOps with LLMOps
Advanced MLSecOps with SBOMs
Summary
Chapter 19: Maturing AI Security
Enterprise security AI challenges
Foundations of enterprise AI security
Protecting AI with enterprise security
Operational AI security
Iterative enterprise security
Summary
Chapter 20: Unlock Your Book's Exclusive Benefits
Index
Other Books You May Enjoy

Preface

The rise of AI is a new revolution in the making, transforming our lives. Alongside the phenomenal opportunities, new risks and threats are emerging, especially in the area of security, and new skills are demanded to safeguard AI systems. This is because some of these threats manipulate the very essence of how AI works to trick AI systems. We call this adversarial AI, and this book will walk you through techniques, examples, and countermeasures. We will explore them from both offensive and defensive perspectives; we will act as an attacker, staging attacks to demonstrate the threats and then discussing how to mitigate them.

Understanding adversarial AI and defending against it poses new challenges for cybersecurity professionals because they require an understanding of AI and Machine Learning (ML) techniques. The book assumes you have no ML or AI expertise, which will be true for most cybersecurity professionals. Although it will not make you a data scientist, the book will help you build a foundational hands-on understanding of ML and AI, enough to understand and detect adversarial AI attacks and defend against them.

AI has evolved. Its first wave covered predictive (or discriminative) AI with models classifying or predicting values from inputs. This is now mainstream, and we use it every day on our smartphones, for passport checks, at hospitals, and with home assistants. We will cover attacks on this strand of AI before we move to the next frontier of AI, generative AI, which creates new content. We will cover Generative Adversarial Networks (GANs), deepfakes, and the new revolution of Large Language Models (LLMs) such as ChatGPT.

The book strives to be hands-on, but adversarial AI is an evolving research topic. Thousands of research papers have been published detailing experiments in lab conditions. We will try to group this research into concrete themes while providing plenty of references for you to dive into for more details.

We will wrap up our journey with a methodology for secure-by-design AI with core elements such as threat modeling and MLSecOps, while looking at Trustworthy AI.

The book is detailed and demanding at times, asking for your full attention. The reward, however, is high. You will gain an in-depth understanding of AI and its advanced security challenges. In our changing times, this is essential to safeguard AI against its abusers.

Who this book is for

The book is for cybersecurity professionals, such as security architects, analysts, engineers, ethical hackers, penetration testers, and incident responders, but also developers and engineers designing, building, and assuring AI systems.

A basic understanding of security concepts is beneficial, and a hacking and tinkering mindset, especially using Python, is the ideal background.

What this book covers

Chapter 1, Getting Started with AI, covers key concepts and terms surrounding AI and ML to get us started with adversarial AI.

Chapter 2, Building Our Adversarial Playground, goes through the step-by-step setup of our environment and the creation of some basic models and our sample Image Recognition Service (ImRecS).

Chapter 3, Security and Adversarial AI, discusses how to apply traditional cybersecurity to our sample ImRecS and bypass it with a sample adversarial AI attack.

Chapter 4, Poisoning Attacks, covers poisoning data and models, and how to mitigate them with examples from our ImRecS.

Chapter 5, Model Tampering with Trojan Horses and Model Reprogramming, looks at changing models by embedding code-based Trojan horses and how to defend against them.

Chapter 6, Supply Chain Attacks and Adversarial AI, covers traditional and new AI supply chain risks and mitigations, including building our own private package repository.

Chapter 7, Evasion Attacks against Deployed AI, explores fooling AI systems with evasion attacks and how to defend against them.

Chapter 8, Privacy Attacks - Stealing Models, looks at model extraction attacks to replicate models and how to mitigate these attacks, including watermarking.

Chapter 9, Privacy Attacks - Stealing Data, looks at model inversion and inference attacks to reconstruct or infer sensitive data from model responses.

Chapter 10, Privacy-Preserving AI, discusses techniques for preserving privacy in AI, including anonymization, differential privacy, homomorphic encryption, federated learning, and secure multi-party computations.

Chapter 11, Generative AI - A New Frontier, provides a hands-on introduction to generative AI with a focus on GANs.

Chapter 12, Weaponizing GANs for Deepfakes and Adversarial Attacks, provides an exploration of how to use GANs to support adversarial attacks, including deepfakes, and how to mitigate these attacks.

Chapter 13, LLM Foundations for Adversarial AI, provides a hands-on introduction to LLMs using the OpenAI API and LangChain to create our sample Foodie AI bot with RAG.

Chapter 14, Adversarial Attacks with Prompts, explores prompt injections against LLMs and how to mitigate them

Chapter 15, Poisoning Attacks and LLMs, looks at poisoning attacks with RAG, embeddings, and fine-tuning, using Foodie AI as an example, and appropriate defenses.

Chapter 16, Advanced Generative AI Scenarios, looks at poisoning the open source LLM Mistral with fine-tuning on Hugging Face, model lobotomization, replication, and inversion and inference attacks on LLMs.

Chapter 17, Secure by Design and Trustworthy AI, explores a methodology using standards-based taxonomies, threat modeling, and risk management to build secure AI with a case study combining predictive AI and LLMs.

Chapter 18, AI Security with MLSecOps, looks at MLSecOps patterns with examples of how to apply them using Jenkins, MLflow, and custom Python scripts.

Chapter 19, Maturing AI Security, discusses applying AI security governance and evolving AI security at an enterprise level.

To get the most out of this book

To follow along with the code, you will need a computer running Windows 10 or 11, macOS, or Linux with at least 16 GB of RAM. Windows users should use the Windows Subsystem for Linux 2 (WSL2) and Ubuntu 20.04. Alternatively, cloud solutions such as Colab or AWS SageMaker notebook instances will provide the processing power you will need. In all cases, you should have a basic understanding of a Bash command-line environment.

Most examples use Python 3.x, virtual environments, pip packages, and Jupyter notebooks. Chapter 2 will take you step by step through setting up the Python environments. Additionally, we will use Docker custom image files and Docker Compose files but we will provide detailed commands and scripts.

To edit or run the examples, you must have a browser or an IDE that supports Jupyter Notebook, such as Visual Studio Code or IntelliJ PyCharm. Both are free and can be found at https://code.visualstudio.com and https://www.jetbrains.com/pycharm, respectively. A browser will be more than sufficient for the examples in this chapter.

Software/hardware covered in the book

Operating system requirements

Python 3.x, TensorFlow 2.x with Keras

Windows, macOS, or Linux

OpenAI and Hugging Face APIs

LangChain

Docker

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book's GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files...

Systemvoraussetzungen

Als PDF speichern Als Link merken