Gradient Masking

Gradient masking is a technique in machine learning, particularly in adversarial machine learning, where a model's gradients are intentionally obscured or made uninformative to prevent gradient-based attacks. It involves modifying the model's architecture or training process to produce flat or noisy gradients, making it difficult for attackers to craft adversarial examples using gradient information. This concept is often discussed in the context of defending against adversarial attacks like the Fast Gradient Sign Method (FGSM) or Projected Gradient Descent (PGD).

Also known as: Gradient Obfuscation, Gradient Hiding, Gradient Flattening, Gradient Noise, Adversarial Gradient Masking

Why learn Gradient Masking?

Developers should learn about gradient masking when building robust machine learning models that need to resist adversarial attacks, such as in security-critical applications like autonomous vehicles, fraud detection, or medical diagnosis systems. It is used to enhance model security by preventing attackers from exploiting gradient information to generate adversarial inputs that cause misclassification. However, it's important to note that gradient masking is often considered a weak defense, as it can be circumvented by more sophisticated attacks, so it should be combined with other techniques like adversarial training.

Learning Resources

• docs: Adversarial Machine Learning at Scale
• docs: Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• video: Adversarial Attacks and Defenses in Deep Learning
• course: Machine Learning Security - Adversarial Examples and Defenses
• docs: Towards Deep Learning Models Resistant to Adversarial Attacks