All projects
May 2025
Revealing Weaknesses in Google reCAPTCHA Using Adversarial ML
reCAPTCHA decides whether a visitor is human using machine-learning models, which makes it a target for attacks on machine learning itself. This project assessed Google reCAPTCHA v2 and v3 against adversarial ML techniques and examined the privacy risks of the models behind it.
PythonAdversarial MLSecurityFGSMPGD
Assessment approach
01
Target
reCAPTCHA v2 image challenges and v3 invisible risk scores
02
Evasion
Adversarial perturbations with FGSM, PGD and SimBA
03
Privacy
Model extraction and membership inference risks
04
Defence
Adversarial training, federated learning hardening, privacy-aware design
Attack techniques
- ▸FGSM (Fast Gradient Sign Method): a single, small change to every pixel in the direction that most increases the model's error. It is invisible to people, but enough to change the model's answer
- ▸PGD (Projected Gradient Descent): a stronger, step-by-step version of FGSM that keeps the change within a tight limit
- ▸SimBA (Simple Black-box Attack): works without access to the model's internals, using only its answers to queries. This is closer to what a real attacker facing a hosted service can do
Privacy risks
- ▸Model extraction: repeatedly querying a model to build a working copy of it, which can then be studied offline to craft attacks
- ▸Membership inference: working out whether a particular record was part of a model's training data, which can leak information about the people in it
Proposed mitigations
- ▸Adversarial training, so the model learns from perturbed examples and becomes harder to fool
- ▸Hardening federated learning, where models are trained across many devices, against poisoned or leaking updates
- ▸Privacy-aware design that limits what can be learned about the model and its training data from its outputs