Benign Overfitting Theory Extended to Adversarial Training of Vision Transformers

arXiv·medium signal

Zhang, Ding, and Fu establish theoretical conditions under which Vision Transformers exhibit benign overfitting during adversarial training — meaning they can perfectly fit training data (including adversarial perturbations) while still generalizing well. This provides theoretical backing for why adversarial training works on ViTs despite apparent overfitting, and identifies regimes where adversarially-trained ViTs maintain both robustness and accuracy.