| Title | On Connections Between Regularizations for Improving DNN Robustness |
| Authors | Guo, Yiwen Chen, Long Chen, Yurong Zhang, Changshui |
| Affiliation | Bytedance AI Lab, Beijing 100000, Peoples R China Peking Univ, Acad Adv Interdisciplinary Studies, Ctr Data Sci, Beijing 100871, Peoples R China Intel Labs, Beijing 100190, Peoples R China Tsinghua Univ, Tsinghua Univ THUAI, Inst Artificial Intelligence,Dept Automat, State Key Lab Intelligent Technol & Syst,Beijing, Beijing 100084, Peoples R China |
| Issue Date | 1-Dec-2021 |
| Publisher | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE |
| Abstract | This paper analyzes regularization terms proposed recently for improving the adversarial robustness of deep neural networks (DNNs), from a theoretical point of view. Specifically, we study possible connections between several effective methods, including input-gradient regularization, Jacobian regularization, curvature regularization, and a cross-Lipschitz functional. We investigate them on DNNs with general rectified linear activations, which constitute one of the most prevalent families of models for image classification and a host of other machine learning applications. We shed light on essential ingredients of these regularizations and re-interpret their functionality. Through the lens of our study, more principled and efficient regularizations can possibly be invented in the near future. |
| URI | http://hdl.handle.net/20.500.11897/629074 |
| ISSN | 0162-8828 |
| DOI | 10.1109/TPAMI.2020.3006917 |
| Indexed | SCI(E) |
| Appears in Collections: | 前沿交叉学科研究院 |
