Free access
Proceedings
Proceedings of the 2023 SIAM International Conference on Data Mining (SDM)

Robust Learning via Golden Symmetric Loss of (un)Trusted Labels

Abstract

Learning robust deep models against noisy labels becomes ever critical when today's data is commonly collected from open platforms and subject to adversarial corruption. The information on the label corruption process, i.e., corruption matrix, can greatly enhance the robustness of deep models but still fall behind in combating hard classes. In this paper, we propose to construct a golden symmetric loss (GSL) based on the estimated corruption matrix as to avoid overfitting to noisy labels and learn effectively from hard classes. GSL is the weighted sum of the corrected regular cross entropy and reverse cross entropy. By leveraging a small fraction of trusted clean data, we estimate the corruption matrix and use it to correct the loss as well as to determine the weights of GSL. We theoretically prove the robustness of the proposed loss function in the presence of dirty labels. We provide a heuristics to adaptively tune the loss weights of GSL according to the noise rate and diversity measured from the dataset. We evaluate our proposed golden symmetric loss on both vision and natural language deep models subject to different types of label noise patterns. Empirical results show that GSL can significantly outperform the existing robust training methods on different noise patterns, showing accuracy improvement up to 18% on CIFAR-100 and 1% on real world noisy dataset of Clothing1M.

Formats available

You can view the full content in the following formats:

Information & Authors

Information

Published In

cover image Proceedings
Proceedings of the 2023 SIAM International Conference on Data Mining (SDM)
Pages: 568 - 576
Editors: Shashi Shekhar, University of Minnesota, U.S.A., Zhi-Hua Zhou, Nanjing University, China, Yao-Yi Chiang, University of Minnesota, U.S.A., and Gregor Stiglic, University of Maribor, Slovenia
ISBN (Online): 978-1-61197-765-3

History

Published online: 12 April 2023

Keywords

  1. Robust training
  2. Deep learning models
  3. Symmetric loss function
  4. Noisy labels

Authors

Affiliations

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

Cited By

View Options

View options

PDF

View PDF

Figures

Tables

Media

Share

Share

Copy the content Link

Share with email

Email a colleague

Share on social media