Label smoothing

1. What does label smoothing mean?
2. What does label smoothing help?
3. Should I use label smoothing?
4. What is one sided label smoothing?

What does label smoothing mean?

Label smoothing is a form of output distribution regularization that prevents overfitting of a neural network by softening the ground-truth labels in the training data in an attempt to penalize overconfident outputs.

What does label smoothing help?

Label smoothing has been used successfully to improve the accuracy of deep learning models across a range of tasks, including image classification, speech recognition, and machine translation (Table 1).

Should I use label smoothing?

Q: When do we use label smoothing? A: Whenever a classification neural network suffers from overfitting and/or overconfidence, we can try label smoothing.

What is one sided label smoothing?

One-sided label smoothing

In GAN, if the discriminator depends on a small set of features to detect real images, the generator may just produce these features only to exploit the discriminator. The optimization may turn too greedy and produces no long term benefit. In GAN, overconfidence hurts badly.