Understanding deep learning requires rethinking…

The random-label control shows that a network can fit arbitrary training targets while performing near chance on held-out examples. Read it to distinguish fitting from generalization, understand what stays fixed in the experiment, and avoid overclaims about regularization or learning theory.

Reading focus: Why random targets must stay fixed across epochs for the paper’s memorization control. How equal training accuracy can accompany very different held-out accuracy in a matched comparison. Why the results challenge coarse explanations without proving that regularization or learning theory is useless.

ICLR 2017. Chiyuan Zhang et al.. 32 min read, medium difficulty.