Masked Autoencoders Are Scalable Vision Learners

Masked autoencoders hide most image patches before a Vision Transformer encoder, then use a lightweight decoder to reconstruct only the missing pixels. Read it to see why the 75 percent masking ratio saves encoder compute, how the asymmetric design changes representation learning, and which ImageNet and COCO results support the paper's claims.

Reading focus: Why the large encoder processes visible patches only and mask tokens appear after it. How a high masking ratio changes both the reconstruction task and the amount of pretraining compute. Why strong fine-tuning and transfer results do not prove that pixel reconstruction itself discovers semantic objects.

CVPR 2022. Kaiming He et al.. 28 min read, medium difficulty.