An Image is Worth 16x16 Words: Transformers for…

Vision Transformer asks how much image-specific structure a model needs when pre-training data becomes very large. Read it to see how a 224-pixel image becomes 196 patch tokens, which convolutional priors disappear, why ResNets lead at smaller scale, where ViT crosses over, and how patch size turns visual detail into a quadratic token-pair bill.

Reading focus: How non-overlapping image patches, a shared projection, learned positions, and one class token create a Transformer sequence. Why removing locality and translation equivariance hurts at smaller data scales but can pay off after large-scale pre-training. How to read the data crossover, compute comparison, attention analysis, and private JFT dependence without turning them into a universal no-convolution claim.

ICLR 2021. Alexey Dosovitskiy et al.. 30 min read, easy difficulty.