---
title: "Let's Verify Step by Step"
description: "Let's Verify Step by Step compares reward models trained on a final answer with reward models trained to judge every reasoning step. Read it to see why first-error labels improve credit assignment, how best-of-N exposes verifier quality without changing the generator, and why the headline result needs a controlled companion experiment."
canonical_url: "https://fanout.sh/daily/2026-10-18-lets-verify-step-by-step"
md_url: "https://fanout.sh/daily/2026-10-18-lets-verify-step-by-step.md"
last_updated: "2026-10-18"
access: "public"
---

# Let's Verify Step by Step

Let's Verify Step by Step compares reward models trained on a final answer with reward models trained to judge every reasoning step. Read it to see why first-error labels improve credit assignment, how best-of-N exposes verifier quality without changing the generator, and why the headline result needs a controlled companion experiment.

## Paper details

- Authors: Hunter Lightman et al.

- Venue: ICLR 2024

- Track: AI Research

- Difficulty: Medium

- Reading time: 32 min

- Original paper: https://arxiv.org/abs/2305.20050v1

## What you will learn

- How process supervision preserves a valid prefix while locating the first broken reasoning step.

- Why product scoring turns many step judgments into one candidate rank for best-of-N selection.

- Which results isolate supervision form and which remain confounded by different training sets.

## Continue exploring

[Explore reward models](https://fanout.sh/knowledge-graph?node=ai-research%3Aconcept%3Areward-model): Continue from process supervision into Fanout's AI research concept graph for reward models.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md