---
title: "Constitutional AI: Harmlessness from AI Feedback"
description: "Constitutional AI asks whether a short, explicit set of principles can replace repeated human harmlessness labels without reducing safety to blanket refusal. Read it to see how one rulebook produces self-revised demonstrations and AI-labelled response preferences, why the policy still depends on human helpfulness data and evaluation, and how inaccurate critiques, judge capability, principle choice, and reward over-optimization bound the result."
canonical_url: "https://fanout.sh/daily/2026-09-23-constitutional-ai-feedback"
md_url: "https://fanout.sh/daily/2026-09-23-constitutional-ai-feedback.md"
last_updated: "2026-09-23"
access: "public"
---

# Constitutional AI: Harmlessness from AI Feedback

Constitutional AI asks whether a short, explicit set of principles can replace repeated human harmlessness labels without reducing safety to blanket refusal. Read it to see how one rulebook produces self-revised demonstrations and AI-labelled response preferences, why the policy still depends on human helpfulness data and evaluation, and how inaccurate critiques, judge capability, principle choice, and reward over-optimization bound the result.

## Paper details

- Authors: Yuntao Bai et al.

- Venue: Anthropic technical report, 2022

- Track: AI Research

- Difficulty: Medium

- Reading time: 35 min

- Original paper: https://arxiv.org/abs/2212.08073

## What you will learn

- How sampled principles turn a model's first response into critiques, revisions, and supervised training targets.

- How an AI feedback model converts the same kind of principles into soft pairwise labels for a preference model and reinforcement learning.

- Why removing direct human harmlessness labels does not remove human values, evaluation, red-team prompts, or reward-model failure modes.

## Continue exploring

[Compare the RLHF pipeline in Fanout](https://fanout.sh/daily/2026-07-31-instructgpt): Compare Constitutional AI's source of harmlessness preferences with InstructGPT's human-ranked responses, while keeping the supervised, reward-model, and policy-optimization stages distinct.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md