---
title: "Neural Machine Translation by Jointly Learning to Align and Translate"
description: "This paper asks how a translation model can look back at the right source words instead of compressing an entire sentence into one fixed vector. Read it to see how a bidirectional encoder leaves one annotation at every source position, how a jointly trained alignment scorer builds a fresh weighted context for each target word, and why strong WMT14 results still need careful treatment of unknown words, sequence length, exhaustive scoring, and causal claims."
canonical_url: "https://fanout.sh/daily/2026-09-16-neural-translation-soft-attention"
md_url: "https://fanout.sh/daily/2026-09-16-neural-translation-soft-attention.md"
last_updated: "2026-09-16"
access: "public"
---

# Neural Machine Translation by Jointly Learning to Align and Translate

This paper asks how a translation model can look back at the right source words instead of compressing an entire sentence into one fixed vector. Read it to see how a bidirectional encoder leaves one annotation at every source position, how a jointly trained alignment scorer builds a fresh weighted context for each target word, and why strong WMT14 results still need careful treatment of unknown words, sequence length, exhaustive scoring, and causal claims.

## Paper details

- Authors: Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio

- Venue: ICLR 2015

- Track: AI Research

- Difficulty: Easy

- Reading time: 30 min

- Original paper: https://arxiv.org/abs/1409.0473

## What you will learn

- Why one fixed context vector can become a bottleneck, and how per-position source annotations keep the sentence addressable.

- How alignment scores, softmax weights, and a weighted sum produce a different context vector for every target word.

- What the WMT14 BLEU and long-sentence results establish, and where vocabulary, compute, and architectural confounders limit the conclusion.

## Continue exploring

[Explore attention in Fanout](https://fanout.sh/knowledge-graph?node=ai-research%3Aconcept%3Aattention): Connect this recurrent target-to-source mechanism to Fanout's attention concept map, then continue into self-attention, cross-attention, Transformers, and modern language models.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md