How do you build a reproducible ML pipeline with DVC and AWS S3?

Build a reproducible spam-detection pipeline with DVC stages, versioned artifacts, and AWS S3 remote storage.

A small spam classifier has three steps: normalize training text, learn a word rule, then evaluate it. You can run three Python scripts by hand. But when one input changes, which scripts need to run again?

A pipeline makes that relationship explicit. DVC records each stage's command, dependencies and outputs, then uses those declarations to decide what is out of date.

Make the files tell the story

Use a fresh directory with Python and DVC installed. The following tiny classifier is for inspecting pipeline behavior, not for filtering real mail.

Save these training examples as raw.json :

Save separate evaluation examples as test.json :

The preparation script, prepare.py , turns each text into lowercase tokens while preserving its label:

python import json from pathlib import Path

rows = json.loads(Path("raw.json").read text()) prepared = [{"tokens": row["text"].lower().split(), "spam": row["spam"]} for row in rows] Path("train.json").write text(json.dumps(prepared, indent=2))

The fitting script, fit.py , remembers words seen in spam but not in non-spam training examples. That learned set is our model:

python import json from pathlib import Path

rows = json.loads(Path("train.json").read text()) positive, negative = set(), set() for row in rows: words = positive if row["spam"] else negative words.update(row["tokens"]) model = {"spam words": sorted(positive - negative)} Path("model.json").write text(json.dumps(model, indent=2))