---
title: "How do you turn an LLM experiment into a publishable AI research paper?"
description: "Build an LLM, compare Muon and Adam, structure experiments, and turn the evidence into a publishable AI research paper."
canonical_url: "https://fanout.sh/ai/lessons/code-write-publish-ai-research-paper"
md_url: "https://fanout.sh/ai/lessons/code-write-publish-ai-research-paper.md"
access: "public"
---

# How do you turn an LLM experiment into a publishable AI research paper?

Build an LLM, compare Muon and Adam, structure experiments, and turn the evidence into a publishable AI research paper.

## Public overview

A research paper makes a claim and gives other people enough evidence to check it. Code, plots and polished writing help only when they support that claim.

The linked video follows an optimizer comparison from implementation to a paper. We can practice the same workflow with a much smaller question: on one small regression task, after equal numbers of training steps, how do AdamW and a Muon-based optimizer compare?

That question names the task, comparison and budget. It does not promise a new best optimizer.

Decide what the result would mean

Our model receives two numbers and predicts their product. It has one hidden layer with eight units. The loss is mean squared error: average the squared differences between predictions and correct products. Lower is better.

AdamW adjusts parameters using moving estimates of gradients and their squared values. Muon applies a matrix operation to a momentum update before changing the weights. Momentum carries information from earlier gradients into the current update. We use the library implementation here; the original Muon explanation derives its matrix operation if you want to inspect it.

Muon is intended for appropriate matrix weights, with other parameters handled separately. Here, “Muon hybrid” means Muon on the hidden weight matrix and AdamW on its bias and the output layer. That distinction must appear in a methods section; calling it simply “Muon on the model” hides part of the experiment.

Before running, fix these choices:

Choice Our pilot --- --- Data 64 training, 32 validation and 32 test examples Training budget 80 full-batch updates per run Tuning budget Two candidate learning rates per method, two seeds each Final comparison Three new seeds, using the selected rates Other settings Same model, data and zero weight decay; hybrid's AdamW rate fixed at 0.01

Matching seeds makes the two methods start from the same weights within a pair. Validation data chooses learning rates. Test data is consulted only after those rates are locked. The candidate ranges below are a small demonstration, not an exhaustive or equally optimal search for each method.

Run an experiment small enough to audit

This CPU program was checked with PyTorch 2.14. It needs a PyTorch release containing torch.optim.Muon ; it downloads no dataset or pretrained weights.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md