How a language model is trained.

Nobody writes the rules an AI agent follows. There is no file where an engineer typed “help other agents” or “don’t break into Hugging Face.” The model’s behavior is the residue of a training process — three broad stages that turn a pile of numbers into something that can hold a conversation, use tools, and pursue goals. Understanding those stages is the difference between “the AI decided to do X” and “the training produced a system that does X under these conditions.”

Stage 1 — Pretraining: learning to predict

A model starts as billions of random parameters (also called weights) — dials with no meaning. Pretraining shows it an enormous amount of text and sets it one relentless task: predict the next token (a token is roughly a word-piece). Guess wrong, and the dials nudge a little to make the right guess more likely next time. Repeat that trillions of times.

To predict text well across the whole internet, the model is forced to absorb an implicit model of the world: grammar, facts, code, arithmetic, argument, the shape of a login page. What comes out is a base model — fluent, knowledgeable, and completely unfocused. It will happily continue a sentence, but it has no notion of being “helpful” or of a task being “finished.” It is pure autocomplete, wearing everything it read.

Stage 2 — Supervised fine-tuning: learning the format

Next, the base model is shown a much smaller, curated set of demonstrations: examples of a request followed by a good response. This is supervised fine-tuning (SFT) — still next-token prediction, but now on hand-picked “this is how to behave” transcripts. SFT is what teaches a model to answer a question instead of writing five more questions, to follow an instruction, to produce a tool call in the right syntax.

SFT sets the style and the interface. But you can’t write a demonstration for every situation, and demonstrations only teach imitation — “sound like these examples.” They don’t teach the model to actually succeed at open-ended tasks it’s never seen. For that you need feedback on the model’s own attempts.

Stage 3 — RL post-training: learning what works

In the final stage, the model tries tasks itself, and a reward signal scores each attempt. Attempts that score well are reinforced; attempts that score poorly are discouraged. This is reinforcement learning, and it’s where most of a modern agent’s “drive” comes from — its tendency to keep going, to route around obstacles, to actually get the reward rather than merely look plausible.

The reward can come from humans rating answers (RLHF — reinforcement learning from human feedback), which shaped the polite, helpful tone you’re used to. Increasingly it also comes from automatic checkers on tasks with a right answer — did the code pass the tests? did the agent capture the flag? This is what turns a chatbot into a goal-seeker. It is also, as the next primers explain, exactly where things go wrong: a model optimized to make a number go up will make that number go up — by any route it can find.

Key terms

worth knowing before the dispatch

parameters / weights

The billions of tunable numbers that are the model. Training just adjusts these.

token

A chunk of text (about a word-piece). Models read and write in tokens, not letters.

base model

The raw result of pretraining: fluent, knowledgeable, but not yet helpful or goal-directed.

SFT

Supervised fine-tuning — teaching format and behavior by imitation of curated examples.

RLHF / RL post-training

Reinforcing the model’s own attempts based on a reward. Where goal-seeking is forged.