aifromzero.dev

Data Is the Model: Curation and Chat Templates (Fine-Tuning in Production, Module 2)

Module 2 of 15 14 min read Lab code ↗

This is Module 2 of Fine-Tuning in Production, a free 15-module course that takes one open model from raw data to a deployed, published tool-calling specialist on the Hugging Face Hub. Start at Module 1 or browse the full syllabus.

The bug that trains perfectly and ships garbage

In Module 1 you handed Qwen3-4B-Base a user request and a couple of tool schemas, and it failed. It wrote prose where you wanted a tool call, or invented a function that didn’t exist, or emitted JSON with a trailing comma the parser choked on. The base model has read most of the internet, but it was never taught to answer with a structured call — that’s the gap this course closes.

The beginner reflex at this point is “I need a better trainer, or more epochs.” It’s the wrong instinct, and it’s an expensive one. A fine-tuned model learns exactly the distribution you show it. If your data is dirty, or — worse — if it’s formatted in a way the model never saw at pretraining, no SFTTrainer setting on earth will save you. Garbage in, garbage out is the first law of fine-tuning, and the nastiest version of “garbage” isn’t bad content. It’s bad format. Hand-roll a prompt that drifts one special token from what the model expects, and training “works” — the loss goes down, the curves look healthy — while the served model produces silent nonsense. This module lays Anvil’s data foundation: clean, in the right schema, and templated by the tokenizer itself, never by you.

In this module

You’ll learn how to:

  1. Explain why fine-tuning is garbage-in, garbage-out — the model learns the distribution of your data, so curation and format matter more than the trainer you pick.
  2. Distinguish an instruction dataset (a plain chat) from a tool-calling dataset (messages plus a set of available tools the model may call).
  3. Apply the chat template the right way — tokenizer.apply_chat_template(messages, tools=..., add_generation_prompt=..., tokenize=False) — and read what it produced: special tokens, the serialized tools block, the EOS that teaches the model when to stop.
  4. Build clean train/eval splits with the datasets library (5.x) and explain why a leak between them silently inflates every metric you’ll compute in Module 7.
  5. Audit a dataset for duplicates and train/eval contamination before a single GPU-hour is spent — and verify the source dataset’s license before redistributing it.

You’ll build: anvil/data.py — load the tool-calling fixture dataset, reformat every example to Anvil’s frozen {messages, tools} schema, apply Qwen3’s chat template with apply_chat_template(..., tools=...), split into a held-out train/eval, and print one fully tokenized example so you can see exactly what the model will be trained on.

Theory areas covered:

  • T2 — Data & chat templates — 9% of the theory map. This module owns T2 end to end: curation, instruction vs tool-calling formats, chat templates, splits and contamination, and the datasets library.

Prerequisites: Module 1 — the pinned stack installed (datasets==5.0.0, transformers==5.12.0, and the rest), HF_TOKEN in your .env, a working Colab/Kaggle T4 notebook, and the baseline Qwen3-4B-Base that fails the tool-call format. No GPU is required for this module — data preparation runs on CPU. You need HF_TOKEN only to download the tokenizer and the source dataset.

Where you are

This is Module 2 of 15. From the road map:

  • ✅ M1 — When (Not) to Fine-Tune. Setup, the pinned stack, and the baseline: Qwen3-4B-Base fails the tool-call format.
  • 👉 M2 — Data Is the Model. anvil/data.py: a frozen {messages, tools} dataset, templated by the tokenizer, split clean.
  • ⬜ M3 — Your First Fine-Tune (SFT). Feed this dataset to an SFTTrainer and watch Anvil v0 learn.
  • ⬜ M4–M15. LoRA, QLoRA, evaluation, DPO, GRPO, merging, quantization, serving, publishing, capstone.

Anvil’s state going in: a base model that fails the format, and no data to teach it. Coming out: a frozen {messages, tools} dataset, templated by the tokenizer, split clean. This is the load-bearing module of the whole course. anvil/data.py is reused unchanged by SFT (M3), evaluation (M7), DPO (M8), GRPO (M10), and the capstone (M15). Everything downstream consumes what you build here — so a formatting mistake made now propagates, in silence, through every training run you do for the next thirteen modules.

Garbage in, garbage out: why data is the model

Here is the thesis the rest of this course rests on. In supervised fine-tuning (you’ll do it for real in Module 3), the model does next-token prediction on your demonstrations. It is, quite literally, learning to imitate the distribution of your dataset — token by token. There is no separate “knowledge” the trainer injects. The data is the model. Change the data and you change the model; keep the trainer fixed and improve the data, and the model improves. That asymmetry is why senior practitioners obsess over data and barely touch hyperparameters.

So here’s an opinion I’ll defend: I’d rather have 2,000 hand-checked examples than 200,000 scraped ones. The second batch teaches the model your noise — every malformed call, every wrong tool name, every truncated argument — with the same diligence it learns the good examples. The model can’t tell which demonstrations you meant and which slipped through. It learns all of them.

This recipe — human (or high-quality) demonstrations turned into a supervised fine-tune — is the lineage of every modern instruction dataset. It traces back to InstructGPT (arXiv 2203.02155), which showed that supervised fine-tuning on demonstrations, then preference optimization, turns a raw language model into one that follows instructions. We’re citing it for the demonstrations-to-SFT idea only; the SFT algorithm itself is Module 3, and the preference/alignment half is Module 8.

The two families of data formats

Almost every post-training dataset falls into one of two shapes, and knowing which one you have decides everything downstream.

An instruction dataset (or chat dataset) is a list of messages — each a {role, content} pair where role is system, user, or assistant — or the older flat form {instruction, input, output} that Alpaca popularized. The model learns to respond: given the conversation so far, produce the assistant’s reply. That’s the right format for a chatbot, a summarizer, a rewriter.

A tool-calling dataset is the same messages plus a set of tools: JSON-schema descriptions of the functions available for that example. An assistant message can carry a tool call — a tool name plus a JSON arguments object — which is typically followed by a tool message (the result) and then a final answer. The model learns when and how to call a tool. This is Anvil’s format, and we chose it on purpose: a tool call is measurable. You can check whether the JSON parses, whether the tool name is right, whether the arguments match — base model versus fine-tuned, on a held-out set. A style chatbot gives you fuzzy vibes; a tool-caller gives you a number.

Instruction / chat datasetTool-calling dataset
What an example containsmessages (a conversation)messages + tools (available functions)
Roles presentsystem, user, assistantsystem, user, assistant (with tool_calls), tool
What the model learnsHow to respond in natural languageWhen and how to call a tool (and when not to)
How you evaluate itFuzzy: LLM-judge, preference, BLEU/ROUGEHard: JSON validity, tool-name accuracy, argument match
Public examplesAlpaca, OpenHermes, UltraChatSynth-APIGen, ToolACE, Hermes-function-calling

Anvil’s source dataset, argilla/Synth-APIGen-v0.1, is the tool-calling kind. It’s worth seeing the whole pipeline at once before we dive into each box — because every box you get wrong, the model learns as truth:

flowchart LR
    raw["raw dataset<br/>(Synth-APIGen:<br/>query · tools · answers)"] --> reformat["reformat to<br/>{messages, tools}<br/>(Anvil's frozen schema)"]
    reformat --> curate["curate:<br/>filter + dedup +<br/>contamination check"]
    curate --> split["train / eval split<br/>(held-out, fixed seed)"]
    split --> template["apply_chat_template<br/>(tools=...)"]
    template --> tok["the tokenized example<br/>the trainer sees<br/>(special tokens · tools · EOS)"]

    note["every box that's wrong<br/>here, the model learns<br/>as truth"] -.-> curate

The first stage — curation — is the work nobody blogs about and everybody needs. Filter out empty or malformed examples, cap how many tools a single prompt carries (long tool lists blow past your max_length and get truncated mid-schema), balance positives against the “no relevant tool” negatives. We do this in the lab as a real cleaning pass, not a quiz. Curation happens first, before a single split or template, because everything after it inherits whatever you let through.

Chat templates: never hand-roll the format

A chat template is a Jinja string that lives inside the tokenizer (tokenizer.chat_template). It turns your structured list of messages into the exact string this model expects — role delimiters, special tokens, and the position of the tools block. Every model family ships its own. Qwen3, Llama, and Gemma do not use the same one, and the differences are not cosmetic: a model trained with one template produces garbage when prompted with another. The template is the contract between your data and the model’s pretraining distribution.

The canonical API is one call. Here’s the heart of anvil/data.py’s format_example — it renders one {messages, tools} example the one right way:

def format_example(example: dict, tokenizer, *, add_generation_prompt: bool = False) -> str:
    """Render a {messages, tools} example through the chat template — the ONE right way."""
    messages = example["messages"]
    if add_generation_prompt:
        messages = [m for m in messages if m["role"] != "assistant"]
    return tokenizer.apply_chat_template(
        messages, tools=example.get("tools"), tokenize=False,
        add_generation_prompt=add_generation_prompt)

Four arguments do all the work, and each one hides a decision that bites people who don’t understand it.

tools= is the argument that makes Anvil possible. You pass your list of JSON-schema tool definitions, and the template serializes them into the prompt in the exact format Qwen3 expects — a system block with the tools wrapped in <tools>...</tools> and an instruction telling the model to answer calls inside <tool_call>...</tool_call> tags. Without tools=, the model never sees the available functions, so it cannot learn to call them. The schemas would simply vanish, and you’d train a model to guess tools it was never shown. (You can also pass Python functions here; the library introspects their signature and docstring. We pass explicit schemas because our dataset already has them.)

add_generation_prompt is the train-versus-inference switch, and confusing it is one of the most common beginner mistakes. At inference, set it True: the template appends the assistant-turn opener (<|im_start|>assistant) so the model continues from there and generates the reply. At training, set it False: you keep the assistant’s full response in the string, because that response is the target you want the model to imitate. Get this backwards in training and you teach the model to echo the assistant opener instead of producing an answer. Note what format_example does: when add_generation_prompt=True it also drops the assistant message, leaving the prompt half only — that’s what you feed the model at eval time in Module 7.

tokenize=False returns the rendered string, so you can read it with your own eyes. tokenize=True returns input_ids directly. We use False throughout this module for one reason: understanding. In SFT (Module 3), SFTTrainer applies the template itself when you hand it a messages-shaped dataset with a tools column — you don’t re-tokenize by hand. So here we render to a string to see the format; we don’t reinvent the trainer’s job. (There’s also a continue_final_message argument for resuming a partial assistant turn — useful, but out of scope here.)

Special tokens and EOS: how the model learns to stop

Render a positive training example through Qwen3’s template and you can see exactly what the trainer will learn from:

<|im_start|>system
# Tools

You may call one or more functions to assist with the user query.

You are provided with function signatures within <tools></tools> XML tags:
<tools>
{"type": "function", "function": {"name": "search_record_by_name", "description": ...}}
</tools>

For each function call, return a json object ... within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call><|im_end|>
<|im_start|>user
Find the record for a person named 'John Doe' in a list of employee records.<|im_end|>
<|im_start|>assistant
<think>

</think>

<tool_call>
{"name": "search_record_by_name", "arguments": {"records": [...], "name": "John Doe"}}
</tool_call><|im_end|>

Three things to notice. First, the <tools> block landed in a system message — the template put it there, not you. Second, the assistant’s answer is the <tool_call>{...}</tool_call> shape; that string is Anvil’s SFT target. Third, and most important for training: each turn closes with <|im_end|> (id 151645). That’s the turn-closing stop token — what the chat template emits to end every turn, so it’s the token chat-templated SFT teaches the model to produce when it’s done. A dataset whose targets are missing that stop token in the right place trains a model that never knows it’s finished, so it generates until it hits a length cap, looping or babbling. Don’t confuse it with the tokenizer’s nominal EOS: tokenizer.eos_token is <|endoftext|> (id 151643) and tokenizer.eos_token_id is 151643, while the turn-closing token the template uses is <|im_end|> — both are special tokens, but they’re not the same one. (Qwen3 also emits a <think>...</think> block; because Anvil is a tool-calling specialist, not a reasoner, we train on the non-thinking target — an empty think block followed by the call.)

One freshness note specific to base models. A purely pretrained -Base checkpoint sometimes ships a minimal chat template, or none at all, because it was never instruction-tuned. As of transformers 5.12.0, June 2026, Qwen/Qwen3-4B-Base’s tokenizer does carry a full tools=-aware chat template (the Hermes/Qwen style shown above), so we use it directly. If you swap in a base model whose tokenizer lacks a usable tool-calling template, the documented fallback is to use the matching -Instruct variant’s tokenizer, or to set tokenizer.chat_template explicitly to a known-good Jinja string. Read the real tokenizer before you trust it — don’t recite a template from memory.

⚠️ Common misconception: “I can just build the prompt string myself with f-strings — it’s only some <|im_start|> tags.”

No. A hand-rolled template inevitably drifts from the one the model saw at pretraining — a missing space, a special token in the wrong place, the tools block in the wrong slot. The model then learns a language that’s slightly wrong. Training appears to succeed (the loss falls; the curves look fine), and the model served in production emits silent gibberish that no error message ever flags. Always go through tokenizer.apply_chat_template(...). It is the single source of truth for the format, and it lives inside the model’s own tokenizer. This is the number-one trap of the module — and Concept-check question 1.

The whole idea, including the anti-pattern, in one diagram:

flowchart LR
    obj["structured input<br/>messages: [...]<br/>tools: [...]"] --> apply["tokenizer.apply_chat_template(...)<br/><b>single source of truth</b>"]
    apply --> str["templated string<br/>· Qwen3 special tokens<br/>· serialized tools block<br/>· EOS &lt;|im_end|&gt;"]
    str --> ids["input_ids<br/>(what the trainer sees)"]

    obj -.->|"hand-rolled f-string ❌"| drift["drifts from the<br/>training distribution<br/>→ silent garbage in prod"]

Splits, dedup, and contamination: don’t fool your own eval

Before you train anything, you split your data. You hold out an eval set the model never sees during training, so you can measure whether it actually generalizes rather than memorizes. This is the honest metric Module 7 is built on: base model versus fine-tuned, on examples neither has been trained on. Skip the held-out set and the only thing you measure is recall of the training data — a number that always looks great and means nothing.

With the datasets library (5.x), the mechanics are a few methods. You load a dataset (load_dataset(...) from the Hub, or Dataset.from_list(...) from local files), reshape it with .map(), drop junk with .filter(), and carve a reproducible split with .train_test_split(test_size=..., seed=...) — which hands you a DatasetDict whose keys are train and test (rename to eval as you like). The fixed seed is not optional: it’s what makes the split reproducible, which is decision F14 of this course. A split with a random seed is a fine-tune you can never reproduce. As of datasets 5.0.0, June 2026, the library stores data as Arrow under the hood — zero-copy, memory-mapped, so you can operate on datasets larger than RAM; .map(batched=True) reformats fast, and streaming=True handles corpora too big to download (the fixture’s build_data.py uses exactly that to subsample 49k rows without holding them all in memory).

Dedup comes before splitting, always. Public tool-calling datasets are full of exact and near-duplicate examples — the same call with a paraphrased prompt, or the literal same row twice. Duplicates silently over-weight some examples in training (the model sees them more, so it weights them more), but the real danger is the split. If a duplicate of a training example lands in your eval set, the model has effectively seen the answer — your eval is contamination-inflated, reporting a score the model didn’t earn. The rule is one line: split on a stable key, then assert train ∩ eval = ∅. A stable key is a hash of the normalized messages + tools; you dedup on it, then check the two splits don’t intersect on it. In Module 7 you’ll trust these numbers to claim Anvil beats the base model — so earn that trust here.

Anvil ships four conceptual splits, but this module builds only two of them. The other two are named now so the schema has room for them; they’re populated later.

SplitWhat it containsBuilt in M2?Consumed by
train{messages, tools} examples to learn from✅ yesSFT (M3), and re-used by every later trainer
evalHeld-out {messages, tools}, never trained on✅ yesTask-grounded evaluation (M7)
preference(prompt, chosen, rejected) pairs⬜ laterDPO (M8)
grpo_promptsPrompts with verifiable rewards⬜ laterGRPO (M10)

We do not build preference pairs or GRPO prompts here. Constructing a chosen/rejected pair is Module 8’s subject; verifiable-reward prompts are Module 10’s. In Module 2 we lay the structure and fill in train and eval — the two splits SFT needs from day one.

The datasets library and Anvil’s frozen schema

This is where we freeze Anvil’s data contract. From this module on, every example — for SFT, eval, DPO rollouts, GRPO — is this exact shape:

{
    "messages": [{"role": "user" | "assistant" | "tool", "content": str, ...}],
    "tools": [ {"type": "function", "function": {...JSON schema...}} ],
}

The training target is the assistant message carrying the expected tool call (or, for a negative example, a short honest refusal). Why this precise schema and not something more convenient? Because it is exactly what apply_chat_template(messages, tools=...) consumes directly. The same object feeds training (M3+), evaluation (M7), and GRPO rollouts (M10) — one schema, the whole pipeline. This schema is frozen here. Any future change to it is a STOP: you update the fil-rouge spec first, then propagate, never the other way around. That discipline is what keeps thirteen downstream modules from quietly breaking.

On disk, Anvil stores a slightly flatter row — {query, tools, tool_calls, negative_reason, source_model} — and anvil/data.py reshapes each row into the {messages, tools} form. A positive row becomes a user message plus an assistant message whose tool_calls is the gold call; a negative row (tool_calls: null) becomes a user message plus a short refusal, teaching Anvil when NOT to call a tool. Here is the reshape, the frozen row_to_messages:

def row_to_messages(row: dict) -> dict:
    """Turn one on-disk row into a conversational {messages, tools} example."""
    user = {"role": "user", "content": row["query"]}
    if row["tool_calls"]:
        assistant = {
            "role": "assistant", "content": "",
            "tool_calls": [{"type": "function",
                            "function": {"name": tc["name"], "arguments": tc["arguments"]}}
                           for tc in row["tool_calls"]],
        }
    else:
        reply = NEGATIVE_REPLIES.get(row.get("negative_reason"), NEGATIVE_REPLIES["unanswerable"])
        assistant = {"role": "assistant", "content": reply}
    return {"messages": [user, assistant], "tools": row["tools"]}

Check the license before you push

Before you redistribute any data in your labs repo, verify the source dataset’s license. This is not an afterthought — a permissive sample license is part of your data pipeline. If the license doesn’t allow redistribution, you do not commit the raw data; you ship a download-and-reformat script and a small fixture instead.

For Anvil, the choice is clean and documented in data/README.md. argilla/Synth-APIGen-v0.1 is Apache-2.0 and non-gated, and it was generated by open models (Llama-3.1-70B and Qwen2.5-72B) via the APIGen pipeline (arXiv 2406.18518) — so no OpenAI or Anthropic terms attach to the outputs. Apache-2.0 + open-model-generated means we can commit a reformatted subsample directly. That mirrors decision F3 (Qwen3-4B is Apache-2.0 too): model and data are republishable without strings. As of June 2026 that’s the license tag — re-verify it on the Hub page the day you run this, because tags can change.

Reproducibility ties the module together. Fixed seed for the split, the dataset version pinned, uv.lock committed. An unpinned dataset is a fine-tune you can’t re-run — and a fine-tune you can’t re-run is a result you can’t defend.

Build it: Anvil’s data layer

Goal: build anvil/data.py (frozen) — load the tool-calling fixture, reshape every row to the {messages, tools} schema, apply Qwen3’s chat template, and print one fully tokenized example you can inspect.

What you’ll see: the split sizes (1,500 train / 200 eval, positives and negatives), one example rendered through the chat template with the <tools> block and <tool_call> shape visible, the token count, and the EOS token. The full, tested code lives in finetune-prod-labs/module-02; its smoke tests pass offline — no GPU, no network — so you can verify the data logic before you spend a single GPU-hour.

Step 1 — Set up. Copy your Module 1 state forward (the repo is cumulative — each module-NN/ carries the full tree). The pins from Module 1 already cover this module: datasets==5.0.0 and transformers==5.12.0. Make sure HF_TOKEN is in .env (copied from .env.example, never committed) — you need it only to download the tokenizer and dataset. No GPU required.

Step 2 — Verify the source license, then get the data. The fixture is a subsample of argilla/Synth-APIGen-v0.1 (Apache-2.0, non-gated — re-check on the Hub page today). Because the license permits it, the reformatted subsample is committed under data/ with a provenance README. To rebuild or resize it, data/build_data.py streams the source, reshapes each row to Anvil’s on-disk schema, and writes anvil_train.jsonl / anvil_eval.jsonl.

Step 3 — Load rows and reshape. anvil/data.py reads the JSONL rows and turns each into a {messages, tools} example. Positives carry the gold tool_calls; negatives carry a short refusal.

def load_anvil_dataset(split: str = "train", *, limit: int | None = None) -> Dataset:
    """Load a split as a 🤗 Dataset with messages + tools columns (ready for SFTTrainer)."""
    rows = [row_to_messages(r) for r in load_rows(split, limit=limit)]
    return Dataset.from_list(rows)

Step 4 — Template one example and inspect it. Load the tokenizer (directly, via AutoTokenizer — the frozen model/tokenizer factory config.py arrives in Module 3), render train[0] through format_example, and read off the token count and EOS.

from transformers import AutoTokenizer
from anvil.data import load_rows, row_to_messages, format_example

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Base")  # HF_TOKEN via os.environ
row = load_rows("train", limit=1)[0]
text = format_example(row_to_messages(row), tok)   # add_generation_prompt=False (training)
print(text)
print("tokens:", len(tok(text).input_ids), "| eos:", tok.eos_token, tok.eos_token_id)

Running uv run --env-file .env python -m anvil.data (or the steps above) prints the split stats and one tokenized example. Trimmed:

train: {'split': 'train', 'n': 1500, 'positive': 1200, 'negative': 300}
eval:  {'split': 'eval', 'n': 200, 'positive': 160, 'negative': 40}

<|im_start|>system
# Tools
...
<tools>
{"type": "function", "function": {"name": "search_record_by_name", ...}}
</tools>
...<|im_end|>
<|im_start|>user
Find the record for a person named 'John Doe' ...<|im_end|>
<|im_start|>assistant
<think>

</think>

<tool_call>
{"name": "search_record_by_name", "arguments": {...}}
</tool_call><|im_end|>

tokens: 1241 | eos: <|endoftext|> 151643

(Note: tokenizer.eos_token prints <|endoftext|> — the model’s nominal EOS — while the token that actually closes each turn in the rendered string above is <|im_end|>. The two are different special tokens; see “Special tokens and EOS” above.)

That string is the entire point of the module: it’s exactly what the trainer will see, special tokens and all, and you produced it without typing a single delimiter by hand.

Step 5 — Run the offline smoke test. The test (tests/smoke_test.py) runs fully offline — no GPU, no network — against a tiny local Qwen3 fixture that carries Qwen3-4B-Base’s real tool-calling chat template. It verifies the schema, that a positive renders a <tool_call> while a negative renders a plain reply, and that the dataset has exactly the {messages, tools} columns.

uv run pytest module-02/tests/
# 4 passed

Try it yourself:

  1. Apply a different model’s chat template (a Llama or Gemma tokenizer) to the same example and diff the output against Qwen3’s. The format differs — proof that a hand-rolled template would be wrong for at least one of them.
  2. Rebuild the fixture with data/build_data.py after changing its SEED, watch the split membership change, then restore the seed and confirm you get the identical split back. That’s reproducibility you can see.

What this lab does NOT do, on purpose. No training, no SFTTrainer/SFTConfig, no config.py (Module 3) — we prepare data, we train nothing. No LoRA/QLoRA, no bitsandbytes, no GPU (Modules 4–5). No preference pairs (Module 8) or GRPO prompts (Module 10) — those splits are named, not populated. No eval metrics (Module 7) — we motivate the held-out set, we don’t measure it. And no hand-rolled chat template, anywhere.

In production

Three things change the moment this stops being a lab.

The data is the asset, not the code. In production, a versioned data pipeline — which dataset, which version, which filters, which split seed — is worth more than the training script. The script is twenty lines you could rewrite tomorrow; the data recipe is the thing that produced your model. Pin the dataset version (a commit or hash) and log the curation recipe so the model is reproducible.

The format breaks in silence. A wrong chat template raises no error. Training converges, the loss drops, and the served model emits garbage. The only defense is to look: inspect a tokenized example (exactly what the lab prints) and compare it to what your inference server (vLLM, Module 13) actually expects. I eyeball five tokenized examples before every run — it costs two minutes and has saved me whole GPU-days chasing a “bad model” that was really a bad template.

Scale changes contamination. At 100k+ examples, exact dedup isn’t enough — near-duplicates and leaked public-benchmark rows slip in. The production step is near-dup detection (MinHash or embedding similarity); this lab teaches the dedup-then-split rule but does not implement it — the fixture is a clean, curated subsample — and you should know near-dup detection exists before you trust an eval at scale. And license and provenance are a real legal risk in a company: redistributing a dataset under the wrong license is the kind of mistake that gets escalated, so provenance is part of the deliverable, not a footnote.

In Module 3 you’ll feed this exact {messages, tools} dataset to an SFTTrainer and watch the loss curve — the data work you do now is what that curve learns.

Concept check

Why this matters. Data and the chat template are the number-one lever on a fine-tune. A wrong format or a train/eval leak quietly ruins everything downstream — the SFT in Module 3, the evaluation you’ll trust in Module 7, the preference and RL stages after. This module is the foundation reused for the rest of the course, so the five questions below stay inside exactly what it taught.

  1. A developer prepares tool-calling training data by building each prompt with f-strings — “it’s just <|im_start|> tags I copied from a blog post.” Training runs, the loss drops nicely. But the served model emits malformed, drifting output. What happened, and what’s the fix?

    • A. The model needs more epochs; train longer.
    • B. The hand-rolled prompt drifted from the model’s pretraining template, so the model learned a slightly-wrong format; fix it with tokenizer.apply_chat_template(...), the single source of truth.
    • C. The learning rate was too high; lower it.
    • D. The dataset was too small; add more examples.

  2. You’re rendering examples to use as SFT training targets. Should add_generation_prompt be True or False, and why?

    • A. True, so the model knows where the assistant turn starts.
    • B. True, because training always needs the generation prompt.
    • C. False, because you keep the full assistant response as the target to imitate; True is for inference, where you want the model to continue from the assistant opener.
    • D. It doesn’t matter for training.

  3. Your eval scores look suspiciously high — near-perfect on the held-out set. You suspect something is wrong with the data. What’s the most likely cause, and the fix?

    • A. The eval set is too small; enlarge it.
    • B. The model genuinely generalizes well; ship it.
    • C. Duplicates are shared between train and eval (contamination), so the model saw the answers; dedup on a stable key before splitting and assert train ∩ eval = ∅.
    • D. The metric is miscalibrated; switch metrics.

  4. A tool-calling dataset is templated without passing tools= to apply_chat_template. What does the model learn, and why is tools= essential for Anvil?

    • A. Nothing changes; tools= is optional formatting.
    • B. The tool schemas never enter the prompt, so the model never sees the available functions and can’t learn to call them; tools= is what serializes the schemas into the prompt.
    • C. The model learns the tools faster without the clutter.
    • D. It learns to ignore tools, which is the goal.

  5. Anvil’s task is to emit a structured tool call given a user request and a set of tools — and you also notice your model sometimes never stops generating. Which format does Anvil need, and what data fault causes the no-stop behavior?

    • A. An instruction dataset; the no-stop is a sampling bug.
    • B. A tool-calling dataset (messages + tools); the no-stop comes from a missing/misplaced EOS token in the training targets, so the model never learned where to stop.
    • C. A plain chat dataset; raise the temperature.
    • D. A tool-calling dataset; the no-stop is unrelated to data.

Answers.

  1. B. A hand-rolled f-string diverges from the template the model saw at pretraining — a space, a special token, the wrong slot for the tools block. The model learns a subtly wrong language; training “works” but the served model drifts. The fix is never to hand-roll: tokenizer.apply_chat_template(...) is the single source of truth, and it lives in the model’s own tokenizer.
  2. C. For training you keep the assistant’s full response, because that response is the target. add_generation_prompt=True appends the assistant opener so the model continues — that’s the inference/eval case. Using True in training teaches the model to echo the opener instead of answering.
  3. C. Suspiciously high held-out scores almost always mean leakage: a duplicate of a training example landed in eval, so the model effectively saw the answer. Dedup on a stable key (a hash of normalized messages + tools) before you split, then assert the intersection is empty.
  4. B. Without tools=, the schemas never make it into the prompt. The model is shown a user request with no functions, so it has nothing to learn to call. tools= is the argument that serializes the tool schemas into the prompt in the format Qwen3 expects — non-negotiable for a tool-calling specialist.
  5. B. Anvil’s task is measurable tool-calling, which needs messages + tools. A model that never stops was trained on targets missing the EOS in the right place, so it never learned the stop signal — the template’s <|im_end|> is what teaches “I’m done.”

Common pitfalls.

  • Hand-rolling the chat template. The number-one trap. An f-string diverges from the pretraining format and breaks in silence. Always tokenizer.apply_chat_template(...).
  • Confusing add_generation_prompt train (False) vs inference (True). A frequent mistake that teaches the model to copy the assistant opener instead of answering.
  • Splitting before dedup — or not deduping at all. Train/eval contamination inflates every Module 7 metric. Dedup first, then split, then assert the intersection is empty.
  • Stale pins. datasets==2.x and transformers==4.x are obsolete — both lines moved to 5.x in 2026. Pin datasets==5.0.0 and transformers==5.12.0, and re-check the day you run.
  • Redistributing a dataset without checking its license. Verify the source license first; if it forbids redistribution, ship a download script and a small fixture, not the raw data.

Key takeaways

  • Garbage in, garbage out. A fine-tune learns the distribution of your data via next-token prediction, so the data is the model — curation and format matter more than the trainer.
  • Instruction vs tool-calling. Anvil is tool-calling: every example is {messages, tools}, and the target is the assistant’s structured call — a measurable task (JSON validity, tool-name accuracy, argument match).
  • Never hand-roll the chat template. Always tokenizer.apply_chat_template(messages, tools=...) — the single source of truth for the format, living in the model’s own tokenizer. tools= is what makes the schemas enter the prompt.
  • add_generation_prompt: True at inference, False at training. And the turn-closing token <|im_end|> (id 151645) — what the chat template emits to end each turn — is what teaches the model when to stop. (Don’t confuse it with tokenizer.eos_token, which is <|endoftext|>, id 151643.)
  • Dedup before you split. Then assert train ∩ eval = ∅. Contamination inflates every eval number you’ll report in Module 7.
  • datasets is on the 5.x line. Pin the version, fix the split seed, commit uv.lock — an unpinned dataset is a fine-tune you can’t reproduce.
  • Check the source license before you redistribute. Provenance is part of the data pipeline, not an afterthought.

What’s next

Anvil now has clean, properly templated data. In Module 3 you’ll run your first fine-tune — SFTTrainer + SFTConfig, the TRL v1 way — and watch Anvil v0 finally learn the tool-call format the base model kept getting wrong. The data work you just did is exactly what that loss curve learns.

Want the full fine-tuning concept-check bank and the next module in your inbox? Subscribe here — it’s free, like everything in this series.

Links: Course index · ← Module 1: When (Not) to Fine-Tune · Module 3: Your First Fine-Tune → · Lab code for this module

References