LoRA: Train 1% of the Weights (Fine-Tuning in Production, Module 4)

This is Module 4 of Fine-Tuning in Production, a free 15-module course that takes one open model from raw data to a deployed, published tool-calling specialist on the Hugging Face Hub. Start at Module 1 or browse the full syllabus.

You rewrote four billion numbers to teach one behavior

Module 3 worked. Anvil — your Qwen3-4B-Base — finally emits a clean <tool_call> instead of babbling prose at a tool schema. The loss curve came down, the generations look right. Then you looked at the checkpoint you just saved, and it’s several gigabytes. Every one of the model’s ~4 billion weights got rewritten to teach it a format.

It cost more than disk, too. Full fine-tuning doesn’t just hold the model in VRAM — it holds the optimizer states for every trainable weight. With Adam, that’s two extra moment tensors per parameter, plus the gradients, on top of the weights themselves. For a 4B model the training footprint balloons to several times the model size, which is exactly why your run pushed the GPU. And the kicker: if you want a second Anvil — a different dataset, a different behavior — you do it all again and store another multi-GB copy.

That’s absurd, when you think about what actually changed. The model already knew language, JSON, and reasoning before you started. You nudged a narrow behavior — “given tools, emit a call.” A nudge that small shouldn’t need four billion edits. This module is the fix: LoRA trains roughly 1% of the weights (often much less), produces an adapter you can email, and leaves the base frozen and reusable. It’s three lines more than your Module 3 code.

In this module

You’ll learn how to:

1. Explain why full fine-tuning updates all the weights — and why LoRA (low-rank adaptation, arXiv 2106.09685) trains a tiny pair of matrices B·A instead, freezing the base entirely.
2. Configure a LoraConfig — r, lora_alpha, target_modules="all-linear", lora_dropout, bias, task_type — wrap a model with get_peft_model, and read print_trainable_parameters().
3. Reason about the two knobs: r is added capacity (the rank of the update), lora_alpha is how strongly the update is scaled (effective scale = lora_alpha / r) — plus the alpha = r and alpha = 2r heuristics.
4. Compare, with real numbers, full FT vs LoRA — trainable params, adapter size on disk, training memory, speed, and tool-calling quality — and decide when LoRA is enough vs when you’d reach for full FT.
5. Swap adapters: load two LoRA adapters onto one frozen base, and explain why adapters are light, composable, and fold into the base at inference with zero added latency.

You’ll build: Anvil now trains as a LoRA adapter. config.py::lora_config() returns the frozen LoraConfig, train_sft(tier="full", use_lora=True) wraps Qwen3-4B-Base with get_peft_model, and the same SFT run from Module 3 now updates well under 1% of the weights, saves a few-MB adapter (not a multi-GB checkpoint), matches full-FT tool-call quality, and is the brick every later module reuses.

Theory areas covered:

• T4 — LoRA: low-rank adaptation — 10% of the course. M4 owns this area; it’s reinforced by QLoRA (M5, a quantized base under the same LoRA), model merging (M11), and merge_and_unload at serving time (M12).

Prerequisites: Modules 1–3. You have the pinned stack installed, an HF_TOKEN in .env, the tool-calling dataset from Module 2, and the Module 3 SFT run wired through config.py / train_sft.py. A free Colab/Kaggle T4 (16 GB) is enough for the LoRA path here — the base loads in bf16, at full precision. (4-bit quantization is Module 5; we do not quantize here.)

Where you are: Anvil goes from full checkpoint to a tiny adapter

You build Anvil over 15 modules, each marking where you are with ✅ (done), 👉 (you’re here), and ⬜ (ahead):

• ✅ M1 — When (Not) to Fine-Tune. Baseline: Qwen3-4B-Base fails the tool-call format.
• ✅ M2 — Data Is the Model. data.py: the tool-calling fixture, formatted via the chat template.
• ✅ M3 — Your First Fine-Tune: SFT. Anvil v0 — full SFT, every weight trained, multi-GB checkpoint.
• 👉 M4 — LoRA: Train 1% of the Weights. Same run, now a LoRA adapter: ~1% trained, few-MB adapter, base frozen and reusable.
• ⬜ M5 — Make It Fit a Free GPU: QLoRA. Quantize the base to 4-bit NF4 under this same LoRA.
• ⬜ M6 — Go Faster: Unsloth and Axolotl. The same run, 2× faster.
• ⬜ M7 — Is It Any Better? Task-Grounded Evaluation. The official base-vs-tuned table.
• ⬜ M8–M9 — Align It: DPO and the DPO family. Preference tuning on top of the adapter.
• ⬜ M10 — Teach It to Reason: GRPO. Verifiable rewards.
• ⬜ M11 — Model Merging with mergekit. Combine variants.
• ⬜ M12 — Shrink It for Serving: GGUF, AWQ, GPTQ. merge_and_unload → quantize.
• ⬜ M13 — Serve It: vLLM, Ollama, TGI. vLLM --enable-lora serves the adapter.
• ⬜ M14 — Ship It: Publishing and Model Cards. Push the adapter with base_model:.
• ⬜ M15 — Capstone: Anvil, End to End.

The adapter you produce here is the reusable brick for everything downstream. QLoRA (M5) slides a quantized base underneath the same LoRA. Serving (M13) serves the adapter directly via vLLM --enable-lora. M12 merge_and_unloads it before quantizing. Nail the adapter now and the back half of the course is plumbing.

The change to Anvil this module:

	Before M4 (Module 3 output)	After M4
Trained model	Qwen3-4B-Base, full SFT, all weights	Qwen3-4B-Base frozen + a LoRA adapter
config.py	get_model_and_tokenizer, sft_args	+ lora_config() (frozen); use_lora=True path
Saved artifact	full checkpoint (multi-GB)	adapter (a few MB): adapter_config.json + adapter_model.safetensors
Trained params	~100%	well under 1% (print_trainable_parameters())
Proven in the lab	loss / generation	trainable %, adapter size, comparable quality, the swap principle

The problem with full fine-tuning

Recall what SFT did in Module 3. For each training example it ran a forward pass, computed a next-token loss on the assistant turn, backpropagated a gradient into every weight, and let the optimizer step them all. The thing being learned, per linear layer, is an update ΔW added to that layer’s weight matrix W₀. For a layer that maps a d-dimensional vector to another d-dimensional one, W₀ is d×d, and so is ΔW — a full, dense d×d matrix of changes.

The real cost isn’t even storing those updated weights. It’s training them. The dominant term during a full fine-tune is the optimizer state: Adam keeps two running moments (first and second) for every trainable parameter, in addition to the gradients. So the VRAM you need is roughly the model weights plus two moment copies plus gradients for all 4 billion parameters — several times the model’s own size. (We’ll measure a rough figure in the lab; don’t take any single VRAM number as universal — it depends on dtype, optimizer, sequence length, and batch.) And there’s a second, quieter cost: every fine-tuned variant is a complete copy of the model to store and to serve. Want Anvil-for-tool-calling and Anvil-for-summaries? That’s two multi-GB checkpoints and two model loads.

The founding insight: the update has low intrinsic rank

Here is the idea that makes LoRA work, from the founding paper — LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021, arXiv 2106.09685). The authors hypothesize that the weight update ΔW you learn while adapting a pretrained model has a low intrinsic rank. In plain terms: even though ΔW is a big d×d matrix, the actual change it encodes lives in a much smaller subspace. A full-rank matrix is overkill for it.

A low-rank matrix factors cleanly into a product of two thin matrices. So instead of learning the full ΔW, you learn

ΔW ≈ B · A

where B is d×r, A is r×d, and r ≪ d (r is on the order of 8–64; d is in the thousands). The number of parameters drops from d² to 2dr — for d = 3072 and r = 16, that’s ~9.4M numbers per layer collapsing to ~98K, a ~95× reduction per adapted layer. Train A and B, leave W₀ untouched, and you’ve captured the adaptation in a fraction of the parameters. That is the whole bet, and on behavior/format tasks like Anvil’s, it cashes out: the quality stays comparable to full FT.

LoRA is the flagship method of PEFT — Parameter-Efficient Fine-Tuning, the family of techniques (and Hugging Face’s peft library) that adapt a frozen model by training a small number of new or selected parameters instead of all of them. There are others (prefix tuning, IA³, adapters in series), but LoRA dominates in practice because of a property we’ll get to: once merged, it adds zero inference cost. And to flag the next module up front: QLoRA (M5) is LoRA on top of a 4-bit-quantized base — same adapter math, smaller base. We are not quantizing here.

How LoRA works: a frozen base plus a low-rank B·A

The mechanism is exact and worth seeing in full. For each adapted linear layer, you freeze the base weights W₀ (set requires_grad = False) and add a parallel, trainable path B·A. The forward pass becomes:

h = W₀·x + (lora_alpha / r) · B·(A·x)

W₀·x is the original layer, untouched. A·x projects the input down into the r-dimensional low-rank space; B projects it back up; the result is scaled by lora_alpha / r and added to the original output. During backprop, gradients flow only into A and B. W₀ never moves.

Why it starts cleanly. A is initialized with small random (Gaussian) values; B is initialized to zero. At step 0, B·A = 0, so the LoRA term contributes nothing and the adapted model is bit-for-bit the base model. Training then perturbs it gradually away from the base, rather than starting from a random jolt. This is a real stability property, not a footnote — it’s why a fresh adapter never makes the model worse on the first step.

Now the four knobs that matter, plus the scaling rule that trips everyone up.

r — the rank, i.e. the capacity

r is the rank of the update: the inner dimension shared by B (d×r) and A (r×d). It sets how expressive the update can be and, directly, how many trainable parameters you add (2dr per layer). Bigger r → more capacity → more parameters. Usual range is 8–64; our frozen lora_config() uses r = 16, a safe default for a behavior task.

lora_alpha — the scale, not the capacity

lora_alpha is a scaling factor. The learned update is multiplied by lora_alpha / r before being added back. This is the single most misunderstood thing about LoRA, so say it out loud: alpha does not add capacity — r does. alpha controls the amplitude of whatever the rank-r update learned. Two practitioner heuristics: alpha = r (effective scale 1.0) or alpha = 2r (effective scale 2.0, a gentle push to let the adapter assert itself more). Our frozen config uses r = 16, lora_alpha = 32 → scale = 32 / 16 = 2.0, the alpha = 2r heuristic.

⚠️ Common misconception: “LoRA modifies or shrinks the base model.”

No. LoRA never alters the base weights — W₀ stays frozen, intact, and shared. LoRA adds a small B·A module alongside it. That’s precisely why the adapter is tiny (you only saved A and B), why the base is reusable across many adapters, and why you can stack and swap adapters on one frozen base. Corollary: LoRA ≠ quantization. We do not reduce the base’s precision here — the base stays in bf16. Reducing the base to 4-bit is QLoRA, Module 5.

Here’s the trap that follows from the scaling rule, and it’s the #1 pitfall of this module: changing r while leaving alpha fixed also changes the effective scale. Bump r from 16 to 32 with alpha = 32 pinned, and the scale silently drops from 2.0 to 1.0 — so you’ve changed both capacity and amplitude at once, and you can’t tell which moved the behavior. If you want to isolate the effect of capacity, move alpha with r to hold alpha / r constant.

target_modules — which layers get an adapter

target_modules decides which linear layers receive a B·A path. The original 2021 paper adapted only the attention projections (q_proj, v_proj, and friends). The modern default — and the one we use — is target_modules="all-linear": peft resolves every linear layer in the model (attention and the MLP/feed-forward blocks) and adapts them all, without you having to hand-list module names that differ across architectures. As of peft 0.19.1 (June 2026), "all-linear" is the robust default; adapting the MLP layers, not just attention, is what closes the quality gap to full FT on most tasks. You can still pass an explicit list (e.g. ["q_proj", "v_proj"]) to adapt attention only — fewer parameters, sometimes enough — but "all-linear" is the one to reach for first.

lora_dropout, bias, task_type — the rest of the config

• lora_dropout=0.05 applies dropout on the LoRA path only — light regularization on the adapter.
• bias="none" means the layers’ bias terms aren’t trained — the standard, lightest choice.
• task_type="CAUSAL_LM" tells peft this is a decoder LM. Omit it and peft won’t wire the LM head and labels correctly — a frequent, silent mistake. It’s mandatory for a model like Qwen3.

flowchart LR
    x(["x"]) --> W0["W₀  (frozen, d×d) 🔒"]
    x --> A["A  (r×d) · trainable"]
    A --> B["B  (d×r) · trainable"]
    B --> scale["× alpha / r"]
    W0 --> add(("+"))
    scale --> add
    add --> h(["h"])
    note["only A and B get gradients · B starts at 0 · r ≪ d"]

Doing it: LoraConfig and get_peft_model

The whole LoRA configuration for Anvil lives in one place — config.py::lora_config(). This is the frozen contract for the rest of the course; nothing else in the codebase constructs a LoraConfig. Here it is (an extract of the real lab file):

from peft import LoraConfig

def lora_config(r: int = 16, lora_alpha: int = 32, lora_dropout: float = 0.05):
    """LoRA hyperparameters for Anvil. Paper: arXiv 2106.09685.

    We freeze the base and train two small matrices B·A per adapted layer (~1% of params).
      * r           — rank (capacity of the update); 8-32 typical, 16 is a safe default.
      * lora_alpha  — scaling; the update is multiplied by alpha / r (= 2.0 here).
      * target_modules="all-linear" — adapt every linear layer (attention + MLP).
    """
    return LoraConfig(
        r=r,
        lora_alpha=lora_alpha,
        lora_dropout=lora_dropout,
        target_modules="all-linear",
        bias="none",
        task_type="CAUSAL_LM",
    )

Two notes on the numbers. First, the scale here is alpha / r = 32 / 16 = 2.0 — the alpha = 2r heuristic, a deliberate “let the adapter push a bit” choice for a behavior task. (You’ll see other defaults in the wild, like alpha = r for scale 1.0; both are reasonable — they’re heuristics, not laws.) Second, r, lora_alpha, and lora_dropout are arguments so you can experiment in “Try it yourself,” but target_modules, bias, and task_type are fixed: those are the parts that have to be right for a causal LM.

Wrapping the model — and reading the proof

You don’t hand-build LoRA. PEFT’s get_peft_model takes your base model and the LoraConfig, walks the module tree, freezes the base, and injects the trainable B·A paths. With TRL’s SFTTrainer you don’t even call it yourself — you pass the config as peft_config= and the trainer wraps the model for you. Anvil’s training function does exactly that:

from trl import SFTTrainer
from . import config, data

def train_sft(*, tier="full", use_lora=False, data_limit=None,
              output_dir="outputs/anvil-sft", **arg_overrides):
    model, tokenizer = config.get_model_and_tokenizer(tier)   # base in bf16 (M3)
    train_dataset = data.load_sft_dataset("train", tokenizer, limit=data_limit)
    args = config.sft_args(output_dir=output_dir, **arg_overrides)

    peft_config = config.lora_config() if use_lora else None   # M4: the ONE new line of intent

    trainer = SFTTrainer(
        model=model,
        args=args,
        train_dataset=train_dataset,
        processing_class=tokenizer,   # TRL v1: processing_class, never tokenizer=
        peft_config=peft_config,      # pass the LoRA config; SFTTrainer wraps the model
    )
    trainer.train()
    trainer.save_model(output_dir)
    return trainer

That’s the headline of the module: you don’t rewrite training, you wrap the model. Module 3’s run is train_sft(use_lora=False); the LoRA run is train_sft(use_lora=True). Same SFTConfig, same TRL v1 idioms (processing_class, max_length on the config, never tokenizer= or max_seq_length), same data. The only new intent is the peft_config.

To prove the 1% claim, call print_trainable_parameters() on the wrapped model right after get_peft_model:

model = get_peft_model(base_model, config.lora_config())
model.print_trainable_parameters()
# trainable params: ... || all params: ... || trainable%: ~0.X

That trainable% is the receipt. For Qwen3-4B with r = 16 and "all-linear", it lands well under 1% — the base’s ~4B parameters are frozen, and only the A/B matrices count. The exact percentage depends on r and target_modules (attention-only ⟹ fewer trainable params; all-linear ⟹ more; bigger r ⟹ more), which is why we report what we measured in the lab rather than quoting a universal figure.

What gets saved: the adapter only

When you save a PEFT-wrapped model — trainer.save_model(output_dir) calls save_pretrained under the hood — PEFT writes only the adapter: an adapter_config.json (the LoraConfig, plus a base_model_name_or_path pointer) and an adapter_model.safetensors (just the A/B weights). A few megabytes, not the multi-GB base. This is expected, not a bug — the base lives on the Hub, reusable; your artifact is the small diff. It’s exactly what you’ll push to the Hub in M14 (with a base_model: tag) and what vLLM serves in M13.

Why this is a big deal: cost, swapping, zero-latency fold-in

Put full FT and LoRA side by side. The figures below mix measured lab numbers with honest orders of magnitude; version-dependent facts are dated.

	Full fine-tuning	LoRA (r=16, all-linear)
Trainable params	~100% (all ~4B)	well under 1% (print_trainable_parameters())
What you save	the whole model, multi-GB	an adapter, a few MB (adapter_config.json + adapter_model.safetensors)
Training VRAM	high — optimizer states + grads for all weights	sharply lower — optimizer states only for the adapter (the frozen base needs no moments/grads)
Training speed	baseline	often faster — far fewer parameters to update
Base reusable across variants?	no — one full copy per variant	yes — one base, N adapters
Quality on the task	baseline	comparable on a well-scoped task like tool-calling (we say comparable, not identical)

The “training VRAM” row is the one to internalize: LoRA’s biggest saving isn’t the frozen base weights — it’s that Adam keeps no moments and stores no gradients for the frozen parameters. You pay the optimizer-state tax only on the adapter. (Note that the full bf16 base still has to sit in VRAM during training — which is exactly the pressure Module 5 relieves by quantizing it to 4-bit.)

Adapters are light, composable, and swappable

Because an adapter is just the small A/B weights bound to a known base, you can keep one base in GPU memory and load many adapters onto it — picking one per request. PEFT exposes this directly: PeftModel.from_pretrained(base, adapter_path, adapter_name="anvil-toolcall") loads an adapter onto a frozen base, you can load a second under another name, and set_adapter("...") switches which one is active — without reloading the base. The output changes with the active adapter; the base never moves.

flowchart LR
    A["Adapter A<br/>(tool-calling)"] -->|load / swap| base
    B["Adapter B<br/>(other behavior)"] -->|load / swap| base
    base["Frozen base:<br/>Qwen3-4B-Base 🔒"] -->|merge_and_unload| merged["Merged model<br/>(same shape, zero added latency)"]
    note["one base in memory · N tiny adapters · pick one at inference"]

This is the foundation of multi-LoRA serving (vLLM --enable-lora, Module 13): one base loaded once, plus a fleet of cheap adapters you switch between. Versioning becomes trivial — you version few-MB adapters, not multi-GB checkpoints.

Fold-in: zero added latency at inference

There’s a subtlety about runtime cost. While you keep the adapter separate (e.g. dynamic multi-LoRA serving), each forward pass does the extra B·(A·x) work — a small but real overhead. But you can also fold the adapter into the base: compute W = W₀ + (alpha/r)·B·A once, and write the result back into the layer. PEFT does this with merge_and_unload(). The merged model has the exact same shape and the exact same latency as the base — the LoRA path is gone, absorbed into W. That’s the property that historical “series” adapters never had: a merged LoRA adds zero inference cost. (The full merge_and_unload → quantize → serve pipeline is Module 12; here we just explain why it’s free.)

When LoRA is enough vs when you’d reach for full FT

My default, stated plainly: for specializing a behavior, style, or output format on a model that’s already competent, LoRA is the right tool — and Anvil (teaching tool-call format to a capable base) is exactly that case. For a behavior/format task on a capable base, LoRA at r = 16–32 usually closes the gap to full FT. I reach for full fine-tuning (or a very high rank) only when the task shifts the model’s distribution a lot: teaching a brand-new language, absorbing a massive new domain, continued pretraining. Those need more than a low-rank nudge. Everything in between, LoRA wins on cost with comparable quality. And Module 5 makes even the frozen full-precision base fit a free T4 by quantizing it — so “LoRA on a free GPU” stops being aspirational.

Build it: turn the SFT run into a LoRA adapter

Goal: convert Module 3’s SFT run to LoRA by adding only lora_config() and the use_lora=True path, prove with numbers that well under 1% of the weights train and the adapter is a few MB, and verify the PEFT plumbing offline.

What you’ll see: the run wraps the base with get_peft_model, prints a trainable% far below 1%, and saves an adapter_config.json + adapter_model.safetensors — a few MB instead of a multi-GB checkpoint. The offline smoke test proves the whole plumbing with no GPU and no network.

The full, tested code lives in finetune-prod-labs/module-04. Its smoke tests pass offline — no GPU, no token, no network — so you can verify the harness before you spend a minute of GPU.

Step 1 — Freeze lora_config(). In anvil/config.py, add the factory shown above. It returns exactly LoraConfig(r=16, lora_alpha=32, target_modules="all-linear", lora_dropout=0.05, bias="none", task_type="CAUSAL_LM"). This is the only place a LoraConfig is constructed.

Step 2 — Add the LoRA path to train_sft. train_sft(use_lora=True) sets peft_config = config.lora_config() and passes it to SFTTrainer(..., peft_config=peft_config). Everything else — the SFTConfig from sft_args(), processing_class=tokenizer, the dataset — is unchanged from Module 3.

Step 3 — Run the LoRA fine-tune (GPU). On a free Colab/Kaggle T4, the base weights load in bf16 (tier="full") and train as a LoRA adapter — the trainer’s compute precision is auto-detected (_precision_kwargs(): fp16 on a T4, which has no bf16 tensor cores; bf16 on Ampere+):

from anvil.train_sft import train_sft

trainer = train_sft(tier="full", use_lora=True, data_limit=1500)
trainer.model.print_trainable_parameters()

Step 4 — Read the trainable %. That print_trainable_parameters() line is the proof of the module. Expect a trainable% well under 1% for Qwen3-4B at r=16 / all-linear.

Step 5 — Measure the adapter on disk. After trainer.save_model("outputs/anvil-sft"), list the output dir and size it. You’ll find adapter_config.json and adapter_model.safetensors — a few MB — and not a multi-GB model file, because only the adapter is saved (the base stays frozen and shared):

ls -lh outputs/anvil-sft/        # adapter_config.json + adapter_model.safetensors
du -sh outputs/anvil-sft/        # a few MB — contrast with the multi-GB full checkpoint

Step 6 — Run the offline smoke test. No GPU, no network — it runs a real 2-step LoRA SFT on the tiny fixture and asserts the adapter wraps the model (trainable params are a fraction of the total) and that the save writes an adapter:

uv run pytest module-04/tests/

Expected output (trimmed):

test_lora_config_fields PASSED
test_lora_trains_a_small_fraction_offline PASSED
test_lora_saves_only_an_adapter PASSED
3 passed

The smoke test doesn’t print a trainable% line — it asserts one (trainable/total < 0.5) on the tiny fixture. If you want to see the number on the tiny fixture, call print_trainable_parameters() on the wrapped model yourself; it reports trainable params: 31,232 || all params: 63,680 || trainable%: ~49. That percentage is loose because the fixture is a ~32k-param toy (the wrapped model is 63,680 params, of which 31,232 are the adapter). The point is the shape: the base is frozen, only A/B train, and only an adapter is saved. On the real 4B base in Step 4 the trainable% drops well under 1%.

Try it yourself:

1. Bump r from 16 to 64 and adjust lora_alpha to 128 to keep the same scale (alpha/r = 2.0). Re-run and compare trainable%, adapter size, and tool-call quality. Then change r without touching alpha and notice the behavior shift — that’s the scale (alpha/r) moving, not just capacity.
2. Restrict the adapter to attention only — pass target_modules=["q_proj", "v_proj"] to LoraConfig — and compare the trainable% and quality against "all-linear". How much do the MLP layers buy you?

What this lab does not do, on purpose. No 4-bit quantization / QLoRA — the base stays in bf16 (Module 5; no BitsAndBytesConfig, no prepare_model_for_kbit_training here). No full task-grounded evaluation — the quality comparison is indicative; the official base-vs-tuned table is Module 7. No end-to-end merge_and_unload → servable format (Module 12). No Hub push or model card (Module 14), no Unsloth (Module 6), no DPO/GRPO (Module 8+).

In production

In production, LoRA isn’t just a training-time VRAM saving — it’s a deployment strategy. One base loaded once into GPU memory, plus dozens of few-MB adapters, means a single server can serve N specializations (vLLM multi-LoRA, Module 13) instead of N full models. Versioning gets trivial: you version adapters, not multi-GB checkpoints, and the Hub push (Module 14) transfers only the adapter with its base_model: metadata. My opinionated default for a behavior task: keep r at 16–32. Bumping r past that mostly grows the adapter and the VRAM without moving my tool-call accuracy — capacity isn’t the bottleneck when you’re teaching a format.

Two production traps to name. (1) Base mismatch. An adapter is bound to one base — same architecture, same dimensions. Load it onto the wrong base and you get silent garbage, not an error. Always pin base_model: (that’s the metadata M14 writes, and it’s why PEFT records base_model_name_or_path in adapter_config.json). (2) Merge or not. To serve a single variant, merge_and_unload it for zero latency. To serve several variants dynamically, keep the adapters separate (multi-LoRA) and pay the small per-call overhead. Choose based on whether you’re serving one Anvil or many.

Module 5 takes this further: it quantizes the base to 4-bit NF4 under this same LoRA, so the whole Anvil training run fits on a free T4 16 GB — the base in 4-bit, the adapter trained in bf16 on top.

Concept check

Why this matters. This module anchors the skills you’ll use through serving and publishing: knowing what r versus alpha actually control, why the base never moves, when LoRA is enough versus when you’d reach for full FT, and why the adapter is tiny and swappable. The adapter you produce here is the reusable artifact for every later module — QLoRA trains it on a quantized base, DPO/GRPO tune more of them, M12 merges and quantizes it, M13 serves it, M14 publishes it. The questions below stay inside what this module taught.

1. A teammate doubles r from 16 to 32 to “give the adapter more capacity,” but leaves lora_alpha at 32. The model’s behavior changes more than they expected, in ways that feel off. What happened, and what should they do?

   • A. Nothing — only capacity changed; the surprise is in their data.
   • B. Doubling r also halved the effective scale (alpha/r went from 2.0 to 1.0), so both capacity and amplitude moved at once; to isolate capacity, raise alpha to 64.
   • C. r has no effect unless you also set target_modules; the change came from elsewhere.
   • D. They should lower lora_dropout to compensate.

2. You want the adapter to learn more nuance but you don’t want it to overpower the base’s existing behavior. Which knob does which job?

   • A. Raise lora_alpha for nuance; lower r to avoid overpowering.
   • B. r is capacity (raise it for more nuance); lora_alpha is amplitude via alpha/r (lower it to keep the adapter from dominating).
   • C. Both are the same knob; tune either one.
   • D. Set bias="all" to add nuance; use task_type to control strength.

3. Two jobs. Job A: take an English-only model and teach it a new language from a huge corpus (a big distribution shift). Job B: teach a capable base to emit a tool-call format. What do you reach for in each?

   • A. LoRA for both — it’s always enough.
   • B. Full FT for both — quality matters.
   • C. Consider full FT (or a very high rank) for A’s distribution shift; LoRA at r=16–32 for B’s behavior/format task.
   • D. QLoRA for A, full FT for B.

4. After a LoRA fine-tune, a colleague asks: “Is my base model now modified or smaller on disk?” What’s true?

   • A. Yes — LoRA rewrites the base in place, which is why the save is small.
   • B. No — the base stays frozen and intact; LoRA adds a B·A adapter (a few MB). The base is unchanged and reusable, and the adapter is bound to that exact base (pin base_model:).
   • C. Yes — LoRA quantizes the base to shrink it.
   • D. No, but the adapter contains a full copy of the base.

5. You need to serve 5 specializations on one GPU, and latency matters. Load 5 full models, or 1 base + 5 adapters — and does merge_and_unload add latency?

   • A. 5 full models; merging adds latency.
   • B. 1 base + 5 adapters (multi-LoRA); a merged adapter adds zero latency — the merged model has the same shape as the base.
   • C. 1 base + 5 adapters, but only if you merge all 5 first.
   • D. It doesn’t matter; LoRA always adds latency.

Answers.

1. B. The effective update is scaled by alpha / r. Pinning alpha = 32 while doubling r to 32 drops the scale from 2.0 to 1.0, so both capacity and amplitude changed and you can’t attribute the behavior shift to one. To study capacity alone, move alpha with r (here, to 64) so alpha/r stays 2.0.
2. B. r is the rank = capacity; raise it to let the adapter represent more. lora_alpha (via alpha/r) is the amplitude of whatever was learned; lower it to keep the adapter gentle relative to the base. They are different knobs — the most common LoRA confusion.
3. C. A big distribution shift (new language, new domain) is where a low-rank nudge can fall short, so full FT or a high rank is on the table. A behavior/format task on a capable base — Anvil’s case — is the LoRA sweet spot at r=16–32.
4. B. LoRA freezes the base; it adds B·A and never edits W₀. The base is unchanged and shareable; your saved artifact is the few-MB adapter, which is bound to that exact base (load it onto a different base and you get silent garbage — pin base_model:). LoRA is not quantization (that’s M5).
5. B. One base in memory plus N cheap adapters is the multi-LoRA pattern (vLLM --enable-lora, M13). Kept separate, each call pays a small overhead; merged via merge_and_unload, the LoRA folds into W and the model has the same shape and latency as the base — zero added cost.

Common pitfalls.

• Confusing r and lora_alpha. r = capacity (rank); alpha = scale. The effective update is multiplied by alpha / r, so moving r alone also moves the scale. Anchor on the heuristics: alpha = r (scale 1.0) or alpha = 2r (scale 2.0, as in our config).
• Believing LoRA modifies or shrinks the base. It doesn’t — the base is frozen and the adapter is added. That’s what enables the tiny adapter and adapter-swapping. And LoRA ≠ quantization — reducing the base’s precision is QLoRA (Module 5).
• Forgetting task_type="CAUSAL_LM" (or adapting the wrong layers). Without the right task_type, peft won’t wire the LM head and labels correctly; without sensible target_modules you adapt almost nothing. "all-linear" is the robust 2026 default (as of peft 0.19.1, June 2026).
• A freshness trap, not a code path: the classic PEFT ban is prepare_model_for_int8_training → use prepare_model_for_kbit_training. Note that neither appears in this module — prepare_model_for_kbit_training belongs to QLoRA (a quantized base, Module 5). And “TRL v1” now is the major version: we pin trl==1.6.0 (the 1.x line shipped on PyPI; the old “never trl==1.0” advice is obsolete), with transformers==5.12.0 and peft==0.19.1. Old tutorials that show tokenizer=, max_seq_length, or transformers==4.x are stale.

Key takeaways

• LoRA freezes the base and trains a tiny B·A per adapted layer — well under 1% of the weights — because the adaptation update has low intrinsic rank (paper: arXiv 2106.09685). The base weights W₀ are never touched.
• r = capacity, lora_alpha = scale; the effective update is multiplied by alpha / r. Moving r alone also moves the scale, so move both together to isolate capacity. Heuristics: alpha = r or alpha = 2r (our config: r=16, alpha=32, scale 2.0).
• target_modules="all-linear" adapts attention and MLP layers (peft resolves the names) and is the robust default — as of peft 0.19.1, June 2026. task_type="CAUSAL_LM" is mandatory for a decoder LM.
• You wrap the model, you don’t rewrite training: peft_config=lora_config() into SFTTrainer, same TRL v1 idioms as Module 3. print_trainable_parameters() is the receipt.
• The saved adapter is a few MB (adapter_config.json + adapter_model.safetensors); one frozen base supports N swappable adapters — the basis of multi-LoRA serving and trivial versioning.
• merge_and_unload folds the adapter into the base (W = W₀ + (alpha/r)·B·A) for zero added inference latency; kept separate (dynamic multi-LoRA) there’s a small per-call overhead.
• LoRA is the default for a behavior/format change on a capable base (Anvil’s case); reach for full FT only when the task shifts the model’s distribution a lot.

What’s next

Anvil trains 1% of its weights now — but the frozen base still sits in full-precision bf16 and barely fits. Module 5 quantizes it to 4-bit NF4 with QLoRA so the whole run fits on a free T4 16 GB — the base in 4-bit, the same LoRA adapter trained on top — with a hard look at the VRAM budget and how to dodge OOM.

Want the full fine-tuning concept-check bank and the next module in your inbox? Subscribe here — it’s free, like everything in this series.

Links: Course index · ← Module 3: Your First Fine-Tune: SFT from Scratch · Module 5: Make It Fit a Free GPU: QLoRA from Scratch → · Lab code for this module

References

• LoRA paper — “LoRA: Low-Rank Adaptation of Large Language Models” (Hu et al., 2021): https://arxiv.org/abs/2106.09685
• PEFT — LoRA conceptual guide: https://huggingface.co/docs/peft/conceptual_guides/lora
• PEFT — LoraConfig API reference: https://huggingface.co/docs/peft/package_reference/lora
• PEFT — loading and switching multiple adapters: https://huggingface.co/docs/peft/developer_guides/lora
• TRL — SFTTrainer documentation (passing peft_config): https://huggingface.co/docs/trl/sft_trainer
• Base model card — Qwen/Qwen3-4B-Base: https://huggingface.co/Qwen/Qwen3-4B-Base