NOTES / FINE-TUNING AND ADAPTATION
When to fine-tune
Fine-tuning changes how a model behaves, not what it can look up. A decision ladder, the evidence behind it, and what to measure before and after.
What fine-tuning is for
Fine-tuning continues training on your examples so the model’s weights change. That is good at teaching a consistent behaviour: an output style, a format, a way of calling tools. It is a poor way to give a model facts it can look up.
The LIMA authors frame it as the superficial alignment hypothesis: a model’s knowledge and abilities come almost entirely from pretraining, and alignment mostly teaches which style of response to use3. Their 65B model, tuned on only 1,000 curated examples, was preferred or tied with GPT-4 in 43 percent of their human comparisons3. That is one study’s result, but it supports the practical rule: tune for behaviour.
For knowledge, Microsoft researchers compared unsupervised fine-tuning with retrieval and found that retrieval consistently won, for both facts the model had seen before and new ones; they also found that models struggled to learn new facts through fine-tuning2. So for changing or citable facts, use retrieval rather than weights.
The ladder
Diagram: my own.
OpenAI’s guidance has long been to try prompt engineering and prompt chaining before fine-tuning1. I agree, and add two steps: retrieval for facts, and an eval set before any of it (Why evals).
Diagram: my own.
Good reasons to fine-tune
- A stable behaviour that prompts do not hold. Long, fiddly instructions that the model follows only most of the time.
- Cost or latency. A smaller tuned model can replace a larger prompted one on a narrow task, and shorter prompts cut input tokens (Latency and cost). This is the usual route into distillation.
- Alignment of style at scale. InstructGPT showed that outputs from a 1.3B model tuned with human feedback were preferred to those of the 175B GPT-34. Behaviour tuning can matter more than size.
- Evidence from your own evals. AWS reports that tuned Claude 3 Haiku beat few-shot prompting on several tasks in its examples6. Treat vendor reports as a reason to test, not a promise.
Reasons to wait
- You have no eval set, so you cannot tell whether tuning helped (Building eval sets).
- The problem is missing information. Fix retrieval and context first (Context window as a budget).
- The task or the facts change weekly. Every change needs retraining.
- You cannot get enough clean examples (Preparing training data).
Risks to plan for
- Forgetting. Full fine-tuning can erode skills outside the target domain; parameter-efficient methods forget less but also learn less on hard domains such as code and maths5 (LoRA and QLoRA).
- Lock-in. A tuned model is tied to a base model. When the base is upgraded, you may need to redo the work.
- Access. Which models you can tune depends on the provider. One third-party write-up reports that the Claude API offers no fine-tuning and that the Bedrock route for Claude 3 Haiku has closed7. Verify with Anthropic and AWS before planning around it; my note may be out of date.
A test before you commit
- Build an eval set from real cases and score a strong prompt baseline.
- Add few-shot examples and retrieval; re-score.
- Fine-tune on a train split and score on the held-out test split.
- Keep the tune only if it beats the best prompted version on your metric and the gain justifies maintenance.
Then continue with supervised fine-tuning.
Sources
- [1]Fine-tuning guide — OpenAI platform docs (earlier text, seen in a reproduced copy)ADVICE TO TRY PROMPTING AND PROMPT CHAINING FIRST
- [2]Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs — Ovadia, Brief, Mishaeli, Elisha · 2024PARAPHRASED
- [3]LIMA: Less Is More for Alignment — Zhou et al. · 2023PARAPHRASED
- [4]Training language models to follow instructions with human feedback — Ouyang et al. · 2022PARAPHRASED
- [5]LoRA Learns Less and Forgets Less — Biderman et al. · 2024PARAPHRASED
- [6]Best practices and lessons for fine-tuning Anthropic's Claude 3 Haiku on Amazon Bedrock — AWS Machine Learning Blog · 2024LIFECYCLE AND THE CLAIM THAT TUNING BEAT FEW-SHOT PROMPTING, AS REPORTED BY AWS
- [7]Can you fine-tune Claude? — Ertas · 2026STATUS OF CLAUDE FINE-TUNING, THIRD-PARTY REPORT; VERIFY WITH ANTHROPIC