Fine-Tuning or Retrieval?

Teams spend weeks on this decision. It rarely deserves weeks, because one question resolves most cases:

How often does the correct answer change?

If the answer changes weekly, you need retrieval. If the format of the answer matters more than its content, you may want fine-tuning. Most production systems that work well use retrieval for facts and, occasionally, a small fine-tune for behaviour.

The straight comparison

The row that decides it most often is "explains itself". A system that can point at the paragraph it answered from is auditable. A fine-tuned model that has absorbed the same information can only assert.

What retrieval is genuinely good at

• Anything with a source of truth that someone else maintains
• Domains where being able to cite the answer matters
• Content that changes on a schedule you do not control
• Cases where "I don't know" must be a possible output

What fine-tuning is genuinely good at

• A house style the model keeps drifting away from
• Rigid output structure that prompting keeps almost getting right
• Narrow classification where you have thousands of labelled examples
• Cutting latency and token cost on a very high-volume, very narrow task

Notice that none of those are about knowledge. That is the point.

The combination that usually wins

1. Retrieval supplies the facts, with citations
2. A carefully written prompt supplies the behaviour
3. If behaviour is still inconsistent after real evaluation, a small fine-tune supplies the last mile

Step three is conditional on purpose. Most teams that believe they need a fine-tune have a prompting or a chunking problem, and the fine-tune buries it somewhere much harder to debug.

A note on cost

Fine-tuning's cost is not the training run. It is:

• Building and maintaining a labelled dataset
• Re-validating on every base-model change
• Losing the ability to explain any individual answer
• Discovering that the thing you baked in is now out of date

Retrieval:     wrong answer → fix the document → correct in minutes
Fine-tuned:    wrong answer → collect examples → retrain → revalidate → days

That asymmetry compounds. It is the reason most systems that started fine-tuned end up with retrieval bolted on, while systems that started with retrieval rarely need to go the other way.

The short version

Start with retrieval and a good prompt. Measure. Add a fine-tune only when you can point at a specific behavioural failure that survived both, and you have the data to fix it.