Here is a failure we have now seen at four different companies, in the same shape every time.

Someone tweaks a prompt on a Friday to fix one annoying output. It works. On Monday, a different output silently breaks — one nobody was watching, in a code path nobody connected to the change. It takes three days to find, because the prompt lives in a database row with no history.

Prompts are production logic. They deserve the treatment production logic gets.

The minimum viable discipline

1. Prompts live in the repository

Not in a database, not in a config UI, not in a Notion page someone copies from. In files, in version control, in the same pull request as the code that calls them.

prompts/
  support-classifier/
    v3.md          ← current
    v2.md          ← kept, still referenced by the eval suite
    CHANGELOG.md

If a non-engineer needs to edit them, give them a pull request — not a bypass.

2. Every prompt has a version identifier the logs record

When an output is wrong, the first question is always "which prompt produced this?" That has to be answerable from a log line, months later.

{
  "request_id": "b8f21c",
  "prompt": "support-classifier@v3",
  "model": "claude-sonnet-5",
  "confidence": 0.91
}

Recording the model alongside the prompt matters just as much. Half of all "the prompt broke" incidents are actually "the model changed underneath it".

3. No prompt change ships without the eval suite

The suite does not need to be sophisticated. Thirty to fifty real cases with known-good outputs will catch the overwhelming majority of regressions:

  • Cases the current prompt handles correctly
  • Cases it historically got wrong, kept as a record
  • Edge cases someone hit in production
  • At least a few where the right answer is a refusal

Run it on every change. A prompt edit that improves one case and breaks four is the normal outcome of an unmeasured tweak.

4. Changes get a reason, not just a diff

v3 — Added explicit "return unknown rather than guessing" instruction. Fixes tickets #418, #422 where the classifier assigned low-confidence categories instead of escalating. Escalation rate expected to rise ~3%.

Six months on, that paragraph is worth more than the diff. It tells the next person what the change was defending against, so they do not undo it.

The thing teams resist

The objection is always speed: *prompts are supposed to be quick to change, that's the whole point.*

They are quick to change. They are not quick to change safely, and the gap between those is where the outages live. A prompt edit is a production deploy that happens to be written in English.

What this looks like when it is working

  1. Someone opens a pull request changing one prompt
  2. CI runs the eval suite and posts the diff in pass rates
  3. A reviewer sees which cases changed and why
  4. It merges with a changelog entry
  5. Logs record the version, so any bad output traces back in seconds

None of that is exotic tooling. It is the workflow the rest of the codebase already has, applied to the files everyone keeps pretending are configuration.