How to write a prompt you can maintain

Somebody opened a pull request on one of my own repos with a nine word edit in it. One line of a system prompt changed, nothing else in the diff.

There is no honest way to review that. I could read the words. I could not tell you which inputs would behave differently afterwards, or whether the sentence being replaced had done anything at all in the six months it sat there.

I approved it. What else was available.

The prompt in question drives the tagger in my video pipeline. It reads a paragraph of script and decides what should be on screen while that paragraph is spoken, which sounds trivial and is the difference between a watchable video and a slideshow. It had grown to a bit over two hundred lines.

Written like prose, maintained like code

Almost everything here falls out of one sentence.

You write a prompt the way you write prose. It is english, it reads like a paragraph, and editing it feels like editing a note you left for somebody. Nothing in that act warns you about consequences.

You then maintain it the way you maintain code. It sits in version control. It ships in a deploy. It has a blast radius. Change it and the behaviour of your product changes.

Prose has no failing test. You cannot look at a paragraph and see which sentence carries the load, and no amount of care while writing gives you that visibility later.

It does not get written, it gets grown

Nobody sits down and composes a two hundred line prompt.

A bad case arrives. You add a sentence. The case goes away. You ship. Another one turns up a fortnight later and gets its own sentence. Each patch was correct on the day it was applied, and after a year you have forty of them stacked in a file with no record of why any single one is there.

When I finally read the tagger prompt properly, I found four separate instructions all trying to stop it choosing footage of faces, written at different times in different phrasings, plus a rule about screen recordings I could not connect to any failure I remembered.

Some of those were working around model behaviour that a provider had since fixed. Some contradicted each other, and one of the pair was quietly winning.

The word I keep reaching for is load bearing. In a wall you can work out which bricks hold the weight. In a prompt you cannot, because there is no structure, only order.

The regression that stays invisible

The fear is rational, which is why the files calcify.

Delete a line from a function that mattered and something throws. Delete a line from a prompt that mattered and everything keeps working. Status two hundred, valid output, sensible looking answer. One class of input starts going wrong and you learn about it a month later, from a person rather than from a metric.

Deletion with a delayed, invisible cost is not a thing humans do. So the file only grows, and every addition makes the next person less willing to touch it.

Split the stable half from the request

The first fix is structural and cheap.

Most prompts are one giant string assembled from scratch on every call. Instructions, examples, the user’s input, some database rows, concatenated inside a single function that returns text.

Separate them. The instruction is stable and lives in one place. The per request data goes in as variables. Two objects instead of one wall.

That reads as tidiness and it is not. Once the halves are apart you can version the stable one, diff it, and answer the question of what the model was told without reconstructing it in your head. The variable half becomes data, with ordinary data problems and ordinary tools for them.

If your output is structured, some of what you are currently writing as instructions belongs in the schema instead, which is a separate discussion with its own failure modes. And a rule you have restated four times in prose may be telling you the behaviour wants a fine tuned model rather than another paragraph.

Examples want to be a list

Second fix. Get the examples out of the prose.

Everyone starts with them written into the paragraph, because that is how you explain something to a person. Here is a good case. Here is one to watch for.

Then you want to add the twelfth one and you are doing surgery. Find the right spot, match the rhythm, check you have not changed the sense of the sentence above it.

Hold them in a list your code walks and renders. Adding one appends an item. Removing one deletes an item. Neither touches the instruction.

There is a side effect worth having. Addressable examples are countable, and counting mine showed nine examples of one easy case and none of the case that actually broke.

The comment is what lets someone delete

Third, and this is the one I will argue for hardest.

Beside every rule, in a comment your code strips before the call goes out, write the failure it was added for. One line. Include the date.

You will not remember otherwise. I could not remember a rule I had written myself four months earlier, and I am the only person who touches that file.

What the comment buys is the ability to delete. Somebody reads the rule, reads what it was for, checks whether that failure is still a failure, and then keeps the line or removes it with a reason. Without it there is no route from doubting a line to acting on the doubt, so the line survives indefinitely.

It is the habit you already have for workarounds in code. This exists because of a bug in 2.1, delete it when 2.2 ships. A prompt with none of those comments has no exit doors in it.

Cases before you edit

Fourth. Do not edit a live prompt without a set of inputs you can run it against, including the ones that provoked each rule in the first place. Run before, run after, look at what moved.

Building that set properly, deciding how many cases you need and what counts as a pass, is its own subject and does not compress into a paragraph. The narrow point here is that without one, every prompt edit is a guess with a deploy attached.

It is also what breaks the deletion deadlock. With twenty cases on hand, pulling a line out stops being an act of faith. Remove it, run them, watch. If nothing moves across twenty cases the line was probably inert. Not proof, and a great deal better than the silence you had.

One field makes all of this cheaper: log the rendered prompt, the exact string that went over the wire, not a template name and a hash. I have written about that at length elsewhere. For maintenance specifically it is the only way to see what your prompt looked like in production rather than what you assume it looks like reading the source, and those two drift furthest in whatever has been running longest.

The inversion

Stand back and look at what most teams have built, mine included.

The prompt changes more often than any other component. Weekly. On the strength of one complaint. Because somebody had an idea on a Friday.

It also has less testing around it than anything else in the stack. The code has tests, the database has migrations and a review, and the prompt has a person editing a string in a pull request that cannot be meaningfully reviewed.

Most frequently changed, least tested. Nobody chose that. Prompts came in through a door with no engineering practice attached, as text, and text does not automatically get a test.

If you cannot delete it, it is not a prompt

Here is the position people push back on.

If you are afraid to delete a line from your prompt, you do not have a prompt. You have a superstition.

The word is exact. A superstition is a behaviour repeated because a good outcome once followed it, with no mechanism joining the two and no willingness to check. A line reading “never use bullet points in your reply”, left in production for eleven months because a model that has since been retired once produced an ugly list, is a superstition. Calling it caution does not promote it to engineering.

The fear itself is the tell. When the only thing keeping a line alive is that nobody dares remove it, that line is surviving on an absence.

What I got wrong

I cut the tagger prompt from a bit over two hundred lines to seventy one, and the output improved. Fewer face shots, better diagram choices, no measurable loss anywhere I looked.

I did it in one commit. Thirty odd rules out at once, one afternoon.

Ten days later a category of paragraph started getting the wrong footage class and I had no idea which deletion caused it. I restored lines in batches until it stopped. Most of a day, entirely self inflicted, immediately after writing a note to myself about small reversible changes.

The second one is worse and still open. I wrote those explanatory comments during the cleanup, months after the rules went in, so roughly half of them record what I think a rule was for, reconstructed from its wording. I have stored my guesses in a format indistinguishable from stored facts, and the next person to open that file has no way to separate them.

The prompt layout I use now, the comment format, and what goes in the case folder are here.