Deciding What Never Goes Into a Prompt

I run automations that read my email, and the first real design decision had nothing to do with the model. It was about what those automations were allowed to send.

Think about it for a second and the default is everything. You have a message, you want the model to do something with it, so you paste the whole message in. That message contains a name, an address, an order number, possibly a phone number, possibly a passport scan somebody attached, and all of it has now left your machine for a third party.

Nobody decided that. It is what happens when you do not decide.

Most advice on this is either “keep sensitive data out of prompts”, which is useless because then you cannot build anything, or a compliance checklist that never tells you how to make the call.

First, what happens to it on the other end

Three separate questions, and they really are separate. Does the provider retain it, and for how long. Is it used for training. Who at the provider can see it, and under what circumstances.

Answers differ by plan and by contract, and they change. The important part is that defaults for a consumer plan and for a business agreement are often different, and people carry assumptions across. Building anything real means reading your actual agreement rather than the marketing page, and knowing that abuse-monitoring retention is separate from training use. “Not used for training” does not mean “not stored”.

The test I actually use

Does the model need this to do the job?

That resolves most cases in about five seconds and beats a sensitivity rating, because sensitivity is a judgement call while necessity is usually a fact.

Classifying a message as support or sales does not require the sender’s full name. It does not require their address. It requires the body text, arguably not even all of it. So those fields stay out, not because they are sensitive but because they are irrelevant, and irrelevant data is pure downside: no benefit, real exposure.

That reframe does a lot of work. It converts a fuzzy privacy question into an engineering question about what the task requires, and it tends to improve results, since a prompt holding only what matters beats one padded with surrounding noise.

Select fields, do not pass objects

The simplest mechanism and the first one to reach for. If your record has twenty attributes and the task needs three, build a small structure with those three and send that.

Obvious, and the opposite of what most code does, because passing the whole object is one line and picking fields is five.

It also fails safe in the right direction. When someone adds a new field to that record next year containing something sensitive, an explicit field list does not include it. A whole-object pass ships it immediately and nobody notices, because nothing broke.

Redaction with placeholders

Replace identifiers with tokens before sending, keep the mapping locally, substitute back afterwards. Names become person one and person two. Account numbers become placeholders. The model works on structure without seeing values.

Good when the job concerns the shape of the text rather than specific identities. Bad when identity matters to the answer.

And it has a failure mode worth knowing. Redaction is pattern matching, and pattern matching misses things: a card number in an unusual format, a name absent from your list, an identifier embedded in a URL. Treat it as reducing exposure rather than eliminating it, and never let a redaction step convince you that anything can now be sent.

Do the easy part locally

Underrated. A lot of what gets sent to a model is work no model is needed for. Extracting a date, matching a known pattern, looking up a record, checking a value against a list. Ordinary code does those exactly and for free.

Every one handled locally is data that never leaves. Look hard at any pipeline sending a model something a regular expression could have answered.

When it genuinely cannot leave

A smaller model on your own hardware. Lower quality, and for classification and extraction the quality is often good enough, and nothing leaves your network.

I would not reach for it first, because running your own inference costs real time and hardware. It is the honest answer when the data cannot go out.

Images break every rule above

Field selection works because text is structured and you can pick pieces out. An image is not.

Attach a scan of an identity document, want one number read off it, and you are sending the whole document: the photograph, the date of birth, everything else on the page. There is no partial version of an image.

So attachments are closer to an all-or-nothing decision, and worth treating as separate from your text rules rather than assuming those cover it. The options are roughly: crop to the region you need before sending, if you can locate it reliably; run that step locally; or decide the automation does not handle attachments and routes them to a person.

That last one is unglamorous and frequently right. Attachment volume is usually low and attachment sensitivity is usually high, which is exactly the ratio where automation is worth least.

Know who else is in the chain

Your provider may use sub-processors. More to the point, using an AI feature built into some other tool rather than calling a provider directly means your data goes wherever that tool sends it, under terms you probably never read, to a provider you did not choose.

Every SaaS product now has an AI feature, and switching one on is a data-sharing decision dressed as a toggle.

Where inference physically runs deserves a thought if residency matters to you. Some providers let you pin a region, most do not by default, so you go looking for the setting, and it may cost more or limit which models you can use.

Deletion is the part that bites later

Processing other people’s data, which most business automation does, means somebody eventually asks you to delete theirs.

You can delete your copy. You cannot delete it from a provider’s retained logs before their retention period expires, and if you fine-tuned a model on data containing it, you cannot meaningfully remove it from the model at all. Weights are not a database you can issue a delete against.

That makes fine-tuning on customer data a far bigger commitment than it looks like at the time, and it is worth knowing beforehand. I would think hard before training anything on data I might be obliged to erase.

The consent question underneath is uncomfortable and worth stating plainly. The person whose email you are processing agreed to your privacy policy, and your privacy policy was probably written before you started sending their messages to a third party. That is a gap. Your actual obligations depend on where you and they sit, so go and check them. Noticing the gap exists is the part people skip.

Your logs are the leak nobody counts

You carefully avoid sending something to a provider, then log the full prompt for debugging. Now it sits in a log file, replicated wherever your logs go, retained for however long your retention says, readable by anyone with access to your logging system. That group may be broader than the people allowed to see the original data.

If you log prompts, and you should log something, log the redacted version or a reference to the input rather than its contents. Think about duration too. Debug logs from a year ago are rarely useful and permanently a liability.

The same applies to memory systems and caches. Anything storing prompts or responses for reuse is a copy of that data with its own lifetime and access rules, and it is easy to build one without thinking of it as a data store, because it feels like an optimisation.

What comes back is also in scope

If a model’s output goes anywhere a person will see it and the input contained something sensitive, the output can contain it too. Models quote their input.

So summarising a document containing account numbers can produce a summary containing account numbers, and if that summary lands somewhere less protected than the original, you have moved sensitive data to a lower-security location by accident. Input controls are half the problem.

A related trap: when the input contains text from an untrusted source and your system acts on the output automatically, anyone who can write into your input can influence what your system does. That is a subject of its own, and it belongs here, because what goes into a prompt and what a prompt can cause are one design question seen from two ends.

Default to withholding

This gets pushback. Start from nothing and add fields deliberately when the task proves it needs them.

That is the reverse of how these get built. Normally you pass everything, it works, and nobody revisits it. Starting empty is slower for about an hour and leaves you with a prompt where every field is present for a reason you could state out loud.

I would rather have that than discover during an incident that a pipeline I wrote eight months ago has been shipping customer addresses to a third party for no purpose at all.

Limits

I am not a lawyer and none of this is a compliance framework. Health records, payment data, or anything with a specific regulatory regime attached come with actual requirements, and my heuristics do not substitute for reading them. Provider terms vary and change, so verify anything about retention against your own agreement rather than taking it from me.

The design principle does not depend on any of that. Send what the task needs. Select fields rather than passing objects. Handle locally whatever does not need a model. And remember your logs, your caches, and the output are all part of the same question.

For more breakdowns like this on how AI tools actually work under the hood, head back to the AI Tool Gazette homepage.