What a language model actually remembers

Ask three people what it means for a chatbot to remember something and you will get three different answers. One is thinking about what the system learned during training. Another means the conversation it can see right now. A third means a note it saved last week.

These are different mechanisms with different risks, and blurring them is how most privacy misunderstandings begin.

Layer one: what training leaves behind

During training, a model adjusts billions of numerical weights so that it predicts text well. It does not store documents in a folder. What remains is a compressed statistical impression of the patterns in its data.

Occasionally that impression is sharp enough to reproduce a passage that appeared many times, such as a famous poem or a common licence. Rare, private text is much less likely to survive, but researchers have shown extraction is possible in some cases.

Layer two: the context window

When you chat, the model sees only the text placed in front of it for that exchange, called the context window. Everything it appears to remember within a conversation is simply text that is still in that window.

Close the chat, and unless something else saves it, the window is gone. The model itself has not changed at all.

Layer three: saved memory features

Some products add a separate memory store. After a conversation, the system may write short notes such as your preferred name or a project you are working on, and place those notes in future context windows.

This is ordinary software, a database with a user interface. It can be inspected, edited and deleted, which makes it the easiest layer to control.

• Check where the notes are stored and for how long.
• Look for a way to view and delete individual entries.
• Find out whether conversations are used to improve future models.

Why the distinction matters

If a company says it does not train on your data, that speaks to the first layer only. A product may still keep your conversation history or saved notes for your own use. Conversely, a memory feature that stores notes does not mean your private text is baked into the model.

The most useful privacy question is not whether an AI remembers you, but where each kind of memory lives and who can read it.

A practical checklist

Treat the context window as a conversation in a room with a recorder you can ask to stop. Treat saved memory as a notebook you can open. Treat training data as the part you should assume is hardest to take back, and share accordingly.