The vocabulary of agent memory is borrowed from cognitive science, databases and the model providers' APIs, and the same word often means different things in each. These are the definitions we use, with a note where Threadless AI gives a term a precise meaning. Research terms link to the paper they come from.
Jump to: A · B · C · D · E · F · G · H · I · K · L · M · N · P · R · S · T · V · W
The window
Token
The unit a model reads and writes: a word, part of a word, or a piece of punctuation. Context windows, prices and limits are all counted in tokens. See tokenizer.
Tokenizer
The program that splits text into tokens. OpenAI publishes its own as tiktoken, "a fast BPE tokeniser for use with OpenAI's models" (tiktoken). Counting tokens before a call is how you know whether a request will fit the context window.
Context window
The most tokens a model can read in one call: instructions, history, tools and the new message together. It is a hard limit, and everything in it is billed and read on every call. See LLM context window management.
Context engineering
Deciding what goes into the context window on each call, and in what order. Memory is the part of context engineering that deals with the conversation's own past.
Context rot
The decline in a model's performance as its input grows longer, even on simple tasks. Chroma tested 18 models and found that "model performance consistently degrades with increasing input length" (Context Rot). It is the main reason a bigger window does not replace memory. See also lost in the middle.
Lost in the middle
The finding that models use information at the start and end of a long prompt better than information in the middle: "performance is often highest when relevant information occurs at the beginning or end of the input context" (Liu et al.).
Needle in a haystack
A test that hides one fact at some depth in a long context and asks the model to retrieve it, at various context lengths (repository). Passing it shows a model can find a fact; it does not show the model reasons well over a long history. See context rot.
KV cache
A store of the attention keys and values computed for tokens already processed, so they are reused rather than recomputed while the model generates (Hugging Face). It speeds up one generation. It is not memory: it holds nothing the model did not already read in this call. See also prompt caching.
Prompt caching
Reusing the computation for a prompt prefix seen recently, so a repeated prefix is cheaper and faster. OpenAI enables it by default for supported models; reuse "requires the entire rendered prefix to match", and cached input is billed at a reduced rate (prompt caching). Trimming the start of a history changes the prefix and loses the cache. It lowers the cost of what you send; it does not decide what to send.
Trimming (sliding window)
Keeping only the last N messages or tokens and dropping the rest. Simple, and it loses whatever was said early on. See How to stop resending the whole chat history.
Summarisation
Replacing old messages with a summary written by a model. It keeps the gist and loses the details, and the summary itself grows. See also compaction and consolidation.
Compaction
OpenAI's name for compressing a Responses API conversation into an encrypted item that later calls carry instead of the full history. You cannot read or edit what it kept. See OpenAI compaction vs a memory layer.
previous_response_id
A Responses API parameter that continues from a stored earlier response, so you do not resend the history yourself. Every earlier input token is still billed on each call. See previous_response_id and the Conversations API.
Stored responses (store)
OpenAI saves Responses API response objects for 30 days by default; setting store to false disables it (conversation state). previous_response_id depends on stored responses. Threadless AI drops previous_response_id and store from requests it decorates, because it holds the history.
Conversations API
OpenAI's durable conversation object for the Responses API. It stores the items of a conversation; it does not decide which of them the model should still see.
Memory
Agent memory
Whatever lets an agent use what happened earlier when it answers now, beyond what fits in one call. It can be a store the agent searches, a block placed in the prompt, or, as in Threadless AI, the management of the whole request. Its usual types are working, episodic, semantic and procedural memory.
CoALA
Cognitive Architectures for Language Agents, a framework paper that organises agent memory into "short-term working memory and several long-term memories: episodic, semantic, and procedural" (Sumers et al.). Most agent-memory writing uses its four types.
Working memory (short-term memory)
What the model sees on this call. In an LLM agent it is the context window: the instructions, the recent history and anything retrieved into it. It is fast and complete, and it is limited; everything beyond it has to come from long-term memory.
Long-term memory
What persists beyond the window and across sessions: stored messages, facts, summaries or embeddings, brought back when needed. See Infinite context for OpenAI agents.
Episodic memory
Episodic memory in an AI agent is its record of specific events: what was said or done, when, and in what order. The term comes from psychology; Endel Tulving coined it in 1972, setting it beside semantic memory (episodic memory), and the CoALA framework carries the distinction into language agents.
An example: "on Tuesday the user asked for a refund on order 1182 and was told to send a receipt". That is an episode. It is tied to a time and a conversation, and it answers questions like "what did we agree last week?" An agent keeps episodic memory as stored messages, logs or passages it can search, or as a memory stream. Over time, events can be condensed into semantic facts through consolidation.
In Threadless AI, older messages are stored word for word and come back, with their dates, when the conversation turns to them. See recall.
Semantic memory
Semantic memory in an AI agent is what it knows as facts, detached from when and where it learned them. Where episodic memory says "on Tuesday the user asked for a refund and was told to send a receipt", semantic memory says "refunds require a receipt" or "the user's launch is on the 14th".
Agents usually build semantic memory by extracting facts from conversations, either with a separate model or with the agent's own model, and store them as text, embeddings or a knowledge graph. The hard parts are keeping facts current when they change (supersession) and recognising the same thing under different names (entity resolution). Semantic memory is not the same as RAG: RAG reads documents that existed before the conversation, while semantic memory is learned from it.
In Threadless AI, semantic memory is the facts your model writes with the note tool, kept in that conversation's memory and sent on every call in the facts block.
Procedural memory
Procedural memory in an AI agent is its knowledge of how to do things: its instructions, rules, tool-use patterns and learned skills. It is the third long-term type in CoALA, beside episodic and semantic memory. In practice it lives in the system prompt, in tool definitions, in reusable skills or playbooks, and in the model's weights. Some systems let an agent rewrite its own instructions from experience, for example by updating a prompt after feedback.
An example: "when a user asks for a refund, check the order date before asking for a receipt". That is a procedure, not a fact about one user.
Threadless AI leaves procedural memory to your system message, which is stored and leads every request.
Memory stream
The Generative Agents paper's long-term memory: "a long-term memory module that records, in natural language, a comprehensive list of the agent's experiences" (Park et al.). Retrieval from it "combines relevance, recency, and importance". It is the best-known design for episodic memory in agents.
Reflection
In Generative Agents, "higher-level, more abstract thoughts generated by the agent" from its memory stream and stored back into it (Park et al.). More broadly, any step where an agent reviews its memories to draw conclusions. See also consolidation.
Consolidation
Turning many events into fewer, more durable facts: merging duplicates, condensing what was learned, and dropping noise. In Threadless AI, your model notes the key facts of older messages before they stop being sent word for word, and newer facts replace older ones.
Infinite context
What a conversation is like when it never hits the end of the window: what matters stays in view, the rest is kept as facts, and details come back when they come up. No model reads an unlimited history; infinite context is assembled from the latest messages word for word, the facts from older messages kept in view, and older messages that come back when they matter. See Infinite context for OpenAI agents.
Virtual context
The idea, from the MemGPT paper, of giving a model "the illusion of an extended virtual memory via paging" between a model's window and outside storage, the way an operating system pages between RAM and disk (Packer et al.).
MemGPT
A 2023 research system that applied virtual context to LLMs, dividing memory into main context (like RAM) and external context (like disk), with the agent moving information between them through tool calls (Packer et al.). Its creators founded Letta, and the open-source project continues as Letta (Letta). See Threadless AI vs Letta.
Retrieval
RAG (retrieval-augmented generation)
Answering from documents fetched at question time and placed in the prompt. RAG brings in knowledge that existed before the conversation; memory carries the conversation itself. See Agent memory vs RAG.
Embedding
A vector that represents the meaning of a text, so that texts with similar meaning sit close together. Used to search by meaning rather than by words.
Vector database
A store that indexes embeddings for nearest-neighbour search. A useful part of a memory system, not a memory system on its own: it finds similar text but does not decide what the model should see.
BM25
A classic ranking function for keyword search, scoring documents by how often the query's words appear, weighted by how rare they are. It finds exact names and identifiers that a search by meaning can miss.
Hybrid search
Searching by words (for example BM25) and by meaning together, so exact names and paraphrases are both found. The rankings are then merged, often with reciprocal rank fusion.
Reciprocal rank fusion
A way to merge several ranked lists by summing, for each item, one over a constant plus its rank in each list. Its authors found it "consistently yields better results than any individual system" (Cormack, Clarke and Büttcher).
Knowledge graph
Facts stored as entities and the relations between them, often as subject, relation, object triples. See also temporal knowledge graph and entity resolution.
Temporal knowledge graph
A knowledge graph that records when each fact held. Zep's, for example, "implements a bi-temporal model": one timeline for when events happened and one for when the system learned them (Zep paper). See temporal validity.
Entity resolution
Recognising that different names refer to the same thing ("Acme", "Acme Corp", "the client") and merging them into one entity. Without it, facts about one thing scatter across several.
GraphRAG
Microsoft's "graph-based approach to question answering over private text corpora": it builds a knowledge graph from documents and summarises its communities to answer broad questions (Edge et al.). It is a RAG technique for documents, not conversation memory.
Recall
In Threadless AI, the older messages and facts that matter to a new user message, brought back with their dates and placed just before it, in the recall block.
Time
Forgetting curve
How the availability of a memory falls with time since it was used. Anderson and Schooler argued that retention follows a power law rather than an exponential, mirroring how often information recurs in the environment (Anderson and Schooler, 1991).
Memory decay
Lowering a memory's weight or availability as it ages, so recent and frequently used memories win. Decay can take a memory out of view without deleting it.
Supersession
When a newer fact replaces an older one it contradicts: "the launch moved to the 21st" supersedes "the launch is on the 14th". The older fact is retired, not necessarily deleted. In Threadless AI, newer facts replace older ones.
Temporal validity
The time range during which a fact was true, recorded on the fact itself, so a system can answer "what held in March?" It is the core of a temporal knowledge graph.
Staleness
When a memory still in use no longer reflects the truth. It is the failure that supersession, temporal validity and memory decay each try to prevent.
Threadless AI
Memory
One conversation's messages and facts, named by its memory id. The memory id is the only scope: recall never crosses memories. See Memory per conversation, not per user.
Fact
One sentence that reads on its own, written by your model with the note tool and sent on every call in the facts block. See semantic memory.
decorate
The call made before each model call. It takes what is new in the conversation and returns the request, ready to submit to OpenAI with your key.
report
The call made after the model replies, carrying the reply so the memory has both sides.
Note tool
The tool added to your model's tools. When older messages are about to stop being sent word for word, the request asks your model to note their key facts with one call to it, answered with noted. See Let the model write its own notes.
Facts block
The instruction message placed right after your instructions, listing the facts from older messages that are no longer sent word for word.
Local tail
The recent turns the SDK keeps in your process. When Threadless AI cannot be reached, the SDK builds the request from it, so the model call goes ahead. See What happens when memory is down.
Benchmarks
LongMemEval
A benchmark of "five core long-term memory abilities of chat assistants: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention", asked after long histories (Wu et al.).
LoCoMo
A benchmark of very long conversations, each about 300 turns and 9,000 tokens on average over up to 35 sessions, with questions that need facts from across them (Maharana et al.). Vendors' published LoCoMo scores vary with setup, model and judge; see Mem0 vs Zep vs Letta.