Fix OpenAI context_length_exceeded errors

Short answer: the request you sent is bigger than the model's context window. Count the tokens before you send, leave room for the reply, and cut or summarise the oldest turns, cutting at a user message so no tool call loses its result. If it keeps happening, the conversation has outgrown "send everything" and needs a memory.

What does context_length_exceeded mean?

The context window is "the maximum number of tokens that can be used in a single request", and that number "includes input, output, and reasoning tokens" (OpenAI, conversation state). Your instructions, tool definitions, every message in the history, images and files all count, and so does the room the model needs to answer.

In the Responses API, truncation defaults to disabled: "If the input size will exceed the context window size for a model, the request will fail with a 400 error" (API reference). Chat Completions has no truncation option, so an oversized messages array fails the same way.

What the error looks like

OpenAI's error codes guide shows request validation errors returned as invalid_request_error, with error.param naming the field and error.code the specific cause. Its Agents API errors page documents context_length_exceeded as "Input exceeds the model's context window" and advises using error.code in application logic and error.message only to explain the failure. Match on the code, not the text.

import openai

try:
    reply = client.responses.create(model="gpt-5", input=items)
except openai.BadRequestError as error:
    if error.code == "context_length_exceeded":
        reply = client.responses.create(model="gpt-5", input=trim(items, budget))
    else:
        raise

import OpenAI from 'openai';

try {
  reply = await client.responses.create({ model: 'gpt-5', input: items });
} catch (error) {
  if (error instanceof OpenAI.BadRequestError && error.code === 'context_length_exceeded') {
    reply = await client.responses.create({ model: 'gpt-5', input: trim(items, budget) });
  } else {
    throw error;
  }
}

Catching the error is the fallback. The fix is to never send a request that is too big.

Step 1: count the tokens

Locally, with tiktoken. OpenAI's tiktoken maps gpt-5, gpt-4.1, gpt-4o and the o-series models to the o200k_base encoding (model table). It is fast and free, and good for a budget check on text.

import json
import tiktoken

enc = tiktoken.get_encoding("o200k_base")

def tokens(message: dict) -> int:
    content = message.get("content") or ""
    if not isinstance(content, str):
        content = json.dumps(content)
    calls = json.dumps(message.get("tool_calls") or [])
    return len(enc.encode(content)) + len(enc.encode(calls)) + 4  # rough allowance for formatting

In TypeScript, js-tiktoken is a pure JS port:

import { Tiktoken } from 'js-tiktoken/lite';
import o200k_base from 'js-tiktoken/ranks/o200k_base';

const enc = new Tiktoken(o200k_base);
export const tokens = (message: { content?: unknown; tool_calls?: unknown }) =>
  enc.encode(typeof message.content === 'string' ? message.content : JSON.stringify(message.content ?? '')).length +
  enc.encode(JSON.stringify(message.tool_calls ?? [])).length +
  4;

Exactly, with the API. OpenAI's token counting guide is blunt about local counts: images and files are not supported, tools and schemas "add tokens that are hard to count locally", and the request's formatting tokens do not appear in the text you tokenise. Its input token count endpoint takes the same payload as responses.create and returns the count the model will receive:

count = client.responses.input_tokens.count(model="gpt-5", instructions=instructions, input=items, tools=tools)
print(count.input_tokens)

Use tiktoken for the cheap check on every call and the endpoint when you are close to the limit or sending images and tools.

Step 2: leave room for the reply

The budget for history is the window minus everything else:

history budget = context window − instructions − tool definitions − max output tokens (reasoning included)

Set max_output_tokens (Responses) or max_completion_tokens (Chat Completions) on purpose. Reasoning models spend output tokens thinking before they answer, so leave more room for them. OpenAI notes that on some models trimming can begin earlier than you expect, because the API subtracts the model's maximum output allowance from its input limit even when you set a lower max_output_tokens (API reference).

Step 3: trim the history safely

Three rules keep a trimmed request valid and useful:

• Keep the system or developer message. It is the agent's instructions; it never gets trimmed.
• Cut at a user message. An assistant message with tool_calls must be followed by a tool message for each call, and in Responses each function_call needs its function_call_output. Cutting just before a user message guarantees no call loses its result and no result loses its call.
• Keep reasoning items with their output. In Responses, replay output items together, as returned.

def trim(messages: list[dict], budget: int) -> list[dict]:
    head = [m for m in messages if m["role"] in ("system", "developer")]
    rest = [m for m in messages if m["role"] not in ("system", "developer")]
    used = sum(tokens(m) for m in head)
    starts = [i for i, m in enumerate(rest) if m["role"] == "user"]
    for start in starts:  # try the longest tail first
        tail = rest[start:]
        if used + sum(tokens(m) for m in tail) <= budget:
            return head + tail
    return head + rest[starts[-1]:] if starts else head

type Message = { role: string; content?: unknown; tool_calls?: unknown };

export function trim(messages: Message[], budget: number): Message[] {
  const head = messages.filter((m) => m.role === 'system' || m.role === 'developer');
  const rest = messages.filter((m) => m.role !== 'system' && m.role !== 'developer');
  const used = head.reduce((sum, m) => sum + tokens(m), 0);
  const starts = rest.flatMap((m, i) => (m.role === 'user' ? [i] : []));
  for (const start of starts) {
    const tail = rest.slice(start);
    if (used + tail.reduce((sum, m) => sum + tokens(m), 0) <= budget) return [...head, ...tail];
  }
  return starts.length ? [...head, ...rest.slice(starts[starts.length - 1])] : head;
}

The one-line version in Responses. Set truncation: "auto" and, if the input exceeds the window, the API drops items "from the beginning of the conversation" to fit (API reference). It stops the error. It also drops whatever was oldest, which is often the user's goal, their name and the decisions made early on.

Step 4: summarise what you cut

Trimming loses everything before the cut. A running summary keeps a shorter version of it:

summary = client.responses.create(
    model="gpt-5",
    instructions="Summarise this conversation for the assistant that continues it. Keep names, numbers, decisions and open tasks.",
    input=dropped_items,
).output_text

messages = [system_message, {"role": "developer", "content": f"Earlier in this conversation:\n{summary}"}, *kept]

This costs a model call each time you summarise, and detail drifts as the summary is rewritten. OpenAI's built-in version of the idea is compaction, which folds old context into an encrypted item you cannot read.

Step 5: stop resending the history

Every fix above is the same trade: the request fits, and something the agent knew is gone. If your conversations keep hitting the limit, the history no longer belongs in the request.

With Threadless AI, your agent sends only what is new and gets back the request to submit:

const request = await memory.chat.decorate(memoryId, [{ role: 'user', content: text }], { model: 'gpt-5', tools });
const reply = await openai.chat.completions.create(request as unknown as OpenAI.ChatCompletionCreateParamsNonStreaming);
memory.chat.report(memoryId, reply);

The request is sized for a 200k-token context window, with room kept for the reply, and token use stays bounded however long the conversation runs (How it works). Before older messages stop being sent word for word, your model notes their key facts with the note tool; those facts stay in view on every call, and the older messages that matter come back, with their dates, when the conversation turns to them. A model with a smaller window is not refused; see Limits.

The fixes compared

Questions people ask

What is the maximum context length error?

It is OpenAI's 400 error with code context_length_exceeded: the request is larger than the model's context window. The window covers input, output and reasoning tokens together.

Does max_tokens count toward the context length?

Yes. The output you allow the model is part of the same window, so a large max_output_tokens or max_completion_tokens shrinks the room for history.

Why do I get the error when my prompt is under the limit?

Usually because something else counts: tool definitions, images, formatting tokens around each message, or the output allowance. Count with the input token count endpoint, which sees the request exactly as the model will.

Which tokenizer does GPT-5 use?

tiktoken maps gpt-5 to o200k_base, the same encoding as gpt-4o and gpt-4.1. For newer models, or when tools and images are involved, use the token counting endpoint.

Does previous_response_id avoid the context limit?

No. The chained history is still part of every request and still counts toward the window. Use compaction, truncation or a memory layer to keep it under.

Should I just switch to a model with a bigger context window?

It buys time, not a fix. A conversation that keeps growing will reach any limit, every call gets slower and dearer on the way, and models attend less well to the part that matters in a very long prompt.

Related

• How to stop resending the whole chat history
• LLM context window management: the four approaches
• OpenAI compaction vs a memory layer
• previous_response_id and the Conversations API
• Quickstart