{"id":134,"date":"2026-09-17T15:45:11","date_gmt":"2026-09-17T07:45:11","guid":{"rendered":"https:\/\/wp.qoraapi.com\/context-window-management\/"},"modified":"2026-09-20T02:51:06","modified_gmt":"2026-09-19T18:51:06","slug":"context-window-management","status":"publish","type":"post","link":"https:\/\/qoraapi.com\/blog\/context-window-management\/","title":{"rendered":"Managing the Context Window: Truncation, Summarization, and Sliding Windows"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">The context window is a budget, not a bucket: input and output tokens draw on the same limit, and cost scales with everything you send. Manage it with four levers \u2014 truncation, summarization, retrieval-on-demand, and routing \u2014 plus a per-section token budget and an offline eval loop that tells you when more context stops helping.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The techniques below are ordered by marginal cost, because the cheapest fix is almost never &#8220;call a model to fix it.&#8221;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why the context window is a budget, not a bucket<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A model with an <em>N<\/em>-token window does not give you <em>N<\/em> tokens of input. It gives you <em>N<\/em> tokens of input <strong>plus<\/strong> output. If you send <em>M<\/em> input tokens, the maximum completion you can request is <em>N \u2212 M<\/em>. Teams discover this the hard way: they set <code>max_tokens=4000<\/code>, send a 6,000-token prompt to an 8,192-token model, and get a 400 back. Worse, some SDKs and wrappers silently trim the prompt instead of failing, so the model answers confidently with the last third of your document missing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The correct sequence is <strong>reserve, then fill<\/strong>: subtract the output reserve from the window first, and treat the remainder as the input budget. Never assemble the prompt and hope it fits.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The second reason it is a budget is that <strong>cost scales with input, and input compounds<\/strong>. A chat request re-sends the entire prefix on every turn. Turn 20 re-bills turns 1 through 19. Pricing is linear in tokens, but the tokens per turn grow with conversation length, so the cost of a session grows roughly quadratically with the number of turns. That is the real reason long conversations get expensive \u2014 not the output.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Unbounded history:<\/strong> session cost is O(turns\u00b2) in input tokens, and time-to-first-token (TTFT) grows every turn as prefill lengthens.<\/li>\n<li><strong>Bounded history of K turns:<\/strong> session cost becomes O(turns \u00d7 K) \u2014 linear again, with flat TTFT. This single change is usually the largest cost win available.<\/li>\n<li><strong>Attention compute<\/strong> grows super-linearly with sequence length, so a 4\u00d7 longer prompt costs more than 4\u00d7 to process even at a flat per-token price.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">So &#8220;the window is 200K, I can send 200K&#8221; is a budgeting error, not a feature \u2014 your effective window excludes the output reserve, and recall degrades before you reach the hard limit.<\/p>\n\n\n<h2 class=\"wp-block-heading\">Strategies when you exceed the budget<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">There are five levers, and applying them in the wrong order is why teams end up paying a summarization call on every turn. Escalate in increasing marginal cost: deterministic transforms first, model calls last, infrastructure changes only when the data justifies it.<\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table class=\"has-fixed-layout\"><thead><tr><th>Lever<\/th><th>What it does<\/th><th>Marginal cost<\/th><th>Use when<\/th><th>Failure mode<\/th><\/tr><\/thead><tbody><tr><td><strong>Drop oldest<\/strong> (turn-boundary truncation)<\/td><td>Removes the oldest messages, keeping the newest K turns verbatim<\/td><td>Zero \u2014 no extra call, no latency<\/td><td>Chat where early turns are genuinely stale; the default first move<\/td><td>Silently deletes a constraint the user set at turn 2, so the model contradicts it later. Mitigate with a pinned-facts list.<\/td><\/tr><tr><td><strong>Rolling summarization<\/strong><\/td><td>An LLM compresses dropped turns into a bounded summary carried forward<\/td><td>One extra call, ~5\u201310% of the summarized tokens, plus latency<\/td><td>Long sessions where early decisions still matter<\/td><td>Recursive drift: summarize a summary enough times and details mutate, then get hallucinated.<\/td><\/tr><tr><td><strong>Retrieve on demand<\/strong> (agentic memory)<\/td><td>Nothing is pre-stuffed; the model calls a search tool over full history or a document store<\/td><td>Only when invoked \u2014 a tool round-trip on those turns<\/td><td>Histories or corpora that can never fit whole<\/td><td>The model cannot know what it does not have. Without an index or hint in the prompt, it never thinks to look.<\/td><\/tr><tr><td><strong>Compress in place<\/strong><\/td><td>Deterministic pruning: strip HTML boilerplate, collapse whitespace, dedupe repeated chunks, normalize JSON<\/td><td>Near zero \u2014 runs in your process<\/td><td>Always, before anything else. Frequently 20\u201360% off the prompt for free<\/td><td>Irreversible. Never compress instructions, constraints, or schema definitions \u2014 only bulk content.<\/td><\/tr><tr><td><strong>Route to a bigger window<\/strong><\/td><td>Same prompt, a model with a larger context limit<\/td><td>Per-token price usually rises; input cost scales with the bigger prompt<\/td><td>Genuine long-document tasks that must be read whole<\/td><td>A bigger window is not better recall. Cost rises immediately, quality often does not.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The decision rule: <strong>compress deterministically, then drop oldest, then summarize, then route.<\/strong> Escalate only when eval data shows the current lever costs you accuracy. Retrieval-on-demand is less a later lever than a different architecture \u2014 if your corpus is a knowledge base rather than a conversation, it is the right first answer, and it is the pattern behind <a href=\"https:\/\/qoraapi.com\/blog\/ai-embeddings-rag\/\">embeddings and RAG<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One constraint cuts across all five: <strong>never split a tool call from its result.<\/strong> Truncating a message whose <code>tool_call<\/code> was dropped produces orphaned tool messages, and most providers reject the request outright. Every cut point must land on a safe boundary \u2014 the code below does this.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Sliding window plus rolling summary memory<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The pattern that survives production combines a sliding verbatim window (recency is what users actually reference), a rolling summary for everything older, and a pinned-facts list that is never summarized away. Pinning is the fix for drift \u2014 names, IDs, units, and explicit constraints bypass summarization entirely.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import tiktoken\n\nENC = tiktoken.get_encoding(\"o200k_base\")   # match your target model's tokenizer\n\ndef count_tokens(text: str) -&gt; int:\n    return len(ENC.encode(text))\n\nSUMMARY_PROMPT = \"\"\"You maintain the running memory of a long conversation.\nMerge EXISTING SUMMARY with NEW TURNS into one summary.\nKEEP: decisions, user-stated facts, constraints, open questions, IDs, names, units.\nDROP: pleasantries, restated context, anything already obvious.\nOutput terse bullets only, at most {limit} tokens.\"\"\"\n\nclass BoundedHistory:\n    \"\"\"Sliding verbatim window + rolling summary + pinned facts + hard token cap.\"\"\"\n\n    def __init__(self, system, client, summarize_model=\"gpt-4o-mini\",\n                 window_turns=8, summarize_at=16, summary_budget=400,\n                 max_input_tokens=12_000):\n        self.system = system\n        self.client = client\n        self.summarize_model = summarize_model\n        self.window_turns = window_turns      # turns kept verbatim\n        self.summarize_at = summarize_at      # fold once history exceeds this\n        self.summary_budget = summary_budget\n        self.max_input_tokens = max_input_tokens\n        self.summary = \"\"\n        self.pinned = []                      # never summarized away\n        self.turns = []\n\n    def pin(self, fact: str) -&gt; None:\n        \"\"\"Call whenever the user states a durable constraint or identifier.\"\"\"\n        self.pinned.append(fact)\n\n    def _summarize(self, dropped) -&gt; None:\n        convo = \"\\n\".join(f\"{m['role']}: {m['content']}\" for m in dropped)\n        r = self.client.chat.completions.create(\n            model=self.summarize_model,\n            messages=[\n                {\"role\": \"system\",\n                 \"content\": SUMMARY_PROMPT.format(limit=self.summary_budget)},\n                {\"role\": \"user\",\n                 \"content\": f\"EXISTING SUMMARY:\\n{self.summary or '(none)'}\\n\\n\"\n                            f\"NEW TURNS:\\n{convo}\"},\n            ],\n            max_tokens=self.summary_budget,\n        )\n        self.summary = r.choices[0].message.content.strip()\n\n    def _safe_cut(self, cut: int) -&gt; int:\n        \"\"\"Move the cut back so a tool call is never separated from its result.\"\"\"\n        while cut &gt; 0 and self.turns[cut - 1].get(\"role\") == \"tool\":\n            cut -= 1\n        return cut\n\n    def add(self, role: str, content: str) -&gt; None:\n        self.turns.append({\"role\": role, \"content\": content})\n        # Fold oldest turns into the summary in batches, amortizing the extra call.\n        while len(self.turns) &gt; self.summarize_at:\n            cut = self._safe_cut(len(self.turns) - self.window_turns)\n            if cut &lt;= 0:\n                break\n            self._summarize(self.turns[:cut])\n            self.turns = self.turns[cut:]\n\n    def _head(self):\n        \"\"\"System, summary, and pins sit at the TOP of the prompt, not the middle.\"\"\"\n        head = [{\"role\": \"system\", \"content\": self.system}]\n        if self.summary:\n            head.append({\"role\": \"system\",\n                         \"content\": \"Conversation summary so far:\\n\" + self.summary})\n        if self.pinned:\n            head.append({\"role\": \"system\",\n                         \"content\": \"Pinned facts (authoritative):\\n- \"\n                                    + \"\\n- \".join(self.pinned)})\n        return head\n\n    @staticmethod\n    def _count(msgs) -&gt; int:\n        return sum(count_tokens(m[\"content\"]) for m in msgs)\n\n    def build(self, user_msg: str, output_reserve: int = 800):\n        head = self._head()\n        budget = self.max_input_tokens - output_reserve   # reserve first\n        turns = list(self.turns)\n        msgs = head + turns + [{\"role\": \"user\", \"content\": user_msg}]\n        while turns and self._count(msgs) &gt; budget:       # clamp if still over\n            turns = turns[1:]\n            msgs = head + turns + [{\"role\": \"user\", \"content\": user_msg}]\n        used = self._count(msgs)\n        if used &gt; budget:\n            raise ValueError(f\"context budget exceeded: {used} &gt; {budget}\")\n        return msgs, output_reserve\n\n# Usage\nmem = BoundedHistory(system=\"You are a support engineer for the billing API.\",\n                     client=client)\nmem.pin(\"Customer is on the Enterprise plan, billed annually.\")\nmem.add(\"user\", \"Our invoice shows a duplicate charge for March.\")\nmem.add(\"assistant\", \"I can see two line items. Let me pull the ledger.\")\nmessages, reserve = mem.build(\"What should I tell finance?\")\nresp = client.chat.completions.create(model=\"gpt-4o\", messages=messages,\n                                      max_tokens=reserve)\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Four parameters carry the whole design. Tuning notes that matter in practice:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><code>summarize_at<\/code> should be roughly 2\u00d7 <code>window_turns<\/code>.<\/strong> Folding on every turn pays a summarization call per message and thrashes the summary. Batching at double the window amortizes it, at the cost of a temporary prompt spike.<\/li>\n<li><strong>Summarize with a small model.<\/strong> Compression is not reasoning, and the summary budget caps output anyway.<\/li>\n<li><strong>Chain as <em>prior summary + newly dropped turns<\/em>, never summary-of-summary alone.<\/strong> Each pass must see raw text for the new material, or drift compounds fast.<\/li>\n<li><strong>Pin aggressively, summarize reluctantly.<\/strong> Drift comes almost entirely from facts that were summarized twice.<\/li>\n<li><strong>Emit metrics per build:<\/strong> <code>prompt_tokens<\/code>, <code>summary_tokens<\/code>, <code>dropped_turns<\/code>, and whether the clamp fired. A clamp that fires on every request means your budget is wrong, not that your code is safe.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For multimodal content, <code>count_tokens<\/code> needs a branch: image and audio parts are billed in units that are not characters, and the provider&#8217;s usage report is the only reliable count.<\/p>\n\n\n<h2 class=\"wp-block-heading\">Token budgeting per section<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A budget you can defend has a number for every section and headroom you did not spend. Here is a working budget for a support assistant on a 128K-window model, deliberately capped at <strong>16,000 input tokens<\/strong> because the eval data showed no accuracy gain above it:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>System prompt + policies: <strong>900<\/strong><\/li>\n<li>Tool schemas (5 tools): <strong>700<\/strong><\/li>\n<li>Pinned facts: <strong>300<\/strong><\/li>\n<li>Rolling summary: <strong>600<\/strong><\/li>\n<li>Retrieved knowledge chunks (8 \u00d7 ~700): <strong>5,600<\/strong><\/li>\n<li>Verbatim history (last 12 turns): <strong>3,000<\/strong><\/li>\n<li>Current user turn + attachment: <strong>1,500<\/strong><\/li>\n<li><em>Subtotal \u2014 input:<\/em> <strong>12,600<\/strong><\/li>\n<li>Output reserve (<code>max_tokens<\/code>): <strong>2,000<\/strong><\/li>\n<li><em>Total against the 16,000 cap:<\/em> <strong>14,600<\/strong> \u2192 <strong>1,400<\/strong> headroom<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Tool schemas are the silent eater.<\/strong> Five tools cost 700 tokens; a full MCP catalog can cost 5,000\u201310,000 before a single user word is sent, and it is re-billed every call. Prune the tool list per request to what the turn plausibly needs \u2014 see <a href=\"https:\/\/qoraapi.com\/blog\/ai-function-calling-tool-use\/\">AI function calling and tool use<\/a> for the selection pattern.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>The cap sits below the model&#8217;s limit, on purpose.<\/strong> The 1,400-token headroom absorbs a tool result you did not plan for, a retry with an error appended, or a user paste. Budget to the hard limit and every surprise becomes a failed request.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Count with the target model&#8217;s tokenizer, and re-tune when you switch models.<\/strong> Characters-divided-by-four is fine for English prose and wrong for JSON, code, and CJK text, where a character can cost a full token or more. If you route across providers, budget in the units of the <em>largest<\/em> tokenizer in the pool. Enforce it with an assertion in the prompt assembler, not with discipline: <code>assert prompt_tokens &lt;= SECTION_BUDGET<\/code> fails in CI, while a review comment fails in production.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The &#8220;lost in the middle&#8221; effect<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Long-context models do not attend uniformly. Retrieval accuracy across a long prompt follows a U-shaped curve: content at the <strong>beginning<\/strong> and the <strong>end<\/strong> is recalled far more reliably than content buried in the middle. This was measured systematically in 2023 and remains visible in current long-context models \u2014 it is a property of how attention distributes, not a bug a bigger window fixes.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Constraints first.<\/strong> System prompt, hard rules, and output format go at the top. Never bury a constraint in the middle of a 2,000-token system prompt; split it into a short always-on core plus policies loaded on demand.<\/li>\n<li><strong>Restate the task at the end.<\/strong> Put the user&#8217;s question <em>after<\/em> the retrieved context, not before it. Repeating it once more immediately before generation is the highest-return change available \u2014 ~20 tokens, and the instruction lands in the high-recall zone.<\/li>\n<li><strong>Bookend your retrieved chunks.<\/strong> Sort by relevance, place the top chunks first and last, fill the middle with lower-ranked material. Or cap at four to six chunks: twenty mediocre chunks dilute two good ones.<\/li>\n<li><strong>Few-shot examples are the most vulnerable.<\/strong> Examples placed mid-prompt get ignored \u2014 move the most representative one to the end.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Measuring quality vs context size<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">More context is not monotonically better. Accuracy is typically concave in context size: it rises as you add the evidence the task needs, plateaus, then declines as irrelevant material competes for attention. A half-day of measurement tells you where the peak is.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Build a labeled set from real traffic.<\/strong> 50\u2013100 logged tasks with the expected answer and a note on the <em>minimum<\/em> evidence required. Synthetic sets miss the messy inputs that actually break you.<\/li>\n<li><strong>Plant a needle at depth.<\/strong> Include a fact that must be recovered, placed at roughly 10%, 50%, and 90% of the assembled prompt. This doubles as your lost-in-the-middle regression test.<\/li>\n<li><strong>Run a context sweep.<\/strong> Same model, same temperature, same tasks at 2K \/ 8K \/ 32K \/ full. Only context size changes.<\/li>\n<li><strong>Score five numbers, not one:<\/strong> task accuracy, contradiction rate against pinned facts, p50\/p95 latency, mean input tokens, and cost per <em>resolved<\/em> task.<\/li>\n<li><strong>Find the knee.<\/strong> Plot accuracy and cost-per-resolved-task against context size. Most apps peak well below the model&#8217;s maximum, often by an order of magnitude.<\/li>\n<li><strong>Ablate one section at a time.<\/strong> Remove retrieved context, then the summary, then the history. If accuracy does not move when a section disappears, that section is pure cost \u2014 delete it.<\/li>\n<li><strong>Gate it in CI.<\/strong> The prompt assembler is a pure function: assert the token count is within budget, the section order is stable, and no tool call is orphaned.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The metric to optimize is <strong>quality per 1,000 input tokens<\/strong>, not raw accuracy. A configuration scoring 2% lower on accuracy at a third of the input cost is usually the better product decision, and it compounds \u2014 smaller prompts also mean lower latency, which users notice. This is the same methodology you use to <a href=\"https:\/\/qoraapi.com\/blog\/reduce-ai-api-costs\/\">reduce AI API costs<\/a> without degrading output.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How a gateway helps<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Context management is a routing problem with one extra input: <strong>prompt size<\/strong>. Measure the assembled prompt before sending it, and token count becomes a routing key alongside task type.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Context-aware routing.<\/strong> Under the threshold, send to the cheap mid-tier model. Over it, or for whole-document reads, send to a long-context model. This is the same table as <a href=\"https:\/\/qoraapi.com\/blog\/choose-right-ai-model-routing\/\">model routing<\/a>, with <code>prompt_tokens<\/code> as the deciding column.<\/li>\n<li><strong>One key, many windows.<\/strong> Without a gateway, every long-context model is a separate SDK, base URL, auth scheme, and error shape \u2014 so &#8220;route to a bigger window&#8221; becomes an integration project instead of a string change.<\/li>\n<li><strong>Normalized overflow errors.<\/strong> Providers signal context-length failures inconsistently. One predictable overflow error makes your handler a single branch that runs the compression ladder and retries, instead of a provider-specific switch statement.<\/li>\n<li><strong>Consistent usage reporting.<\/strong> Uniform <code>prompt_tokens<\/code> \/ <code>completion_tokens<\/code> across models gives you the per-call budget actuals your eval loop needs.<\/li>\n<li><strong>Fallback that respects the budget.<\/strong> When the preferred model is throttled, the fallback must also fit the prompt. Exposing each model&#8217;s window lets you filter the fallback chain by capacity instead of discovering the mismatch as a 400.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">That is the case for an AI API relay: one OpenAI-compatible endpoint in front of models with different windows and prices, so context-aware routing becomes a configuration change rather than a rewrite. <a href=\"https:\/\/qoraapi.com\/\" target=\"_blank\" rel=\"noopener\">qoraapi.com<\/a> exposes many models through a single key, which is what makes the routing table above deployable in an afternoon.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">What actually happens when a request exceeds the context window?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Most providers return a 400-class error naming the context length, and no output is billed. The dangerous case is a client library or proxy that trims the prompt to fit instead of failing \u2014 you get a confident answer computed from a silently truncated document. Never rely on the SDK to enforce your budget: count tokens before the call, reserve the output allowance, and fail closed in your own code.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Should I just use a long-context model and skip summarization?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Only for tasks that genuinely require reading a document whole. Input cost scales with everything you send, prefill latency grows with it, and recall degrades in the middle of long prompts regardless of the advertised window. Treat long context as a deliberate route for specific tasks, not a substitute for a memory strategy \u2014 and validate it against a smaller-context configuration on cost per resolved task before committing.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How often should I summarize the conversation history?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Fold older turns when the verbatim history exceeds roughly twice the number of turns you keep in the window. Summarizing every turn pays a model call per message and accelerates drift; summarizing too rarely lets the prompt spike before it folds. The 2\u00d7 ratio amortizes the call while bounding the spike.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Can truncating history break tool calling?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes, and it is the most common cause of mysterious 400s in agent loops. If you drop an assistant message containing a <code>tool_call<\/code> but keep the matching <code>tool<\/code> result, the message list is malformed and most providers reject it. Adjust your cut point backwards to a safe boundary so each tool call and its result stay together, and treat the pair as one indivisible unit when counting turns.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Treat the context window as a budget you allocate, not a bucket you fill. Reserve the output allowance first, cap history at a fixed number of turns so session cost stays linear, keep a sliding verbatim window with a bounded rolling summary and a pinned-facts list, and put instructions at the start and the end rather than the middle. Then measure: sweep context size against accuracy and cost per resolved task, ablate each section, and keep only what earns its tokens.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Ready to wire it up? Start with the <a href=\"https:\/\/qoraapi.com\/blog\/ai-api-gateway-guide\/\">AI API gateway guide<\/a> and the <a href=\"https:\/\/qoraapi.com\/blog\/openai-compatible-api-guide\/\">OpenAI-compatible API explainer<\/a>, then drop the <code>BoundedHistory<\/code> class above into your call path.<\/p>\n\n\n\n\n<h3 class=\"wp-block-heading\">Related reading<\/h3>\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/qoraapi.com\/blog\/prompt-caching-guide\/\">Prompt Caching Explained: How to Cut Costs on Repeated Context<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/ai-embeddings-rag\/\">AI Embeddings Explained: Vectors, Similarity, and Building Your First RAG<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/choose-right-ai-model-routing\/\">How to Choose the Right AI Model: A Practical Model-Routing Guide<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/ai-agent-memory\/\">Giving AI Agents Memory: Working, Episodic, and Retrieval Memory<\/a><\/li><\/ul>\n\n","protected":false},"excerpt":{"rendered":"<p>The context window is a budget, not a bucket. Learn truncation, summarization, and sliding-window memory to keep quality high without exceeding the limit.<\/p>\n","protected":false},"author":1,"featured_media":133,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[5,6,9,7],"class_list":["post-134","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-api","tag-ai-api","tag-api-gateway","tag-developer-tools","tag-developers"],"_links":{"self":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/134","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/comments?post=134"}],"version-history":[{"count":1,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/134\/revisions"}],"predecessor-version":[{"id":188,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/134\/revisions\/188"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/media\/133"}],"wp:attachment":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/media?parent=134"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/categories?post=134"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/tags?post=134"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}