Give the model a token budget and watch six sources of context compete for it.
A chat answer is built from several sources at once: the system instructions, what the assistant remembers about you, documents it looked up, live tool readings, the conversation so far, and your actual question. Together they are usually bigger than the space the model has to read them in.
Type a number and that becomes the size of the window. The service works out how much each source gets, shortens what can be shortened, drops what will not fit, and prints a receipt explaining every decision. Type a question after the number and the documents are re-ranked against it, so the sentences that survive are the ones about your question.
Underneath the receipt is the same request done the usual way, by gluing every source together and cutting the end off when it is too long. That version regularly throws away the question itself, because the question happens to be at the end of the string.
THE REQUEST
question: Why do we keep getting 429 responses at night and will more tokens help?
model window: 800 tokens
completion reserve: 240 tokens, held back so the model can answer
budget for context: 560 tokens
sources competing: 6, offering 951 tokens between them
==============================================================================
WHAT THE ASSEMBLER DECIDED
==============================================================================
CONTEXT RECEIPT request=web-800
query: Why do we keep getting 429 responses at night and will more tokens help?
window=800 reserve=240 budget=560 used=521 headroom=39
section order: instructions -> user_memory -> retrieved_docs -> chat_history -> tool_output -> question
section prio offered granted used keep in/cmp/drop note
-------------------------------------------------------------------------------
instructions 10.0 105 105 105 100% 2/0/0 fully funded
user_memory 6.0 67 67 67 100% 2/0/0 fully funded
retrieved_docs 5.0 459 179 164 36% 0/6/0 partially funded, compression required
tool_output 4.0 152 129 128 84% 0/1/0 partially funded, compression required
chat_history 2.0 147 39 35 24% 2/0/6 partially funded, compression required
question 10.0 21 21 21 100% 1/0/0 fully funded
compressed:
doc.auth-overview extractive 87 -> 32 (37% kept)
doc.plans extractive 78 -> 19 (24% kept)
doc.sdk extractive 71 -> 26 (37% kept)
doc.errors extractive 76 -> 17 (22% kept)
doc.audit extractive 59 -> 25 (42% kept)
doc.rate-limits extractive 86 -> 43 (50% kept)
tool.metrics truncate_tail 147 -> 123 (84% kept)
dropped:
turn.5 18 tokens reason: section budget exhausted
turn.4 16 tokens reason: section budget exhausted
turn.3 23 tokens reason: section budget exhausted
turn.2 12 tokens reason: section budget exhausted
turn.1 21 tokens reason: section budget exhausted
turn.0 22 tokens reason: section budget exhausted
deduplicated across sources:
mem.limit (user_memory) == doc.rate-limits (retrieved_docs) containment=1.0 saved 24 tokens
savings: 319 tokens from compression, 24 tokens from deduplication
==============================================================================
THE PROMPT THAT RECEIPT DESCRIBES
==============================================================================
# Instructions
You are Meridian Support Copilot. Answer only from the evidence supplied in this prompt. If the evidence does not contain the answer, say so and name the document the user should read instead. Never guess a number, a limit or a date.
You may cite the tool output block. Tool readings are live and override any figure in a document when the two disagree, but you must say which one you used.
# What we already know about this workspace
The user's workspace is on the Growth plan and is pinned to the eu-west region.
Two weeks ago this user hit sustained 429s during a nightly batch job and asked whether adding a second token would raise their ceiling.
# Documentation
Tokens are issued per workspace and never per user. Tokens are shown exactly once at creation and stored only as a salted hash.
Meridian publishes official SDKs for Python, TypeScript and Go. The Python SDK requi
... (preview cut here, the real prompt continues)
==============================================================================
THE SAME REQUEST, GLUED TOGETHER THE USUAL WAY
==============================================================================
everything joined, no budget awareness: 976 tokens
budget: 560 tokens
over the budget by: 416 tokens
in production that is a rejected call, or an answer that stops mid-sentence
assembled instead: 521 tokens, 39 to spare
tokens saved: 455
cut the joined string down to the budget and this is what is left of each source:
source offered kept what the cut did
--------------------------------------------------------------------
instructions 103 103 kept whole
user_memory 81 81 kept whole
retrieved_docs 457 362 cut mid-source, 79% left
tool_output 147 0 deleted entirely
chat_history 142 0 deleted entirely
question 19 0 deleted entirely
high-value sources still represented: 77% cutting the string, 100% assembling it
the cut keeps the front of the prompt whole and deletes whatever ran last, which
is why the question is the thing at risk. the assembler spends a little of every
source's budget instead, and writes down what it spent it on.