Give the model a token budget and watch six sources of context compete for it.
A chat answer is built from several sources at once: the system instructions, what the assistant remembers about you, documents it looked up, live tool readings, the conversation so far, and your actual question. Together they are usually bigger than the space the model has to read them in.
Type a number and that becomes the size of the window. The service works out how much each source gets, shortens what can be shortened, drops what will not fit, and prints a receipt explaining every decision. Type a question after the number and the documents are re-ranked against it, so the sentences that survive are the ones about your question.
Underneath the receipt is the same request done the usual way, by gluing every source together and cutting the end off when it is too long. That version regularly throws away the question itself, because the question happens to be at the end of the string.
THE REQUEST
question: Why do we keep getting 429 responses at night and will more tokens help?
model window: 320 tokens
completion reserve: 96 tokens, held back so the model can answer
budget for context: 224 tokens
sources competing: 6, offering 951 tokens between them
==============================================================================
WHAT THE ASSEMBLER DECIDED
==============================================================================
CONTEXT RECEIPT request=web-320
query: Why do we keep getting 429 responses at night and will more tokens help?
window=320 reserve=96 budget=224 used=135 headroom=89
section order: instructions -> user_memory -> question
section prio offered granted used keep in/cmp/drop note
-------------------------------------------------------------------------------
instructions 10.0 105 71 63 60% 1/0/1 partially funded, compression required
user_memory 6.0 67 51 51 76% 1/1/0 partially funded, compression required
retrieved_docs 5.0 459 0 0 0% 0/0/6 shed: guaranteed minimums of higher-priority sections filled the window
tool_output 4.0 152 0 0 0% 0/0/1 shed: guaranteed minimums of higher-priority sections filled the window
chat_history 2.0 147 0 0 0% 0/0/8 shed: guaranteed minimums of higher-priority sections filled the window
question 10.0 21 21 21 100% 1/0/0 fully funded
compressed:
mem.incident extractive 32 -> 16 (50% kept)
dropped:
sys.tools 42 tokens reason: section budget exhausted
doc.auth-overview 87 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
doc.plans 78 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
doc.sdk 71 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
doc.errors 76 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
doc.audit 59 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
doc.rate-limits 86 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
tool.metrics 147 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
turn.7 17 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
turn.6 13 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
turn.5 18 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
turn.4 16 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
turn.3 23 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
turn.2 12 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
turn.1 21 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
turn.0 22 tokens reason: shed: guaranteed minimums of higher-priority sections filled the window
deduplicated across sources:
mem.limit (user_memory) == doc.rate-limits (retrieved_docs) containment=1.0 saved 24 tokens
savings: 16 tokens from compression, 24 tokens from deduplication
==============================================================================
THE PROMPT THAT RECEIPT DESCRIBES
==============================================================================
# Instructions
You are Meridian Support Copilot. Answer only from the evidence supplied in this prompt. If the evidence does not contain the answer, say so and name the document the user should read instead. Never guess a number, a limit or a date.
# What we already know about this workspace
The user's workspace is on the Growth plan and is pinned to the eu-west region.
Two weeks ago this user hit sustained 429s during a nightly batch
# Question
Why do we keep getting 429 responses at night and will more tokens help?
==============================================================================
THE SAME REQUEST, GLUED TOGETHER THE USUAL WAY
==============================================================================
everything joined, no budget awareness: 976 tokens
budget: 224 tokens
over the budget by: 752 tokens
in production that is a rejected call, or an answer that stops mid-sentence
assembled instead: 135 tokens, 89 to spare
tokens saved: 841
cut the joined string down to the budget and this is what is left of each source:
source offered kept what the cut did
--------------------------------------------------------------------
instructions 103 103 kept whole
user_memory 81 81 kept whole
retrieved_docs 457 26 cut mid-source, 5% left
tool_output 147 0 deleted entirely
chat_history 142 0 deleted entirely
question 19 0 deleted entirely
high-value sources still represented: 46% cutting the string, 31% assembling it
the cut keeps the front of the prompt whole and deletes whatever ran last, which
is why the question is the thing at risk. the assembler spends a little of every
source's budget instead, and writes down what it spent it on.