Act as one of three tenants on a shared API and watch the gateway decide what you are allowed to do.
Three companies share one API. Each has its own key, its own permissions, its own request rate and its own monthly token allowance. Type a company name and a question and the gateway runs the whole check list on your request: is this key real, is it allowed to do this, is this company sending requests too fast, and is there enough allowance left to pay for the answer.
The most important part is the privacy check. One company stores a confidential document, then a second company asks for that exact document by its id. The answer it gets back is the same "not found" it would get for an id that never existed, so it cannot use the API to work out what the first company is storing. The page shows both replies side by side.
You also see the money side: the allowance is reserved before the model is called and settled at the real cost afterwards, every request is metered including the rejected ones, and the billing summary at the bottom is the invoice that falls out of it.
MULTI-TENANT API GATEWAY driven in process, no socket bound
==============================================================================
Three tenants on one gateway. Ordering of the checks is fixed:
authenticate -> authorise (scope) -> rate limit -> reserve budget -> model
tenant plan scopes on the key rate limit monthly tokens
------------------------------------------------------------------------------------
acme growth read write complete usage 20 burst, 10 per sec 4000
globex starter read write complete usage 20 burst, 10 per sec 260
initech starter read usage (read only) 3 burst, 0.5 per sec 1000
key issued to globex mtk_globex_dfbcf3b4_c6aac03b86b5871d0646b4bd352479d2
written to the log as mtk_globex_dfbcf3b4_********
stored on the server salted sha256 bef71bc54f884e34971e...
the plaintext secret exists nowhere on the server after issue
------------------------------------------------------------------------------
1. YOUR REQUEST
------------------------------------------------------------------------------
tenant globex
request POST /v1/complete prompt='onboarding checklist'
authentication accepted, key mtk_globex_dfbcf3b4
scope required llm:complete granted: yes
rate limit 19.0 tokens in the bucket before, 18.0 after
status 200
grounding 1 of globex's own documents matched and went to the model; no other tenant's rows are reachable
budget reserved 106 tokens (prompt estimate plus the 64 token ceiling), held before the model ran
budget settled 71 tokens actually used, the hold released and recharged at the real number
allowance 260 available before, 189 after, out of 260
answer Globex onboarding checklist: an administrator verifies the domain,
sets the seat count, then invites members by email [S1].
billing line request req_6721c7423528 route POST /v1/complete status 200 tokens 71 cost $0.000024
------------------------------------------------------------------------------
2. TENANT ISOLATION globex asks for a document acme owns
------------------------------------------------------------------------------
acme POST /v1/documents -> 201 id=78502483b2ce4844bd875ad2283219ea
acme GET its own document -> 200 text='Acme internal: Q3 severance list, do n'...
globex GET acme's real document id -> 404 not_found
globex GET an id that never existed -> 404 not_found
the two reply bodies, with the id the caller itself supplied masked out
real id {"error":"not_found","message":"no document <the id the caller sent>"}
fake id {"error":"not_found","message":"no document <the id the caller sent>"}
byte identical: True (70 bytes each), status 404 == 404
so the endpoint cannot be used to test whether a document exists. A 403
here would confirm the id is real and let globex map acme's id space.
globex GET /v1/documents -> 200 count=1 (its own row only, not acme's two)
the store is keyed on (tenant, document), so there is no accessor
a developer could call that forgets to name the owner
------------------------------------------------------------------------------
3. PERMISSIONS AND KEYS
------------------------------------------------------------------------------
no key at all -> 401 unauthorized
forged key -> 401 unauthorized
initech writes with a read-only key -> 403 forbidden: needs documents:write
initech holds ['documents:read', 'usage:read']
a route that does not exist -> 404 not_found
------------------------------------------------------------------------------
4. RATE LIMIT initech's bucket holds 3 and refills at 0.5 per second
------------------------------------------------------------------------------
six requests back to back [200, 200, 200, 429, 429, 429]
burst absorbed 3 requests, first 429 at request 4
Retry-After header 2 s (a 429 with no Retry-After produces a hot retry loop)
the clock is moved forward 2.2 s instead of sleeping, which is the same
refill arithmetic the bucket does against a real clock
after the refill 200, bucket now holds 0.10 tokens
------------------------------------------------------------------------------
5. TOKEN BUDGET globex has a 260 token monthly allowance
------------------------------------------------------------------------------
call status reserved charged committed available
---------------------------------------------------------
1 200 95 56 127 133
2 200 95 56 183 77
3 402 95 - 183 77 budget_exhausted
the reservation is taken before the model is called, so a request that
would blow the cap is refused rather than billed. 402 is not 429: a 429
clears in seconds, a 402 clears when the quota changes or the month rolls.
reservations left open after settlement: 0 (any other number means quota leaked)
------------------------------------------------------------------------------
6. REVOCATION AND THE BILL
------------------------------------------------------------------------------
acme before revoke -> 200 after revoke -> 401 api key has been revoked
tenant requests billable rejected tokens cost usd statuses
----------------------------------------------------------------------------
acme 5 4 1 0 0.000000 {'201': 2, '200': 2, '404': 1}
globex 8 5 3 183 0.000063 {'201': 1, '200': 4, '404': 2, '402': 1}
initech 8 4 4 0 0.000000 {'403': 1, '200': 4, '429': 3}
rejected requests are metered too, at zero cost. A meter that counts only
successes cannot answer the question support actually gets asked.
the last two structured log lines, keys redacted:
{"ts":1791824357.809,"request_id":"req_62bcb0831f1c","tenant":"acme","key":"mtk_acme_111181d3_********","route":"GET /v1/documents","status":200,"toke
{"ts":1791824357.809,"request_id":"req_14c85dcc8774","tenant":"-","key":"mtk_acme_111181d3_********","route":"GET /v1/documents","status":401,"tokens"
24 requests handled in this page load, all in process.
Nothing here bound a port: the same Gateway.handle that server.py calls was
called directly, and the rate-limit clock was advanced instead of slept.