Act as one of three tenants on a shared API and watch the gateway decide what you are allowed to do.
Three companies share one API. Each has its own key, its own permissions, its own request rate and its own monthly token allowance. Type a company name and a question and the gateway runs the whole check list on your request: is this key real, is it allowed to do this, is this company sending requests too fast, and is there enough allowance left to pay for the answer.
The most important part is the privacy check. One company stores a confidential document, then a second company asks for that exact document by its id. The answer it gets back is the same "not found" it would get for an id that never existed, so it cannot use the API to work out what the first company is storing. The page shows both replies side by side.
You also see the money side: the allowance is reserved before the model is called and settled at the real cost afterwards, every request is metered including the rejected ones, and the billing summary at the bottom is the invoice that falls out of it.
MULTI-TENANT API GATEWAY driven in process, no socket bound
==============================================================================
Three tenants on one gateway. Ordering of the checks is fixed:
authenticate -> authorise (scope) -> rate limit -> reserve budget -> model
tenant plan scopes on the key rate limit monthly tokens
------------------------------------------------------------------------------------
acme growth read write complete usage 20 burst, 10 per sec 4000
globex starter read write complete usage 20 burst, 10 per sec 260
initech starter read usage (read only) 3 burst, 0.5 per sec 1000
key issued to acme mtk_acme_ae957492_d42453f23bc770ac7af3830c479f6c58
written to the log as mtk_acme_ae957492_********
stored on the server salted sha256 3a092a6817471a8d5adc...
the plaintext secret exists nowhere on the server after issue
------------------------------------------------------------------------------
1. YOUR REQUEST
------------------------------------------------------------------------------
tenant acme
request POST /v1/complete prompt='refund policy'
authentication accepted, key mtk_acme_ae957492
scope required llm:complete granted: yes
rate limit 18.0 tokens in the bucket before, 17.0 after
status 200
grounding 1 of acme's own documents matched and went to the model; no other tenant's rows are reachable
budget reserved 123 tokens (prompt estimate plus the 64 token ceiling), held before the model ran
budget settled 105 tokens actually used, the hold released and recharged at the real number
allowance 4000 available before, 3895 after, out of 4000
answer Acme refund policy: a refund is issued within fourteen days of
purchase, charged back to the original card, and the workspace
stays active until the end of the paid period [S1].
billing line request req_d7ebeb7d6adb route POST /v1/complete status 200 tokens 105 cost $0.000036
------------------------------------------------------------------------------
2. TENANT ISOLATION globex asks for a document acme owns
------------------------------------------------------------------------------
acme POST /v1/documents -> 201 id=49699d75573246299710e7d0b7ba96a7
acme GET its own document -> 200 text='Acme internal: Q3 severance list, do n'...
globex GET acme's real document id -> 404 not_found
globex GET an id that never existed -> 404 not_found
the two reply bodies, with the id the caller itself supplied masked out
real id {"error":"not_found","message":"no document <the id the caller sent>"}
fake id {"error":"not_found","message":"no document <the id the caller sent>"}
byte identical: True (70 bytes each), status 404 == 404
so the endpoint cannot be used to test whether a document exists. A 403
here would confirm the id is real and let globex map acme's id space.
globex GET /v1/documents -> 200 count=1 (its own row only, not acme's two)
the store is keyed on (tenant, document), so there is no accessor
a developer could call that forgets to name the owner
------------------------------------------------------------------------------
3. PERMISSIONS AND KEYS
------------------------------------------------------------------------------
no key at all -> 401 unauthorized
forged key -> 401 unauthorized
initech writes with a read-only key -> 403 forbidden: needs documents:write
initech holds ['documents:read', 'usage:read']
a route that does not exist -> 404 not_found
------------------------------------------------------------------------------
4. RATE LIMIT initech's bucket holds 3 and refills at 0.5 per second
------------------------------------------------------------------------------
six requests back to back [200, 200, 200, 429, 429, 429]
burst absorbed 3 requests, first 429 at request 4
Retry-After header 2 s (a 429 with no Retry-After produces a hot retry loop)
the clock is moved forward 2.2 s instead of sleeping, which is the same
refill arithmetic the bucket does against a real clock
after the refill 200, bucket now holds 0.10 tokens
------------------------------------------------------------------------------
5. TOKEN BUDGET globex has a 260 token monthly allowance
------------------------------------------------------------------------------
call status reserved charged committed available
---------------------------------------------------------
1 200 95 56 56 204
2 200 95 56 112 148
3 200 95 56 168 92
4 402 95 - 168 92 budget_exhausted
the reservation is taken before the model is called, so a request that
would blow the cap is refused rather than billed. 402 is not 429: a 429
clears in seconds, a 402 clears when the quota changes or the month rolls.
reservations left open after settlement: 0 (any other number means quota leaked)
------------------------------------------------------------------------------
6. REVOCATION AND THE BILL
------------------------------------------------------------------------------
acme before revoke -> 200 after revoke -> 401 api key has been revoked
tenant requests billable rejected tokens cost usd statuses
----------------------------------------------------------------------------
acme 6 5 1 105 0.000036 {'201': 2, '200': 3, '404': 1}
globex 8 5 3 168 0.000059 {'201': 1, '404': 2, '200': 4, '402': 1}
initech 8 4 4 0 0.000000 {'403': 1, '200': 4, '429': 3}
rejected requests are metered too, at zero cost. A meter that counts only
successes cannot answer the question support actually gets asked.
the last two structured log lines, keys redacted:
{"ts":1791824403.404,"request_id":"req_01375235185f","tenant":"acme","key":"mtk_acme_ae957492_********","route":"GET /v1/documents","status":200,"toke
{"ts":1791824403.404,"request_id":"req_461252404afe","tenant":"-","key":"mtk_acme_ae957492_********","route":"GET /v1/documents","status":401,"tokens"
25 requests handled in this page load, all in process.
Nothing here bound a port: the same Gateway.handle that server.py calls was
called directly, and the rate-limit clock was advanced instead of slept.