Ask for a calculation or a lookup and watch the request turn into a checked tool call, including the ones the sandbox refuses to run.
When an assistant can run code on your behalf, the dangerous part is not the answer, it is everything between the request and the call. The arguments arrive as text and may be the wrong type. The tool name may not exist. The expression may be an attempt to open a file, import a module or run an exponent big enough to pin a processor for a week.
Type a request and this page shows the whole path. Which of the four tools was picked, the exact description and parameter list that tool publishes, the arguments as they were drafted, what they looked like after being checked and converted, and then the sandbox: allowlist, time budget, retries, output size cap, and the one line of result the assistant would actually see.
At the bottom, a fixed set of hostile inputs runs on every request so you can watch them get turned down and read the reason for each one.
THE REQUEST
convert 5 miles to km
STEP 1 WHICH TOOL
--------------------------------------------------------------------------
available: calculate, convert_units, current_time, search_docs
picked: convert_units
because: '5 miles to km' is a unit conversion
This page has no API key, so the choice above is made by a small router
in the page rather than by a model. Everything below it is the real thing.
STEP 2 WHAT THAT TOOL PUBLISHES (generated from its signature and docstring)
--------------------------------------------------------------------------
name: convert_units@1.0.0
description: Convert a quantity between units of length, mass or temperature.
tags: math, convert time budget: 1s
parameters:
value number required
The quantity to convert.
from_unit string required, one of m|km|cm|mm|mi|ft|in|kg|...
The unit the value is currently in.
to_unit string required, one of m|km|cm|mm|mi|ft|in|kg|...
The unit to convert to.
STEP 3 ARGUMENTS
--------------------------------------------------------------------------
drafted: {"value": "5", "from_unit": "mi", "to_unit": "km"}
checked: {"value": 5.0, "from_unit": "mi", "to_unit": "km"}
value: '5' -> 5.0
problems: none
STEP 4 THE SANDBOX
--------------------------------------------------------------------------
on the allowlist: yes
circuit breaker: closed, this sandbox is fresh for your request
outcome: ok
cost: 0.29 ms, 1 attempt(s), 59 bytes returned
result:
{
"value": 5.0,
"from": "mi",
"to": "km",
"result": 8.04672
}
the single line the assistant is given back:
{"value": 5.0, "from": "mi", "to": "km", "result": 8.04672}
WHAT THE SANDBOX TURNS DOWN (run fresh on every request)
--------------------------------------------------------------------------
input outcome why
__import__("os").system("echo pwned") error double underscore is not allowed
().__class__.__base__.__subclasses__() error double underscore is not allowed
2 ** 10000000 error exponent 10000000 exceeds the limit of 64
open('/etc/passwd').read() error only calls to a plain allowlisted function name are permitted
eval('1+1') error function 'eval' is not allowed
1/0 error division by zero
delete_database(table=customers) unknown_tool there is no tool by that name
calculate, on a read-only agent denied this agent's allowlist holds search_docs only
calculate(precision=99) invalid_args must be <= 10, got 99
1+1, after three refusals in a row circuit_open the tool is taken out of service for a cooldown
--------------------------------------------------------------------------
None of these are matched against a list of banned words. The calculator
parses the expression and allows a short list of arithmetic node types,
so anything it has not been taught about is refused by default.