Ask for a calculation or a lookup and watch the request turn into a checked tool call, including the ones the sandbox refuses to run.
When an assistant can run code on your behalf, the dangerous part is not the answer, it is everything between the request and the call. The arguments arrive as text and may be the wrong type. The tool name may not exist. The expression may be an attempt to open a file, import a module or run an exponent big enough to pin a processor for a week.
Type a request and this page shows the whole path. Which of the four tools was picked, the exact description and parameter list that tool publishes, the arguments as they were drafted, what they looked like after being checked and converted, and then the sandbox: allowlist, time budget, retries, output size cap, and the one line of result the assistant would actually see.
At the bottom, a fixed set of hostile inputs runs on every request so you can watch them get turned down and read the reason for each one.
THE REQUEST
__import__("os").system("echo pwned")
STEP 1 WHICH TOOL
--------------------------------------------------------------------------
available: calculate, convert_units, current_time, search_docs
picked: calculate
because: the text looks like code rather than a question, so it goes to the calculator, which refuses anything that is not arithmetic
This page has no API key, so the choice above is made by a small router
in the page rather than by a model. Everything below it is the real thing.
STEP 2 WHAT THAT TOOL PUBLISHES (generated from its signature and docstring)
--------------------------------------------------------------------------
name: calculate@1.0.0
description: Evaluate an arithmetic expression safely, without eval.
tags: math time budget: 1s
parameters:
expression string required, maxLength=200
An arithmetic expression such as "600 * 60 / 1000" using + - * / // % ** and the functions abs, round, min, max, sqrt, floor, ceil, log, log10 and exp.
precision integer default=4, minimum=0, maximum=10
Decimal places to round the result to.
STEP 3 ARGUMENTS
--------------------------------------------------------------------------
drafted: {"expression": "__import__(\"os\").system(\"echo pwned\")", "precision": "4"}
checked: {"expression": "__import__(\"os\").system(\"echo pwned\")", "precision": 4}
precision: '4' -> 4
problems: none
STEP 4 THE SANDBOX
--------------------------------------------------------------------------
on the allowlist: yes
circuit breaker: closed, this sandbox is fresh for your request
outcome: error
cost: 0.29 ms, 1 attempt(s), 0 bytes returned
refused with code error:
ValueError: rejected expression: double underscore is not allowed
the single line the assistant is given back:
ERROR error: ValueError: rejected expression: double underscore is not allowed
WHAT THE SANDBOX TURNS DOWN (run fresh on every request)
--------------------------------------------------------------------------
input outcome why
__import__("os").system("echo pwned") error double underscore is not allowed
().__class__.__base__.__subclasses__() error double underscore is not allowed
2 ** 10000000 error exponent 10000000 exceeds the limit of 64
open('/etc/passwd').read() error only calls to a plain allowlisted function name are permitted
eval('1+1') error function 'eval' is not allowed
1/0 error division by zero
delete_database(table=customers) unknown_tool there is no tool by that name
calculate, on a read-only agent denied this agent's allowlist holds search_docs only
calculate(precision=99) invalid_args must be <= 10, got 99
1+1, after three refusals in a row circuit_open the tool is taken out of service for a cooldown
--------------------------------------------------------------------------
None of these are matched against a list of banned words. The calculator
parses the expression and allows a short list of arithmetic node types,
so anything it has not been taught about is refused by default.