Guardrails
Constrain what an agent can do so autonomy stays safe.
An autonomous agent taking real actions needs guardrails — limits that keep it safe, on-budget, and under control.
Common guardrails
- Step limits — cap the number of loop iterations.
- Tool permissions — only expose safe tools; keep destructive ones off-limits or behind approval.
- Human-in-the-loop — require confirmation before high-impact actions (payments, deletions, emails).
- Input/output validation — check tool arguments and results.
- Budget limits — stop when a token or time budget is exceeded.
Example
{
"agent_config": {
"max_steps": 8,
"allowed_tools": ["search", "read_file", "summarize"],
"require_approval_for": ["send_email", "delete_record", "charge_card"],
"token_budget": 50000
}
}When to use it
- A customer-service agent has a guardrail that prevents it from issuing refunds above $500 without escalating to a human agent.
- A code-execution agent is restricted to a sandboxed directory and cannot run rm commands, preventing accidental data deletion.
- An autonomous email agent is limited to reading and drafting emails, but requires human approval before actually sending any message.
More examples
Allowlist of permitted tool actions
Checks every tool call against an allowlist before execution, blocking dangerous actions at the dispatch layer.
PERMITTED_TOOLS = {'read_file', 'search_web', 'send_email'}
BLOCKED_TOOLS = {'delete_file', 'run_shell', 'modify_db'}
def safe_dispatch(tool_name, args):
if tool_name in BLOCKED_TOOLS:
return f'ERROR: Tool "{tool_name}" is not permitted for this agent.'
if tool_name not in PERMITTED_TOOLS:
return f'ERROR: Unknown tool "{tool_name}".'
# Route to real implementation
return f'[{tool_name}] called with {args}'
print(safe_dispatch('delete_file', {'path': '/data/users.db'}))
print(safe_dispatch('search_web', {'query': 'AI news'}))Human-in-the-loop approval gate
Pauses the agent before high-risk actions and requires explicit human approval before proceeding.
HIGH_RISK_TOOLS = {'send_email', 'process_payment', 'delete_record'}
def approve_action(tool_name, args):
if tool_name in HIGH_RISK_TOOLS:
print(f'APPROVAL REQUIRED: {tool_name}({args})')
answer = input('Approve? (yes/no): ').strip().lower()
return answer == 'yes'
return True # low-risk tools auto-approved
# Simulate agent wanting to send an email
tool_name = 'send_email'
args = {'to': '[email protected]', 'subject': 'Your order shipped'}
if approve_action(tool_name, args):
print(f'Executing {tool_name}')
else:
print('Action rejected by operator')Output content moderation guardrail
Runs the agent's output through the moderation API before it is shown to the user, blocking policy violations.
from openai import OpenAI
client = OpenAI()
def moderate(text):
r = client.moderations.create(input=text)
return r.results[0].flagged
agent_output = 'Here is the information you requested: ...'
if moderate(agent_output):
print('Output blocked by moderation.')
else:
print('Safe output:', agent_output)
Discussion