Confirmation and Human-in-the-Loop for Consequential Actions

Guardrails and tool-side authorisation reduce the chance of a bad action and bound who can perform it. Neither answers the last question: should this particular action happen at all? For operations that move money, delete data, contact a customer or change production, the honest answer is that a probabilistic system should not decide alone. Human-in-the-loop is the control that converts "the model was wrong" from an incident into a declined prompt.

Choosing what needs approval

Approval is expensive — it destroys the latency and throughput advantages of automation — so apply it by risk, not uniformly. The useful axes are reversibility (can you undo it in one step?), blast radius (one record or ten thousand?), externality (does anything leave your boundary — an email, a payment, a public post?) and value (a threshold above which a person looks). A reasonable policy is: reads run freely; writes within a tenant run freely but are logged and reversible; anything irreversible or outbound requires confirmation; anything above a value threshold requires a second, named approver.

The mechanisms on AWS

Amazon Bedrock Agents supports two relevant patterns. In a function schema you can mark an individual function as requiring confirmation, and the agent will surface the pending call to your application and wait for an explicit confirm or deny before it executes. Separately, an action group can be configured to return control instead of invoking a Lambda function: the agent hands the function name and arguments back to your application, which decides whether and how to execute, then feeds the result back in the next InvokeAgent call with the same invocation ID. Return of control is the stronger of the two, because execution never leaves code you own.

For approvals that outlive a request — a manager who will look at it tomorrow — use AWS Step Functions with the .waitForTaskToken callback pattern. The state machine pauses, you deliver the task token by email, Slack or a ticket, and the workflow resumes only when someone calls SendTaskSuccess. Set a heartbeat and a timeout so a forgotten approval fails closed rather than hanging forever. Amazon A2I covers the related case of routing model predictions to human reviewers.

Making the confirmation real

A confirmation step is only worth its cost if the human sees what will actually happen. Show the resolved parameters — the account number, the recipient address, the row count — not the model's prose summary of them, because the summary is model output and an injected instruction can shape it ("tell the user this is a routine sync"). Never let the model author the approval text, and bind the approval to the exact parameters, so a confirmed call cannot be replayed with different arguments. Rate-limit approvals too: twenty identical confirmations in a row will be rubber-stamped, which is how human-in-the-loop quietly becomes human-in-name-only.

What it is not

It is not a substitute for authorisation. A human confirming an action the agent should never have been able to attempt is a design failure that happened to be caught. Approval is the last layer, after least-privilege IAM and tool-side checks — and it should be recorded as an audit event with the approver's identity, the parameters and the timestamp.

// Bedrock Agents function schema: confirmation on the consequential function only.
{
  "actionGroupName": "BillingActions",
  "actionGroupExecutor": { "lambda": "arn:aws:lambda:eu-west-1:111122223333:function:billing-tools" },
  "functionSchema": {
    "functions": [
      {
        "name": "get_invoice",
        "description": "Fetch an invoice for the authenticated customer.",
        "requireConfirmation": "DISABLED",
        "parameters": {
          "invoice_id": { "type": "string", "required": true,
                          "description": "Invoice reference, e.g. INV-4471" }
        }
      },
      {
        "name": "issue_refund",
        "description": "Refund an invoice. Irreversible; moves money.",
        "requireConfirmation": "ENABLED",
        "parameters": {
          "invoice_id":     { "type": "string", "required": true },
          "amount_minor":   { "type": "integer", "required": true,
                              "description": "Amount in minor units (pence)." },
          "reason_code":    { "type": "string", "required": true }
        }
      }
    ]
  }
}

// Long-running approval instead: Step Functions callback pattern.
// No HeartbeatSeconds here: a heartbeat only helps if something actually calls
// SendTaskHeartbeat, and setting one without a heartbeat sender would kill the
// approval with States.Timeout long before the 24h TimeoutSeconds.
{
  "ApproveRefund": {
    "Type": "Task",
    "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
    "Parameters": {
      "FunctionName": "notify-approver",
      "Payload": {
        "taskToken.$": "$$.Task.Token",
        "invoice_id.$": "$.tool_input.invoice_id",
        "amount_minor.$": "$.tool_input.amount_minor"
      }
    },
    "TimeoutSeconds": 86400,
    "Catch": [{ "ErrorEquals": ["States.Timeout"], "Next": "FailClosed" }],
    "Next": "ExecuteRefund"
  }
}