A pilot shows that an agent can finish a task while its builders watch. Production asks for more. The agent has to finish that task thousands of times with nobody watching, inside a budget, and with a named person accountable when it gets one wrong.
Most organizations are somewhere between the two. An EY survey fielded in mid 2026 found 91% of senior AI executives using agentic AI in pilots or full deployment, and about half of them said their governance had not been updated for it (Dark Reading). Projects stall in the space between those two numbers. I think the way through is the discipline we already apply to any production service: decide what the service is worth, set targets for how it must behave, and only then make promises to other people.
Start with one number the business already tracks
"The demo went well" is how a pilot gets approved. A production budget needs a number that finance or operations already reports. That might be tickets closed without a human touching them, or days from application to decision.
Measure that number before the agent goes live. Then work out the cost of one completed task: model tokens, tool calls, and the human time spent reviewing or correcting the output. If a completed task costs more than the manual process it replaces, the agent isn't ready, however good its answers look.
Write SLOs for the agent
A service level objective is an internal target for how a service behaves. Agents need them as much as any API does.
An agent is harder to pin down than an API because the same input can produce different outputs. So measure over a window, such as a rolling 28 days, and score a sample of real runs with a fixed evaluation set and human review.
| What to measure | How to measure it | Example target |
|---|---|---|
| Task success rate | Sampled runs scored by evaluators and human reviewers | 95% of tasks correct |
| Latency | End-to-end time per task, 95th percentile | Under 30 seconds |
| Cost per task | Tokens plus tool calls for each completed task | Under $0.40, with a hard stop at $2 |
| Escalation rate | Share of tasks handed to a person | 5% to 15% |
| Unauthorized actions | Tool calls blocked by policy or made outside it | None made, and every block reviewed |
The targets in the table are examples. Yours come from the baseline you measured.
Escalation rate gets a range on purpose. A high rate means the agent isn't doing the job. A rate near zero usually means it is acting on cases it should have handed off.
Each SLO also gives you an error budget. A 95% success target leaves 5% of tasks that can go wrong. When the agent uses up that budget, stop shipping prompt and model changes until it is back inside the target. Product and engineering agree to that rule before the argument starts, which is when it is easiest to agree.
Promise an SLA only after the SLOs hold
A service level agreement is a commitment to a customer or to another department, usually with a consequence attached. Set it looser than the SLO, and sign it after you have held the SLO for at least one full measurement window.
Be careful what you promise. You can commit to availability, to response time, and to how fast a person picks up an escalated case. I would not commit to the agent always being right. I would commit to how quickly a wrong action is caught and reversed.
Your SLA also can't be better than the services it depends on. Model quotas and throttling limit how much work the agent can do in an hour. Check your account's quotas for the models you use against your peak load, and decide in advance what happens when the model is slow or unavailable. The usual choices are to queue the work, fall back to a smaller model, or hand the task to a person.
Put a ceiling on spend
In May 2026 The Register reported on a customer building on Amazon Bedrock who expected a bill in the hundreds of dollars and received roughly $30,000 to $38,000. Cost anomaly alerts never fired, because the third-party model charges were billed through AWS Marketplace and the alerting did not cover them (The Register).
Budget alerts report spending after it has happened. One 2026 analysis put the worst-case delay between a cost and its alert at about 33 hours (paddo.dev). An agent stuck in a retry loop can spend a lot in 33 hours, and the AWS Well-Architected guidance for agentic AI lists runaway loops as a primary cost risk (AWS).
Four controls cover most of this:
- Cap the tokens and the number of steps in a single run, so one task can't run indefinitely.
- Separate each agent's usage so its cost appears on its own line.
- Add a circuit breaker that disables the agent when it reaches its daily budget. An alert alone leaves the agent running.
- Find out which of your model charges are billed through AWS Marketplace, and confirm your alerts can see them.
Scope the agent's access like a new hire's
An agent with tools is an identity that takes actions. Give each agent its own role, with permissions for the specific data and tools its task needs. Require a person's approval for anything that is hard to undo, such as payments, deletions, and messages to customers. Log every model call and every tool call so you can reconstruct what happened afterward.
Two references help here. In December 2025 AWS announced policy controls for Amazon Bedrock AgentCore that check each tool call against rules before it runs (Amazon). The same month, OWASP published its Top 10 for Agentic Applications, which works well as a review checklist before launch (OWASP).
Roll out in stages
- Shadow mode. The agent runs on real inputs, and its output is recorded and compared with what people did. It takes no action.
- Assisted. The agent proposes an action and a person approves each one.
- Supervised autonomy. The agent acts alone on low-risk tasks, and people review a sample.
Move to the next stage only when the SLOs held at the current one.
Treat your evaluation set like regression tests, and run it before every change to a prompt, a tool, or a model. The same AWS announcement included built-in evaluators in AgentCore, if you would rather not build your own. Give the agent an owner, a runbook, and an off switch that the person on call knows how to use.
A readiness check
A pilot is finished when you can answer all six of these:
- What number does this agent move, and what was it before?
- What does one completed task cost?
- What are the SLOs, and who reviews them each week?
- What stops the agent when it goes over budget?
- What can the agent touch, and which actions need a person?
- Who gets called when it goes wrong, and how do they turn it off?
Want a second pair of eyes before launch?
Forti365 reviews AI workloads on AWS for spend guardrails and security at a fixed price. Details are on the AI on AWS Guardrail and Security Review page.