Skip to content
Nubent

/Glossary · DevOps

What is AgentOps?

AgentOps is the set of practices for taking AI agents to production and running them safely: which tools and permissions each agent has, how its decisions are logged, how its quality is evaluated, what each run costs, and where a person signs off.

From DevOps to AgentOps: what changes and what stays

DevOps brought order to how software is built, deployed, and operated. MLOps then added the lifecycle of machine learning models, and LLMOps the lifecycle of applications built on language models: prompts, model versions, evaluations, and cost per token. AgentOps is the next step, because an agent does not just answer: it decides what to do and takes actions with tools.

What does not change is the DevOps foundation: infrastructure as code, pipelines, observability, incident management, and continuous improvement. AgentOps does not replace those practices; it extends them to a system that acts on its own. That is why teams that already run AWS well are halfway to running agents.

What you are operating when you operate an agent

An agent combines a model, such as Claude on Amazon Bedrock, with instructions, memory, and tools that let it read data or change systems. Operating it means versioning all of that together, knowing which tool it called at each step, with which arguments and what result, and being able to replay a run when something goes wrong.

It also means limits: how many steps it can take, how much it can spend on tokens, which actions need human approval, and what happens when a tool fails. Without those limits, an agent that works in a demo can loop, overspend, or touch something in production it never should have.

AgentOps practices on AWS

The first practice is least privilege: each agent gets its own IAM role, read-only by default, and changes arrive as pull requests instead of being applied directly. Tools are exposed in a controlled way, for example through MCP servers, and every invocation is logged in CloudTrail and in the application logs.

The second is measurement: traces for every run, automated evaluations on real cases before switching models or prompts, per-agent cost alarms in CloudWatch, and budgets in AWS Budgets. The third is human control at the risky points, with an audit trail of who approved what and based on which evidence.

/FAQ

Questions about AgentOps

Another question? Write to hello@nubent.com and an engineer will answer.

Is AgentOps the same as AIOps?

No. AIOps means using AI to operate IT systems, for example to correlate alerts or detect anomalies. AgentOps means operating the AI agents themselves: their permissions, traces, evaluations, and costs. They complement each other: an agent that helps resolve incidents is AIOps, and running it well is AgentOps.

Does AgentOps replace DevOps?

No. AgentOps builds on DevOps: agents are deployed through pipelines, their infrastructure is defined as code, and they are observed with the same tools. What it adds are controls specific to systems that make decisions and act on their own, such as step limits, evaluations, and human approvals.

What is the difference between AgentOps, MLOps, and LLMOps?

MLOps covers the lifecycle of models you train yourself: data, training, deployment, and monitoring. LLMOps covers applications that use language models: prompts, versions, and evaluations. AgentOps adds what happens when the model uses tools and takes actions: permissions, step-by-step traces, and human control.

When do I need AgentOps practices?

As soon as an agent can read sensitive data or change a system, even a test environment. Start with least privilege, logging of every tool call, and a spending cap, then add evaluations and approvals as the agent takes on more responsibility.

Next step

What should we build next on AWS?

30 minutes with an engineer. Concrete next steps, whether or not you work with us.