I’ve written before about how AI agents are just CI pipelines with an LLM plugged in. That post mapped agent concepts to infrastructure patterns you already know. But there’s a discipline forming around the infrastructure side of agents that deserves its own name.
Harness engineering. It’s the practice of building everything around the LLM — the execution environment, tool definitions, safety boundaries, observability, and lifecycle management. The stuff that turns a chatbot into a production system.
If you work in DevOps, you’ve been doing this for years. You just called it something else.
Why Harnesses Matter More Than Models
Pick any AI agent demo. Strip out the model. What’s left?
A container or sandbox. A set of callable tools. A loop that reads output and decides what happens next. Logging. Timeouts. Cleanup.
That’s the harness. And it’s where agents succeed or fail. A great model in a bad harness hallucinates, loops forever, leaks secrets, or silently does nothing useful. A decent model in a good harness stays bounded, recovers from errors, and produces auditable results.
DevOps engineers already think this way. You don’t just pick a good application — you build the infrastructure that makes it reliable. Same thing here.
The Five Parts of a Harness
Here’s how I break down harness engineering into components. Each one maps directly to something you’ve built before.
1. Execution environment. Where does the agent run? A container, a VM, a temporary directory, a git worktree. You need isolation so the agent can’t corrupt shared state. You need reproducibility so runs are consistent. This is the same problem as CI job runners. Docker, Firecracker, nsjail — pick your isolation boundary.
2. Tool definitions. Tools are the agent’s API surface. Read a file. Run a command. Query a database. Call an endpoint. Each tool needs input validation, output formatting, error handling, and permission scoping. Think of it like designing an API — you wouldn’t expose raw database access through a REST endpoint. Don’t give an agent raw shell access either. The tool layer is your contract.
3. Control loop. Observe, decide, execute, verify. The loop is what makes an agent an agent instead of a one-shot prompt. Your job as a harness engineer is to decide: how many iterations? What’s the timeout per step? What happens when a tool call fails? When does the loop escalate to a human? This is the same logic you put in health check loops and deployment rollback controllers.
4. Guardrails. Cost caps. Token limits. Command allowlists. File path restrictions. Rate limiting on external calls. Without guardrails, an agent can burn through your API budget in minutes or write to paths it shouldn’t touch. Every guardrail is a policy decision — same as IAM policies, network rules, and resource quotas you already manage.
5. Observability. If you can’t see what the agent did, you can’t debug it, audit it, or trust it. Log every tool call, every LLM response, every decision point. Capture diffs, timing, token usage, and cost. This is no different from structured logging in any production system. The difference is that agent traces are longer and less predictable than HTTP request traces, so you need good tooling to navigate them.
Where DevOps Context Overlaps
Here’s where your existing skills plug in directly.
Infrastructure as code. Agent harnesses should be declarative and version-controlled. The tool definitions, policies, and environment specs should live in config files, not hardcoded in application logic. When you change a tool’s behavior, that change should be reviewable in a PR.
Pipeline orchestration. Multi-agent systems look a lot like multi-stage pipelines. One agent does research, passes context to a planning agent, which passes a plan to an implementation agent. You’re managing handoffs, shared artifacts, and failure propagation — the same coordination problem as CI/CD stages.
Incident response. When an agent goes wrong, you need the same muscle memory. Check the logs. Find the failing step. Understand the input that caused it. Roll back if needed. The debugging workflow is identical.
Security boundaries. Least privilege applies to agents just like it applies to services. What tools can this agent access? What files can it read? Can it make network calls? Can it spend money? Every agent needs a security boundary, and DevOps engineers already think in terms of boundaries.
Getting Started
If you want to start building harnesses, you don’t need a new framework. Start with what you have.
Take a simple task — say, analyzing a failed CI build. Write a script that collects the logs, sends them to an LLM with a prompt, parses the response, and posts a summary to Slack. That’s a harness. A minimal one, but it has all the components: environment setup, tool use (log collection, Slack posting), a control flow, and output handling.
Then add complexity. Let the LLM decide which logs to fetch. Add a retry loop. Add a cost cap. Add structured logging. Each addition is a harness engineering decision.
You don’t need to learn ML. You don’t need to fine-tune models. You need to build the infrastructure that makes models useful — and that’s the job you already do.
Harness engineering isn’t a new discipline. It’s DevOps applied to a new kind of workload. The sooner you see it that way, the faster you’ll build agents that actually work in production.