Anthropic’s Claude Code documentation puts it in one line: “Claude Code is the harness; Claude is the model inside it.” OpenAI uses the same word for the shared execution core behind every Codex surface (CLI, IDE extension, web), all of them powered by the same Codex harness. When both labs independently settle on a term, it’s worth pinning down what it means.

One thing it doesn’t mean: Harness.io, the CI/CD and software delivery company. Same word, coincidental collision, unrelated domain. If you searched “harness devops” and landed here expecting deployment pipelines, that’s them. This post is about the LLM concept — though if you build deployment pipelines for a living, stick around, because you already have most of the mental model.

Model, Scaffold, Harness, Agent

These four words get used interchangeably, and they shouldn’t be. A Hugging Face glossary essay from May 2026, itself a sign the terminology had gotten muddy enough to need one, draws the lines this way:

• Model: the LLM. It can express intent to call a tool. It cannot execute anything.
• Scaffold: the behavioral configuration. System prompt, tool descriptions, how responses get parsed, what carries over between steps.
• Harness: the execution layer. It calls the model, runs the tool calls, feeds results back, and decides when to stop.
• Agent: model + harness. Something that acts, not just responds.

In casual usage “harness,” “scaffold,” and “framework” blur together, and for a blog post that’s usually fine. The distinction that actually matters is model versus everything-else, because the everything-else is where most of the engineering lives — and, as we’ll see, where a surprising amount of benchmark performance comes from.

Where the Term Came From

“Harness” is old software vocabulary. A test harness is code that sets up conditions, drives the thing under test, and scores the output. LLM research inherited that sense directly: EleutherAI’s lm-evaluation-harness, started in the GPT-3 era, became the standard way to benchmark models and the backend of Hugging Face’s Open LLM Leaderboard. In that world the harness was deliberately boring: standardized scaffolding, so that what you measured was the model, not an accident of prompt engineering.

Then models learned to use tools, and the word migrated. Once an LLM is taking actions in a real environment, the code that drives it stops being a measurement device and becomes a runtime. A datable marker of the shift: in September 2025, Anthropic renamed the Claude Code SDK to the Claude Agent SDK, on the reasoning that “the agent harness that powers Claude Code can power many other types of agents, too.” By early 2026 the term was everywhere. Mitchell Hashimoto’s February 2026 post crystallized “harness engineering” as a named practice, and OpenAI published an essay with that exact title the same month, describing a team that shipped a product with zero manually-written code. Their framing of the new engineering job: “design environments, specify intent, and build feedback loops.”

A fact-check aside, since I nearly repeated this myself: the term is often attributed to Andrej Karpathy. His widely-cited 2025 year in review doesn’t contain the word “harness” at all. The vocabulary came out of the labs and the eval community, not a single coinage.

What’s Actually Inside a Harness

Every serious harness is built around the same loop (gather context, take action, verify the result, repeat), but the implementations differ in revealing ways. A quick tour of the ones I use or have studied:

The loop and tools. Claude Code ships file access, shell execution, and search as first-class tools, plus subagents: child instances with their own context window and restricted tool access, so exploration doesn’t pollute the main conversation. Codex runs one shared Rust core under every product surface.

Context management. The context window is the scarcest resource, and each harness spends it differently. Aider builds a “repo map” — a graph-ranked summary of your codebase’s important symbols, compressed into a token budget, sent with every request. Claude Code compacts: when the window fills, older tool outputs get cleared and the conversation gets summarized. Cursor assembles context from a workspace index plus rules files that activate by file-glob.

Memory files. Nearly every harness converged on the same idea: a Markdown file in your repo that gets injected at session start. Claude Code reads CLAUDE.md; Codex, Cursor, and twenty-plus other tools read AGENTS.md, now an open format under the Linux Foundation. The harnesses differ; the convention is shared.

Permissions and guardrails. This is where harnesses look most like infrastructure. Claude Code layers permission rules (deny → ask → allow) over sandboxed shell execution. It’s IAM policy thinking applied to a model’s tool calls.

Hooks. Deterministic scripts that fire at lifecycle points: before a tool runs, after an edit, at session start. On my machine, a hook rewrites git and other CLI calls through a token-optimizing proxy before they execute, and another injects a reminder to persist session learnings into my local memory store. The model never decides whether those run. That determinism is the point — hooks are the part of the harness you control completely.

If that list reads like a platform engineering backlog (isolation, resource budgets, policy, lifecycle events, observability), that’s not an accident. I’ve argued before that harness engineering is a DevOps skill; this is the anatomy behind that claim.

More Harness Isn’t Better

Here’s the part that surprised me. Given how much engineering goes into these harnesses, you’d expect the elaborate ones to decisively beat simple scaffolds. The measured answer is: not reliably.

METR tested this directly in February 2026, running the same models under production harnesses and under deliberately simple scaffolds. Claude Code against bare-bones ReAct (an agent that just takes an action, sees the result, and repeats) was a statistical coin flip: Claude Code won in 50.7% of bootstrap samples. Codex against METR’s generic Triframe scaffold actually lost most of the time, winning only 14.5% of samples. And mini-swe-agent, a harness in roughly 100 lines of Python, scores above 74% on SWE-bench Verified — competitive with systems orders of magnitude more complex.

So the harness doesn’t matter? No, the opposite. Swapping scaffolds changes what the same model scores, which is exactly why METR controls for it when measuring capability. What the elaborate harness buys you just isn’t raw benchmark points. It’s everything a benchmark doesn’t measure:

• Safety: permission gates, sandboxes, and cost caps that make it survivable to let an agent run unattended. A 100-line loop with full shell access benchmarks fine right up until it doesn’t.
• Ergonomics: memory files, hooks, and skills that encode your project’s conventions, so you stop re-explaining them every session.
• Recoverability: compaction, session resumption, observable tool traces. The difference between an agent you can debug and one you re-run and hope.

One line from a Hacker News thread on harness engineering sums up the practitioner view: “A decent model with a great harness beats a great model with a bad harness.” The benchmark data says the sophistication isn’t free capability. The lived experience says it’s what makes the capability usable. Both are true, and the tension between them is basically the design brief for every harness team right now.

That brief keeps expanding. Anthropic’s latest iteration lets Claude generate its own orchestration harness per task — the harness stops being a fixed artifact a human designs once and becomes something the agent composes on the fly. Whether that’s the future or a detour, it tells you where the labs think the leverage is.

Why This Matters to You

If you’re choosing between coding agents, you’re mostly choosing between harnesses. The frontier models are closer to each other than the scaffolding around them is. Compare them on harness terms: how they manage context, what their permission model lets you safely automate, whether their memory and hooks let you encode your conventions once.

And if you build one — even a script that collects CI failure logs, asks a model what broke, and posts the answer to Slack — you’re doing harness engineering. The model is the part you rent. The harness is the part you own, and it’s where your effort compounds. For how to approach building them with the infrastructure skills you already have, see Harness Engineering: The DevOps Skill Nobody Told You About.

I built the application through Codex. I set the product direction, chose the constraints, reviewed the work, and decided what counted as done. Codex inspected the repository, planned implementation slices, wrote and revised the code, ran the test suites, diagnosed failures, captured screenshots, and prepared the release.

The result is a native desktop application with a React and TypeScript interface, a Rust backend, live local transcription, local LLM summaries, system-audio capture, meeting detection, search, playback, accessibility coverage, and a checksummed private release pipeline.

This is how the work happened, including the parts that broke.

What Minute does

Minute records a meeting, transcribes it with Whisper, and turns the transcript into a summary, decisions, and action items with a local language model. It can answer questions about a meeting and cite the exact timestamps that support its answer.

The complete workflow runs on the Mac. Minute has no account system, cloud backend, analytics service, or local HTTP server. Its only deliberate network request downloads a model when the user chooses one.

That privacy constraint shaped the product:

• Audio, transcripts, summaries, and Markdown stay in ordinary folders on disk.
• Whisper and llama.cpp run as linked libraries in the application process.
• Fonts ship with the app.
• The content security policy allows no network origin.
• A meeting remains recordable, searchable, and summarizable with Wi-Fi off.

Each note stays readable without Minute:

<note-id>/
├── audio.wav
├── transcript.json
├── summary.json
└── note.md

The architecture is compact:

React 19 + TypeScript
          │
    typed Tauri IPC
          │
       Rust process
     ├── microphone capture
     ├── system-audio capture
     ├── whisper.cpp
     ├── llama.cpp
     └── folder-based note store

Tauri gave me a webview for the interface and direct access to native Rust code. The backend uses cpal for microphone input, ScreenCaptureKit bindings for system audio, whisper.cpp for speech recognition, and llama.cpp for summaries and questions. Metal accelerates inference on Apple Silicon.

Keeping these pieces in one process removed a server from the architecture. It also put audio callbacks, native frameworks, model inference, file persistence, and a webview inside the same failure surface. That became the hard part of the build.

What “built with Codex” meant

I did not ask Codex for the whole application in one prompt.

Each unit of work had a concrete outcome. A typical request included the user problem, the relevant repository context, the constraints, and the evidence needed before completion. Codex then worked through the repository and returned with code, tests, and the result of running them.

A recording task, for example, included requirements such as:

• keep the audio callback free from blocking file or model work;
• write recoverable audio even when transcription falls behind;
• show the real microphone name reported by the backend;
• keep pause time out of the meeting duration;
• leave the note usable if summarization fails; and
• test the pure timing, resampling, buffering, and persistence logic.

That level of specificity mattered more than prompt length. “Add recording” invites a demo. “Preserve captured audio when the transcription worker fails” defines a product behavior.

The working loop stayed consistent:

1. Describe one user-visible outcome and its constraints.
2. Let Codex inspect the current implementation before proposing a change.
3. Implement the smallest coherent slice.
4. Run focused tests, then the wider verification suite.
5. Inspect the diff and exercise the behavior.
6. Capture screenshots or native evidence when tests cannot show the result.
7. Record the remaining gaps instead of hiding them.

This resembles OpenAI’s current Codex best-practices guidance, especially its emphasis on goals, context, constraints, completion criteria, and validation. In practice, the last item carried the most weight. Codex could move quickly because every task had to leave proof behind.

The bugs that made the workflow useful

The feature list shows what Codex produced. The failures show whether the workflow could handle real engineering.

AppKit crashed on the detector thread

Minute can notice that Zoom, Teams, FaceTime, or another meeting app is using the microphone and offer a small recording prompt. The detector runs on a background Rust thread so it does not block the interface.

Version 0.5.0 crashed when that thread tried to show the prompt. The macOS crash report ended inside NSPanel setFloatingPanel, called through tauri-nspanel, with Minute’s detector thread lower in the stack.

Codex traced the call path from the crash report into my code and then into the pinned dependency source. Converting the Tauri window into a panel already mutated AppKit state, so moving only the final show call would have left the bug in place.

The fix routed panel creation, conversion, configuration, positioning, showing, and hiding through Tauri’s main-thread dispatcher. Debug assertions using MainThreadMarker now guard each native boundary. The detector still does its polling away from the interface thread.

The useful output was the explanation of the complete native boundary. A smaller patch could have stopped one crash while leaving the same threading mistake elsewhere.

A microphone stream could succeed and still record silence

My early preflight asked CoreAudio for an input stream. On macOS, that stream can open while microphone permission is unresolved or denied and then deliver silence. Minute could claim it was ready even though it had no usable audio.

I found this by launching an isolated app identity with a fresh permission state. Codex added an explicit AVFoundation authorization check, a visible “Allow microphone” action, and backend gates around preview and recording. When permission is denied, the preflight stays open and points to the exact System Settings location. It does not expose an input meter or pretend recording has started.

The frontend component tests passed, but the decisive evidence came from the native bundle with a reset permission state. That distinction became a rule for the rest of the project. Simulate logic in tests and verify operating-system behavior on the operating system.

The Intel app contained Apple Silicon libraries

The first Intel build had an x86_64 executable and arm64 ggml and llama dynamic libraries. Code signing passed. The application architecture was still wrong.

The staging script always copied libraries from one local target directory, even during a cross-architecture build. Codex updated it to respect Tauri’s target triple and added lipo checks for the executable and every nested library. The release workflow now fails when a single binary does not match the intended architecture.

This bug changed my definition of a valid macOS bundle. A signature checks integrity and identity. It does not prove that every binary can run on the recipient’s processor.

Product taste still required a person

The first complete interface worked, but it looked like a competent collection of panels. I wanted Minute to feel closer to a notebook: warm paper surfaces, ink-like text, thin rules, and one oxide accent reserved for recording.

I supplied that direction and judged the screenshots. Codex translated it into design tokens, rebuilt the major views, captured the application in light and dark appearances, and corrected the details exposed by each pass. I repeated the process at the minimum window width, at 200 percent text size, and with reduced motion enabled.

One review found that the recording details rail clipped at the native minimum width. The fix allowed the secondary rail to recede below 1280 pixels while the capture source remained visible. Another pass found that VoiceOver announced recording health every second. The visible status stayed live, while the assistive announcement changed only when the health category changed.

Codex could measure overflow, inspect the accessibility tree, compare screenshots, and implement the correction. I still had to decide whether the screen felt calm, which information deserved priority, and whether a tradeoff fit the product.

Verification became part of the product

Fast code generation creates a review problem. More code arrives before a person can inspect every path with equal care. Minute handled that by turning acceptance criteria into executable checks wherever possible.

On July 28, a fresh npm run verify completed with:

• 676 passing frontend tests across 32 test files;
• 384 passing Rust tests, with 10 hardware, model, and network tests explicitly ignored;
• a clean lint run;
• a successful TypeScript and Vite production build; and
• valid release metadata for Minute 0.7.0.

The broader release work added a three-hour logical recording soak, a large-library performance scenario, automated axe checks, screenshot diffs, keyboard-only review, and a native VoiceOver pass. I tested light and dark appearances, minimum width, enlarged text, and reduced motion.

Failure paths received their own acceptance criteria. A simulated disk-write failure must preserve samples already written, keep transcription forwarding alive, and mark the final note as needing review. A failed summary must leave the audio and transcript usable. Deleting a note moves it into local recovery and exposes Undo.

These checks influenced the design. Once a recovery behavior had to be asserted, vague states such as “something went wrong” stopped being acceptable. The app had to say what survived and what the user could do next.

The division of work

Codex handled a large share of execution:

• repository inspection and implementation planning;
• React, TypeScript, Rust, and macOS framework code;
• unit, component, accessibility, soak, and regression tests;
• dependency and crash-source investigation;
• repeated build, lint, and test runs;
• browser and native visual inspection;
• release scripts, CI workflows, and documentation; and
• small commits that kept each change reviewable.

My work centered on judgment:

• choosing an offline product and keeping that promise narrow;
• deciding what Minute should show before, during, and after a recording;
• rejecting placeholder actions and claims the app could not prove;
• setting the visual direction;
• choosing which failures required device evidence;
• accepting tradeoffs around model size, latency, and memory; and
• deciding that an ad-hoc private beta was honest while a public release was premature.

Agentic development increased the number of decisions I could test in working software while leaving their ownership with me.

Where the approach worked

Codex performed best when the repository could answer whether a change was correct. Rust’s type system caught unsafe boundaries. Tests pinned timing and persistence behavior. Screenshots exposed layout regressions. Native crash reports and accessibility trees gave the agent evidence beyond source code.

Small commits also helped. The repository reached 124 commits in five days, but the history remains readable because most commits contain one feature, fix, or verification step. When a later failure appeared, Codex could trace the relevant decision without reconstructing one enormous change.

The workflow was especially effective for tasks that crossed layers. Adding a marker touched keyboard handling, recording state, Tauri commands, Rust persistence, Markdown generation, post-recording editing, Undo behavior, and tests. Keeping one agent responsible for the complete outcome reduced the handoff errors I would expect from splitting those layers into isolated coding tasks.

Where it struggled

Native desktop work still depends on physical state. A test cannot unplug a microphone that I do not have. A controller running on the Mac cannot put its own machine to sleep and continue observing the result. An architecture check cannot prove that an Intel user can complete a real transcription.

Long sessions also accumulated assumptions. Starting a fresh task for each coherent outcome, preserving decisions in repository documents, and asking Codex to inspect the current code reduced that drift.

Visual work needed several passes. Codex could reproduce a reference and measure the output, but phrases such as “make it feel native” produced weak results. Concrete direction about hierarchy, spacing, material, color, and state produced much better work.

The agent also needed explicit permission to stop. Without that, it could keep polishing a passing implementation while a higher-risk release gap remained. A written backlog made unfinished work visible and gave me a place to draw the line.

What I would repeat

I would use the same approach for another product, with a few rules in place from the first commit:

1. Write the product constraints before choosing the architecture.
2. Give each task a user-visible outcome and a completion check.
3. Ask for tests and implementation in the same task.
4. Use screenshots, logs, crash reports, and native inspection as agent input.
5. Test failure recovery before adding secondary features.
6. Keep commits small enough to explain in one sentence.
7. Maintain an honest list of work that automation cannot verify.
8. Reserve release decisions and product taste for the human directing the work.

Minute’s current release reflects those limits. The Apple Silicon and Intel private builds are ad-hoc signed, checksummed, and architecture-verified. They are not notarized because I do not yet have the required Apple Developer credentials. Physical Intel launch and transcription, removable-microphone recovery, sleep and wake, an overnight recording, clean-Mac installation, and the signed updater remain open release checks.

You can inspect the Minute source and the 0.7.0 private beta. The repository includes the test suites, visual baselines, reliability matrix, accessibility review, release checklist, and commit history behind the claims in this post.

A new task appears in PENDING. It changes to RUNNING, but the old task does not go away. A minute later the old one starts draining. Eventually it disappears and the service returns to steady state. If you only watch the task count, the whole thing looks oddly slow and a little random.

It is neither. Four systems are working on the deployment at the same time:

• The ECS scheduler is enforcing minimum and maximum task counts.
• The ECS agent or Fargate runtime is provisioning the task and starting its containers.
• The load balancer is deciding whether the new target should receive traffic.
• The old container is trying to finish requests before ECS kills it.

Most of the confusing failures I see come from treating those four systems as one thing. A task can be running but not ready. It can be healthy in ECS but unhealthy in the target group. It can be removed from the load balancer and still spend another minute shutting down.

We are going to follow one deployment from the API call to the final stopped task. At each pause, we will look at which system owns the wait and what condition it needs before the rollout can continue.

The example service

We will use a small Fargate service behind an Application Load Balancer:

service: api
desired count: 4
current task definition: api:41
new task definition: api:42
minimumHealthyPercent: 100
maximumPercent: 200
healthCheckGracePeriodSeconds: 60

The service begins with four healthy tasks running revision api:41. Our pipeline registers api:42, then calls UpdateService to point the service at the new revision.

The deployment settings create a range:

minimum healthy = ceil(4 × 100%) = 4
maximum running or pending = floor(4 × 200%) = 8

ECS must keep at least four healthy tasks available, but it may temporarily run as many as eight. With enough Fargate or EC2 capacity, the scheduler can start all four replacements before touching the old tasks.

When a rollout behaves strangely, I calculate this range before looking anywhere else.

The service revision changes first

Registering a task definition does not deploy it. It only creates an immutable blueprint such as api:42.

Updating the service is what starts the deployment:

aws ecs update-service \
  --cluster production \
  --service api \
  --task-definition api:42

ECS records the configuration it is leaving and the configuration it is trying to reach. In the current ECS deployment model, these are the source and target service revisions. A service revision includes more than the task definition: it records the workload configuration ECS is attempting to deploy and gives rollback a known previous state.

No customer traffic has moved yet. ECS has only changed desired state:

source: api:41, four healthy tasks
target: api:42, zero healthy tasks
deployment: IN_PROGRESS

The service scheduler now begins reconciling reality with the new declaration.

The scheduler finds room for the new tasks

The scheduler does not blindly replace one task at a time. It works inside the range created by minimumHealthyPercent and maximumPercent.

minimumHealthyPercent is the availability floor. ECS rounds it up.

maximumPercent is the concurrency ceiling for tasks in RUNNING or PENDING. ECS rounds it down.

The same four-task service behaves quite differently as those settings change:

These percentages affect availability, but on ECS with EC2 they also decide how much spare capacity a deployment needs. A service configured for 100/200 may run the old and new revisions together. If the cluster is already full, those new tasks have nowhere to go. Fargate hides the hosts, but account quotas, IP availability, and platform capacity can still block placement.

There is also a rounding trap. With a desired count of three and maximumPercent: 125, the upper limit is:

floor(3 × 1.25) = 3

ECS cannot start a fourth task. If minimumHealthyPercent also prevents it from stopping an old task, the deployment has no legal move. ECS emits a service event telling you that the deployment configuration cannot start or stop a task.

The first task pins the image

Task definition api:42 may say:

{
  "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/api:production"
}

That tag is mutable. It can point to a different image tomorrow.

By default, ECS resolves image tags to digests during a deployment so that every task in the service runs identical image content. For a service with multiple tasks, the first new task is used to establish the image digest, and the remaining tasks use that digest.

Overwriting a tag does not change a running task. ECS needs a new deployment before it resolves and launches the new image. Unique tags, preferably the Git SHA, also save a lot of guesswork during an incident.

api:production                         mutable pointer
api@sha256:8d6c...                     actual image content

The task then moves through the early lifecycle:

PROVISIONING → PENDING → ACTIVATING → RUNNING

Depending on the launch type and configuration, this includes finding capacity, attaching an ENI, pulling images, creating containers, configuring networking, registering targets, and starting essential containers.

RUNNING is one of the most misleading words in the ECS console. It means the containers are running. It does not mean the application is ready for customer traffic, and ECS may not count the task as healthy yet.

RUNNING is followed by another wait

A task can be evaluated by two separate health systems:

1. A container health check defined in the ECS task definition.
2. A load-balancer target group health check.

If an essential container has an ECS health check and the service uses a load balancer, both must pass before the scheduler counts the task as healthy for the deployment.

A container health check runs inside the container:

{
  "healthCheck": {
    "command": [
      "CMD-SHELL",
      "curl -f http://localhost:8080/health || exit 1"
    ],
    "interval": 10,
    "timeout": 5,
    "retries": 3,
    "startPeriod": 30
  }
}

The startPeriod belongs to the container health check. Failed checks during that bootstrap window do not count toward the retry limit. If a check succeeds during the start period, the container becomes healthy and later failures count normally.

healthCheckGracePeriodSeconds is different. It belongs to the ECS service and tells the service scheduler to ignore unhealthy container, load-balancer, or VPC Lattice health status for a period after each task starts.

The two timers sound interchangeable, but they sit at different layers:

container startPeriod
  protects the container health-check retry counter during startup

service health-check grace period
  prevents the ECS scheduler from replacing a new task during startup

Neither setting sends traffic to an unhealthy target. The Application Load Balancer still follows its own target health state. The grace period only changes how the ECS scheduler reacts to an unhealthy result.

For a newly registered ALB target, one successful health check is enough to mark it healthy. The target group’s healthy-threshold count applies when a previously unhealthy target is recovering. With the default 30-second health-check interval, even a healthy application may spend noticeable time waiting for the next probe.

Before the first api:42 task counts toward the rollout, all of this must be true:

ECS task state: RUNNING
container health: HEALTHY
ALB target health: healthy
counts toward deployment minimum: yes
receiving customer traffic: yes

Now the new revision has proved that it can serve traffic. Until this point, stopping an old task would spend availability on a replacement that had not earned it.

New tasks come in, old tasks drain

As new tasks become healthy, the scheduler gains room to remove old ones without crossing the availability floor.

With our 100/200 settings, the rollout may look like this:

time     api:41 healthy     api:42 starting/healthy     total
t0              4                     0                   4
t1              4                     4 starting          8
t2              4                     4 healthy           8
t3              0                     4 healthy           4

Do not read that table as a promise that all four tasks move together. The scheduler chooses the batches. Placement capacity, startup time, health results, and throttling can turn the same configuration into a more incremental rollout.

When ECS decides to stop an old task behind an ALB, the task does not jump directly from RUNNING to STOPPED.

It moves through the shutdown side of the lifecycle:

RUNNING → DEACTIVATING → STOPPING → DEPROVISIONING → STOPPED

During DEACTIVATING, ECS deregisters the task from the target group. The target enters draining, and the load balancer stops assigning it new requests while allowing existing connections time to complete according to the target group’s deregistration delay.

Then ECS stops the containers. On Linux, the container receives the signal defined by its image STOPSIGNAL, which is SIGTERM by default. ECS waits for the container’s stopTimeout. If the process is still alive after that window, it receives SIGKILL.

For graceful shutdown, all three layers need to agree:

ALB deregistration delay
    ≥ longest request or connection you intend to preserve

ECS stopTimeout
    ≥ time the application needs after SIGTERM

application shutdown handler
    stops accepting work and exits before stopTimeout

If the application ignores SIGTERM, no ECS setting can make its shutdown graceful. ECS will eventually kill it.

Long-lived WebSocket connections, streaming responses, background jobs, and queue consumers require special attention. A web server can stop accepting new requests and finish active ones. A worker may need to stop polling, return an in-flight message to the queue, or extend its visibility timeout. Rolling deployment safety is partly an application concern.

ECS reaches steady state

The deployment completes after the target revision reaches the desired count and the old revision no longer has active tasks in the rollout.

For our service:

api:41: 0 running tasks
api:42: 4 running and healthy tasks
desired count: 4
deployment: SUCCESSFUL

Your CI system does not perform the rollout. A deployment action usually registers the task definition, updates the service and, if configured, polls ECS until the service stabilizes. That is what the official GitHub action does when wait-for-service-stability is set to true. The scheduler still runs the deployment.

So when a pipeline appears frozen, the useful evidence is often somewhere else. ECS may be waiting for a health check, placement capacity, a draining connection, or the circuit-breaker threshold.

Start with ECS service events and deployment details, not the CI runner logs.

Useful commands include:

aws ecs describe-services \
  --cluster production \
  --services api

aws ecs list-service-deployments \
  --cluster production \
  --service api

aws ecs describe-service-deployments \
  --service-deployment-arns SERVICE_DEPLOYMENT_ARN

aws ecs list-tasks \
  --cluster production \
  --service-name api

Service deployment history includes the source and target revisions, deployment state, failed-task count, alarm state, timestamps, and rollback details. ECS retains recent deployment history, which is much more useful than reconstructing a rollout from a few console event messages.

What happens when the new tasks fail?

Failed deployments usually split into two groups. The difference tells you where to start looking.

The task never reaches RUNNING

The task may have no capacity, fail to pull its image, or start with an execution role that cannot retrieve a secret. ENI attachment failures also happen here. So does an essential container that exits during startup.

These are launch failures. In its first stage, the deployment circuit breaker counts consecutive tasks that fail to reach RUNNING.

The task runs but never becomes healthy

Sometimes the process starts but listens on the wrong port, or the health endpoint returns a failing status. The ALB security group may not reach the task. A slow application can run past its grace period. I have also seen health checks fail because the image did not contain curl, even though the application itself was fine.

These are health failures. After at least one new task reaches RUNNING, the circuit breaker moves to its second stage and watches container, load-balancer, and service-discovery health.

The circuit breaker threshold is based on half the desired count, bounded to a minimum of 3 and a maximum of 200:

threshold = ceil(0.5 × desired count), bounded to [3, 200]

For a service with a desired count of one, the threshold is still three. A tiny service may launch the same broken task several times before ECS finally calls the deployment failed.

Enable rollback explicitly:

{
  "deploymentCircuitBreaker": {
    "enable": true,
    "rollback": true
  },
  "minimumHealthyPercent": 100,
  "maximumPercent": 200
}

When the deployment fails, ECS can roll back to the last service revision that completed successfully. CloudWatch alarms cover a different failure mode: the tasks are technically healthy, but latency or error rate gets worse after the release.

I use both because they answer different questions:

circuit breaker
  catches tasks that cannot launch or become healthy

CloudWatch deployment alarms
  catch applications that are running but behaving badly

A /health endpoint returning 200 cannot tell you that checkout latency tripled or every database write is failing. The scheduler only knows what you expose to it.

Why healthy deployments still cause errors

ECS can execute a perfect rolling deployment and users can still see failures.

The new and old versions are incompatible

During a rolling deployment, both versions receive traffic. A destructive database migration, incompatible queue payload, or changed cache format can break one revision while the other is still alive.

Database changes should generally follow expand-and-contract:

1. Add the new schema while keeping the old schema valid.
2. Deploy code that can work with both representations.
3. Migrate data.
4. Remove the old schema in a later deployment.

Readiness is too shallow

Returning 200 because the HTTP process started is not enough if the application cannot reach a dependency required for serving requests. But checking every downstream system can also cause a cascading failure by removing all targets during a shared dependency outage.

The useful question is narrower: what condition means this particular task should stop receiving traffic? Write the health check around that contract.

Shutdown is not graceful

If the process exits immediately on SIGTERM, active requests die even though the ALB is draining the target. If it waits longer than stopTimeout, ECS kills it anyway.

Sticky sessions or local state hide the overlap

Rolling deployments assume tasks are replaceable. Sessions stored only in process memory, local uploads, or singleton background work make replacement unsafe. The scheduler cannot protect state it does not know exists.

The deployment settings I start with

For a normal stateless HTTP service on Fargate behind an ALB, I start here:

desired count: at least 2
minimumHealthyPercent: 100
maximumPercent: 200
deployment circuit breaker: enabled with rollback
CloudWatch alarms: 5xx rate and latency, with rollback
health-check grace period: measured startup time plus margin
image tag: immutable Git SHA
container stopTimeout: matched to graceful shutdown behavior

I keep the ALB health check cheap and local. I test SIGTERM handling outside production. I also alarm on SERVICE_DEPLOYMENT_FAILED through EventBridge because an automatic rollback that nobody notices is still a failed release.

If doubling task count is too expensive or the EC2 cluster cannot hold it, lower maximumPercent carefully and verify that the scheduler still has room to make progress. Saving temporary capacity is not useful if it creates a deployment deadlock or reduces availability below what the application can tolerate.

A practical debugging order

When a deployment is slow or stuck, inspect it in this order:

1. Deployment state: What are the source and target service revisions?
2. Scheduler math: Do the minimum and maximum percentages allow a task to start or stop?
3. Service events: Is placement, capacity, IAM, image pulling, or networking failing?
4. Task lifecycle: Are new tasks stuck in PENDING, stopping before RUNNING, or running and then replaced?
5. Stopped-task reason: What did ECS report for the failed task and essential container?
6. Target health: Is the ALB target initial, unhealthy, healthy, or draining? What reason does the target group report?
7. Application logs: Did the process bind the expected port and finish startup?
8. Timers: Are the container start period, service grace period, ALB interval, deregistration delay, and stop timeout consistent?
9. Rollback controls: Is the circuit breaker enabled, and are application alarms attached?

The order matters. There is no point staring at application logs if ECS never started the container. Likewise, changing a health endpoint will not fix a target group that cannot reach the task through its security group.

What to remember

An ECS rolling deployment is a constrained reconciliation loop:

declare a new service revision
        ↓
calculate the legal task-count range
        ↓
start tasks from the new revision
        ↓
prove they can run and receive traffic
        ↓
drain traffic from the old tasks
        ↓
give old processes time to exit
        ↓
stop the old revision and reach steady state

The scheduler protects task counts. Health checks decide which tasks receive traffic. Draining gives active requests time to finish. The circuit breaker catches a rollout that cannot converge. CloudWatch alarms catch the more annoying case, where the rollout converges and the application still gets worse.

When the ECS console sits on Deployment in progress, I check three things: the task state, the target state, and whichever timer is active. So far, the pause has always been hiding in one of them.

Sources and further reading

• Deploy Amazon ECS services by replacing tasks
• Amazon ECS task lifecycle
• How the ECS deployment circuit breaker detects failures
• Amazon ECS service deployment history
• Amazon ECS service revisions
• Optimize load-balancer health checks for ECS
• Determine task health with container health checks
• Automate rollback with CloudWatch alarms

I’ve written before about the ECS decisions that waste six weeks. This post is the prequel: what ECS actually is, how it maps onto the Kubernetes concepts you already know, and what you stop carrying on your pager when you choose it. There are a few interactive diagrams below. Click around in them; they teach the model faster than prose does.

One thing before we start: this is not a “Kubernetes bad” post. EKS is the right choice for some teams, and I’ll tell you exactly which ones at the end. But I’ve watched too many three-person teams default to EKS because it felt like the serious choice, without anyone explaining what they were signing up to operate.

ECS is an orchestrator. That’s it.

Strip away the branding and every container orchestrator does the same job: you declare what should be running, and a control loop makes reality match the declaration. Kubernetes does this. Nomad does this. ECS does this.

ECS just exposes far fewer moving parts to you. Here’s the whole object model. Click each piece:

If you know Kubernetes, the translation table is short enough to memorize over coffee:

That last column is where the story actually lives. In Kubernetes, “where compute comes from” and “how pods get IPs” and “how workloads get cloud credentials” are all decisions with an ecosystem of competing answers. In ECS they’re defaults. You don’t pick a CNI. You don’t install an IRSA webhook. There’s one way, it’s boring, and it works.

The reconciliation loop — same idea, fewer layers

The core idea both systems share: you declare desired state, a control loop enforces it. This is the part I find people understand instantly once they watch it instead of reading about it.

Below is an ECS service with desired count: 4. Click a task to kill it, then watch the scheduler notice and replace it. Then hit deploy and watch a rolling deployment do exactly what a Kubernetes Deployment rollout does: bring up new tasks, drain old ones, never drop below healthy.

That’s a Deployment rollout and a ReplicaSet self-heal, except nobody installed anything to get it. There’s no controller manager to version. You get all of this the moment you create a service.

When I help teams ship on ECS, this is where it clicks: you already understand ECS. If you can reason about desired state and reconciliation, the orchestration knowledge transfers completely. What doesn’t transfer is the operational surface area, and that’s the actual argument.

What you stop operating

This is the comparison that matters for a small team, and it’s the one nobody draws. The question isn’t which scheduler is smarter. They’re both fine. The question is whose pager each layer lands on.

Toggle ECS between Fargate and EC2 to see the middle ground:

Look at the EKS column. Six of the seven layers are yours. None of them are your product.

The upgrade treadmill deserves special attention because it’s the one that quietly eats small teams. Kubernetes ships about three releases a year, and EKS standard support for each lands around 14 months. That means a recurring, unskippable project roughly once a year, forever: test the control plane upgrade, upgrade the add-ons in the right order, chase whatever deprecated APIs your manifests use, then roll the nodes. Skip it and AWS moves you to extended support at six times the control plane price. For a platform team of 15, that’s Tuesday. For a team of four, it’s a sprint per year spent running to stand still. And there’s a quieter cost on top: you have to stay the kind of team that can do this safely.

ECS doesn’t have a version. I want to make sure that lands. There is no upgrade, no deprecation cycle, no “v1.29 removes the API your ALB controller depends on.” The control plane changed under you a hundred times last year and you never noticed. I have ECS services from 2021 that have never needed a maintenance commit. Infrastructure that doesn’t generate homework is worth more to a small team than anything on the Kubernetes feature list. It’s the same reason I tell teams to pick boring options everywhere else in the stack: boring means you debug your app, not your platform.

On raw cost, the EKS control plane is about $73 a month per cluster and ECS’s is free, and that’s the least interesting line in the comparison. Run the numbers on engineering time instead. One sprint of one engineer’s time per year on cluster maintenance is $10-20k. The biggest AWS savings I’ve ever found came from deleting complexity, not from rightsizing it.

What you give up

If this were one-sided, EKS wouldn’t exist. Here’s what you actually lose.

The big one is the operator ecosystem. Kubernetes has operators for Postgres, Kafka, cert-manager, external-dns, ArgoCD, all debugged by thousands of teams over a decade. ECS has no CRDs and no operator pattern. The AWS answer is “use the managed service”: RDS instead of a Postgres operator, MSK instead of Strimzi. That works right up until you need something AWS doesn’t sell.

Tooling in general follows the same line. Vendors ship a Helm chart, not a task definition. Kustomize, the CNCF landscape, none of it targets ECS. And your deployment layer is AWS-native, so a future move off AWS means rewriting it. Your containers move unchanged, but the wiring around them doesn’t.

There’s also the hiring thing, and I won’t pretend it isn’t real. Engineers want Kubernetes on their CV. ECS knowledge is real orchestration knowledge and the concepts transfer completely, as the diagrams above show, but nobody’s career was ever advanced by the phrase “task definition.”

And ECS has a control ceiling. Custom schedulers, topology spread, network policy, the more exotic probe and init semantics: Kubernetes gives you knobs ECS simply doesn’t have. Most web products never touch them. If yours genuinely does, you’ll feel the ceiling and you’ll resent it.

So when is EKS the right call?

EKS earns its keep when at least one of these is true:

• Someone owns the platform. You have, or are hiring, people whose actual job is cluster operations, so the pager layers above land on a team that exists.
• You’re running stateful infrastructure on-cluster that AWS doesn’t offer as a managed service, and you need the operator ecosystem for it.
• Multi-cloud or on-prem is a real requirement: contractual, regulatory, or your customers deploy your software into their clusters.
• Your team is already fluent. K8s veterans ship faster on EKS than they would learning anything else. The tax is only a tax if you haven’t already paid it.

If none of those describe you, and for most sub-ten-engineer teams shipping a web product none do, then Kubernetes isn’t buying you capability. It’s buying you a second job.

The takeaway

ECS is not “Kubernetes for beginners.” It’s the same control loop idea with a deliberately smaller operational surface. Same desired state, same reconciliation, same rolling deploys, minus the version treadmill, the add-on stack, and the node fleet. You’ve seen the whole object model in this post. There is no part two where the hidden complexity lives.

Small teams don’t lose because they picked the wrong orchestrator. They lose because their best engineers spent the year operating infrastructure the product didn’t need. Pick the tool that generates the least homework, ship, and revisit when you have the head count to afford opinions.

If you’re starting an ECS build-out, the companion post on the 5 ECS decisions that waste 6 weeks covers the concrete choices: Fargate vs EC2, service discovery, CI/CD, secrets, and monitoring.

If this post saved you a meeting, it did its job. I write about AWS, DevOps, and building things from scratch. Subscribe via RSS, or find me on Twitter.

They are both right, which is the problem. The number on the screen is real and it is also not the number you actually wait for. And the format you should pick was never really about that number anyway.

I have gone through this decision enough times now, on my own machine and for clients standing up local inference, that I want to write down the part nobody puts in the comparison tables: GGUF versus MLX is a five-question decision, and only one of those questions is about speed.

What you are actually choosing between

GGUF is the file format from the llama.cpp project. One file holds the quantized weights, the tokenizer, the chat template, and the metadata, and any runtime that can load it will run the model. That includes llama.cpp itself, Ollama, LM Studio, KoboldCPP, and a handful of others. It runs on basically everything: CPU, NVIDIA, AMD, Apple Metal, even a Raspberry Pi if you are patient. Portability is the whole point of the format.

MLX is not a file format. It is Apple’s array framework, the rough equivalent of PyTorch built specifically for Apple Silicon. An MLX model is a directory of safetensors files plus a config that the runtime reads directly. You convert and quantize a model in one command with mlx_lm.convert. The catch is in the name: MLX runs on Apple Silicon and nowhere else.

One thing worth clearing up before we go further, because it shows up in half the comparisons and it is out of date: people say GGUF does clever mixed-precision quantization while MLX is stuck on flat uniform 4-bit. The first half is true. The second half is not. Apple walked through per-layer mixed precision in their WWDC25 session on running large language models with MLX, including the trick of keeping the embedding and output layers at 6-bit while the rest of the model sits at 4-bit. MLX can do it. It is just that most of the MLX builds floating around Hugging Face do not bother, so in practice you often are comparing GGUF’s mixed precision against a uniform MLX quant. Worth knowing when you read someone else’s quality benchmark.

The number on the screen is lying to you

Quick detour, because it poisons most of the benchmarks you will find.

The tokens-per-second figure your runtime prints while text is streaming measures decode speed, the rate at which the model emits new tokens. It does not include prefill, the time the model spends reading your prompt before it says anything. For a chatty exchange with a short prompt that does not matter much. For an agent that stuffs tool output, a chunk of a file, and a system prompt into every turn, prefill is most of what you wait for, and the streaming counter never sees it.

There is a benchmark writeup that made the rounds on r/LocalLLaMA where the author’s UI proudly reported nearly twice the tokens per second on MLX as on GGUF, and then the actual wall-clock time had GGUF finishing first on most of the real tasks. Same machine, same model. The counter was not wrong. It was just answering a different question than the one that mattered.

Keep that in your head for the whole rest of this post. When I say one format is “faster” below, I mean wall-clock on a real workload, not the number that scrolls past while tokens stream.

Five questions that actually decide it

1. How big is the model relative to your RAM?

This is the question that quietly settles a lot of arguments. Token generation is bounded by memory bandwidth, not compute. To emit one token the GPU has to read the entire model out of memory. On an M4 Pro with roughly 273 GB/s of bandwidth, a 4-bit 27B model weighing about 17 GB caps out near 16 tokens per second no matter what software you run. MLX cannot fetch bytes faster than the hardware allows, and neither can llama.cpp.

So for large models, the ones that fill most of your unified memory, the format barely matters for speed. They both hit the same wall. The interesting differences show up on smaller models, under roughly 8 to 14B, where the model fits comfortably and the bottleneck shifts from bandwidth to framework overhead. That is where MLX’s tighter, Apple-specific kernels pull ahead, often in the 15 to 40 percent range on single-user decode, and wider still on very small models that lean hardest on framework efficiency.

Small model, want it snappy: MLX has something real to offer. Big model that barely fits: pick on the other four questions, because speed is a wash.

2. Will this ever need to run somewhere other than a Mac?

If there is any chance the same artifact has to run on a Linux box, a cloud GPU, or a teammate’s non-Apple machine, you want GGUF. The same file moves between all of them. MLX does not leave Apple Silicon, full stop. If you ship MLX as your only build and then need a CUDA fallback, you are re-quantizing under pressure.

This one overrides almost everything else. Portability is not a performance feature, but it is the feature you miss most when it is gone.

3. What does your workload actually look like?

Not “what model,” but the shape of the traffic. Specifically the ratio of input to output.

Workloads that feed the model a lot and ask for a little (classification, tool-calling agents with short replies, RAG with a big injected context) lean toward GGUF. llama.cpp has more battle-tested prompt caching and FlashAttention, and MLX’s prefix caching has historically been the less reliable of the two, especially on newer hybrid-attention models. When prefill dominates the wall clock, that maturity wins.

Workloads that take a short prompt and generate a lot (summaries, long-form chat, brainstorming) lean toward MLX. Once the model is past prefill and just streaming tokens, MLX’s decode advantage compounds, and the longer the reply the more it pays off.

There is a crossover point that depends on both context size and reply length. With a small prompt, MLX needs a couple hundred output tokens before its faster decode makes up for slower prefill. With a few thousand tokens of context, it needs several hundred more. If your agent’s replies are 150 tokens and its context keeps growing, you are living on the wrong side of that crossover, and GGUF is the better call.

4. Do you want to train, or just run?

GGUF is an inference format. You download it, you run it, that is the relationship. If you want to fine-tune, you convert back to safetensors, find a GPU, do the work, and convert forward again.

MLX is a full framework. You can fine-tune with LoRA or QLoRA directly on the Mac, merge adapters, and run speculative decoding with a small draft model, all natively. If part of your reason for going local is to actually adapt models and not just serve them, MLX is the only serious option on Apple Silicon, and this question alone can decide the whole thing.

5. How much do you care about ecosystem and exact fit?

Two practical edges for GGUF here. First, coverage: every open model gets GGUF builds within hours of release, including the obscure ones. MLX coverage is good for popular models and lags for everything else. Second, granularity. GGUF gives you a long ladder of quant levels, Q4_K_M, Q5_K_M, Q6_K, the I-quants, and so on, so when you have exactly 16 GB to work with you can usually find a quant that fits. MLX builds are mostly published at 4-bit and 8-bit, so you sometimes get a 4-bit that is a hair too small for the quality you want and an 8-bit that will not fit.

The edge on MLX’s side: it tends to get support for new Apple hardware features first, because Apple ships the metal abstraction in MLX before llama.cpp catches up.

The flowchart

Put the five questions in order and most decisions fall out in about ten seconds.

• Need to run on anything other than Apple Silicon, now or later? → GGUF. Stop here, portability wins.
• Staying on Apple Silicon. Do you want to fine-tune or train on-device? → MLX.
• Inference only. Is your workload short-output and prefill-heavy (agents, RAG, classification)? → GGUF.
• Long outputs, interactive, single user, latency you can feel? → MLX.
• Need a precise quant to fit tight RAM, or running a just-released or obscure model? → GGUF.
• Still undecided? → GGUF. It is the conservative default. Ship it, and A/B an MLX build later if throughput becomes the constraint.

The short version: GGUF is what you pick when you are not sure, because it is the one that is hard to regret. MLX is what you pick when you own the hardware, run single-user, and have a specific reason, throughput on long outputs or on-device training, to want it.

Once you have picked, pick a quant level

The format is half the decision. The bit width is the other half, and the defaults are good but not always right.

Start at Q4_K_M for GGUF or 4-bit for MLX. Q4_K_M is the community default for a reason. It keeps most tensors at 4-bit, then bumps the quality-sensitive ones to 6-bit: the attention value weights and the feed-forward down-projection, on a portion of the layers. That holds quality better than a flat 4-bit quant at a small size cost. The reported quality loss against FP16 on MMLU is model-dependent but small: well under a point on a big model, creeping up toward a point or so on something under 8B, and a little more again for a uniform 4-bit MLX build. On a 30B-plus model that gap is noise. On something under 8B, especially on coding tasks where attention precision matters, it is visible, and you have two outs: stay on GGUF Q4_K_M, or move to MLX 6-bit, which closes the gap for roughly a 30 percent larger file.

If RAM is genuinely tight, GGUF’s I-quants with an importance matrix are the quality-per-byte champions at low bit widths. The cost is slower decode on CPU, so they make more sense when you are squeezing a model onto limited memory than when you are chasing speed.

One rule regardless of format: do not drop below roughly 3-bit without measuring quality on your own task. The aggregate benchmarks stop predicting what you will actually see down there.

Two traps that will flip your results

The bf16 trap on M1 and M2. A lot of MLX builds ship as bf16, and on the M1 and M2 that data type does not get the accelerated path that fp16 does. During prefill those weights run un-accelerated and the penalty multiplies across every input token, which is part of why some “MLX is slow” reports come from older hardware. The fix is a one-minute reconvert with --dtype float16. If you are on an M1 or M2 and MLX feels sluggish, check this before you blame the format.

Caching is the real variable. The biggest swings I have seen between runtimes were not about GGUF versus MLX at all, they were about whether prompt and KV caching actually worked for that model on that runtime. A runtime that reprocesses the full conversation every turn will lose to one that caches the prefix, regardless of format. Test caching with your real context lengths before you commit, and do not trust the streaming counter to tell you about it, because it never measures the part that caching fixes.

So which one

If you want the one-line version: GGUF is the conservative default, and you should reach for it whenever you are uncertain, need portability, or want a specific quant. Reach for MLX when you are locked to Apple Silicon, run single-user interactive workloads with long outputs, or want to fine-tune on the machine you already own.

And if you are choosing this for a team rather than a laptop, treat it as the architecture decision it is. The format you standardize on shapes your model coverage, your fallback options, and your serving setup for as long as the stack lives, and re-quantizing a fleet after the fact is the kind of avoidable week of work I keep getting hired to clean up. Decide it on the five questions, not on a screenshot.

I kept watching this happen. Ten minutes of exploration before any real work, every single time. If you work across multiple repos or come back to a project after a couple weeks, it’s worse. The agent is essentially reading the codebase for the first time, again.

I wanted to fix this.

The idea

A few weeks ago Karpathy tweeted about using LLMs to build personal knowledge bases. The workflow: collect raw sources, have an LLM compile them into a structured wiki of markdown files, then query and build on that wiki over time. Every query makes the wiki richer. The knowledge adds up.

The part that stuck with me: he’s not using fancy RAG. The LLM maintains its own index files and summaries, and at his scale (~100 articles, ~400K words) it just works. The LLM reads its own compiled knowledge to answer questions.

Codebases are raw data too. Source files are unstructured information that happens to be executable. What if the LLM compiled a codebase into a wiki the same way, with module overviews, architecture docs, concept articles, and then used that wiki as its starting point for every session?

That’s CodeWiki.

How it works

CodeWiki is a thin Rust CLI called cw paired with a Claude Code skill. The CLI handles git ops, directory scaffolding, and metadata. The agent does all the actual reading and writing. No API keys, no LLM calls from the CLI. Your agent is the intelligence.

When you run cw init in a repo, it creates a wiki directory at ~/.codewiki/<project>/ with this structure:

~/.codewiki/my-project/
├── _index.md         # master index
├── _architecture.md  # system overview
├── _patterns.md      # recurring patterns
├── _meta.yaml        # last compiled commit
├── modules/          # one article per module
├── concepts/         # cross-cutting concerns
├── decisions/        # why things are the way they are
├── learnings/        # bugs fixed, patterns discovered
└── queries/          # past Q&A, filed back

The first time you start a Claude Code session after init, the skill kicks in. The agent walks your codebase, reads the source files, and writes wiki articles. Module articles describe what each part of the code actually does. Not what it’s supposed to do, what it does. Key files, functions, data flow, connections to other modules.

Concept articles cut across modules. “How does error handling work across the system” or “how does data flow from request to response.” These are the questions that normally require reading eight files across four directories. The wiki answers them in one place.

Keeping it fresh

The wiki is only useful if it stays current. Every article has YAML frontmatter with a source_files field:

---
title: Authentication Module
type: module
source_files:
  - src/auth/middleware.py
  - src/auth/tokens.py
tags: [auth, middleware, jwt]
---

The CLI tracks which commit the wiki was last compiled against. When you start a new session, cw status diffs against that commit and cross-references changed files against every article’s source_files:

$ cw status
Changed since last compile (4964c23):
  M src/auth/middleware.py
  M src/auth/tokens.py

Stale articles:
  ! modules/auth.md

The agent sees this and knows exactly what to re-read and update. No guessing, no full recompile.

At session end, the agent writes learnings and decisions back into the wiki. Fixed a bug? That becomes learnings/auth-token-race-condition.md. Made a design decision? That’s decisions/switched-to-redis-sessions.md. Then it updates _meta.yaml with the current commit hash.

Next session picks up where this one left off.

The CLI

About 400 lines of Rust. Here are the commands:

cw init                # scaffold wiki for current repo
cw status              # what changed since last compile
cw path                # print wiki path
cw projects            # list all wikis
cw index               # rebuild _index.md from article frontmatter
cw meta update         # record current commit as compiled

cw setup claude-code   # install skill into Claude Code
cw setup codex         # install instructions for Codex
cw setup qmd           # register wiki as QMD search collection

The CLI doesn’t make any LLM calls. It handles the things agents are bad at: tracking git state, knowing which files changed, maintaining timestamps. The agent handles what it’s good at: reading code and writing about it.

Search with QMD

For larger wikis, QMD by Tobi Lutke adds proper search. It’s a local search engine for markdown with hybrid BM25 plus vector search plus a small reranker model. Running cw setup qmd registers your wiki as a searchable collection. The agent can then query the wiki through QMD’s MCP server during a session.

At the scale of most repos people actually work in, you probably don’t need it. A well organized wiki with an index file is enough for the LLM to navigate on its own. But when the wiki gets large, QMD keeps retrieval fast.

Viewing with Obsidian

All wiki articles live at ~/.codewiki/. Open that directory as an Obsidian vault and you get a browsable knowledge graph of all your projects. Articles use [[backlinks]] so modules connect to each other. The auth article links to [[database]] and [[api]]. You never have to write or edit these articles yourself. The agent maintains everything.

Why not RAG

Traditional RAG chunks your code, embeds it, retrieves fragments when you ask a question. You get decontextualized snippets and hope the LLM can stitch them together.

CodeWiki is different. The LLM reads the code and writes structured articles about it. The auth article already connects the middleware to the token service to the database layer. That connection doesn’t exist in any single source file. It exists in the compiled understanding.

Karpathy found the same thing with his research wiki. You don’t need vector search over raw data when you have a well organized collection of articles. The LLM reads the index, finds the relevant articles, reads those. Simple and it works.

Getting started

git clone https://github.com/mraza007/codewiki.git
cd codewiki
cargo install --path .

cd your-project
cw init
cw setup claude-code

Start a Claude Code session and the skill handles the rest. The project is MIT licensed and on GitHub.

Coding agents help with this. I can point Claude Code at an issue and let it work while I do something else. But that’s still one agent, one issue, one terminal. If I have 10 issues labeled “agent-ready,” I’m not babysitting 10 terminal tabs.

I wanted something that just watches for new issues and sends agents after them. Then OpenAI released their Symphony spec, an orchestrator pattern for their Codex agent. The architecture was solid: poll an issue tracker, dispatch agents into isolated workspaces, reconcile when issues close. But it was built around Codex and Linear, and I use Claude Code and GitHub Issues.

So I took the ideas I liked from Symphony and built my own. That’s Baton.

What it does

Baton is a Python daemon. You start it in your repo, it polls GitHub Issues matching your configured labels, creates an isolated git worktree per issue, and runs Claude Code CLI as a subprocess. When the agent finishes and opens a PR, Baton releases the claim and grabs the next issue.

One config file. One command. Go do something else.

WORKFLOW.md -> Orchestrator -> Worker (per issue)
                  |              |
                  |              +-- git worktree create
                  |              +-- hooks (before_run)
                  |              +-- claude -p "<prompt>"
                  |              +-- check issue state
                  |              +-- hooks (after_run)
                  |
                  +-- Poller (gh issue list)
                  +-- Dispatcher (concurrency control)
                  +-- Reconciler (stale run detection)

The name comes from relay races. You hand off the baton and the runner goes.

The config

Everything lives in WORKFLOW.md. YAML front matter for configuration, Jinja2 template below for the prompt. Baton reloads this file on every poll cycle, so you can change settings without restarting.

---
tracker:
  kind: github
  labels: ["agent"]
  exclude_labels: ["blocked"]

polling:
  interval_ms: 30000

agent:
  max_concurrent: 3
  max_turns: 5
  command: claude
  permission_mode: bypassPermissions

hooks:
  before_run: |
    git fetch origin main && git rebase origin/main
  timeout_ms: 60000
---

You are an autonomous software engineer working on issue #{{ issue.number " }}: {{ issue.title " }}.

{{ issue.body " }}

{% if attempt %}
This is continuation attempt {{ attempt " }}. Review what was done and continue.
{% endif %}

## Instructions

1. Understand the issue requirements
2. Write clean, well-tested code
3. Run existing tests to make sure nothing breaks
4. Commit your changes with a descriptive message
5. Push the branch and create a pull request linking to #{{ issue.number " }}

Labels filter which issues get picked up. max_concurrent controls parallel agents. max_turns is the retry limit per issue. Hooks run shell commands at different points. I use before_run to rebase on main so the agent starts from fresh code.

The prompt template gets issue.number, issue.title, issue.body, issue.labels, and attempt for retries. Standard Jinja2.

Why worktrees

Each issue gets its own worktree under .symphony/worktrees/, with a branch name slugified from the issue title: baton/fix-login-redirect-42.

I thought about Docker containers and temp directories but worktrees won out. They share the git object database so creating one is almost instant, unlike a full clone. They’re real checkouts, so linters and test runners and build scripts all work without any path hacking. And they’re isolated. If one agent trashes its branch, the others don’t care.

Why gh CLI instead of the GitHub API

Baton shells out to gh issue list and gh pr create instead of using PyGitHub or the REST API. Seems odd, but think about setup.

With the API, you need a personal access token. You need to configure it somewhere. You need to handle rate limits.

With gh, you authenticate once (gh auth login) and everything on your machine uses the same credentials. No token management in the orchestrator. The tradeoff is speed, but Baton polls every 30 seconds. The overhead of a subprocess call doesn’t matter at that pace.

The permission problem

This tripped me up. Claude Code has permission modes: default asks for everything, acceptEdits auto-approves file edits but prompts for shell commands, and bypassPermissions auto-approves everything.

I started with acceptEdits because it felt like the right balance. Let the agent write code freely, but make it ask before running commands. Problem: “ask” means a human clicking yes, and in an autonomous orchestrator there’s no human. The agent just blocks forever waiting for a prompt nobody will answer.

I wasted about 20 minutes watching it hang before I figured this out. For autonomous operation you need bypassPermissions, which maps to --dangerously-skip-permissions. The flag name is honest about the risk. I’m comfortable with it because the agents run in isolated worktrees on disposable branches, not in my main checkout.

Auto-releasing on PR creation

My first version had a dumb problem. The agent would finish its work, create a PR on turn 2 of 5, and Baton would keep scheduling continuation turns for the remaining 3. The slot was occupied but nobody was doing anything useful.

The fix: after each worker finishes, check if a PR exists for that issue’s branch. If yes, release the claim immediately and free up the slot. If not, schedule a short retry.

pr_exists = await self.tracker.check_pr_exists(issue.number)
if pr_exists:
    log.info(f"PR_READY #{issue.number} -- PR found, releasing claim")
    return "pr_created"
return "no_pr"

Small change, but it meant the orchestrator stopped wasting time on finished work.

Extensibility through skills and MCP servers

Baton itself is deliberately simple. It polls, dispatches, and manages worktrees. The interesting part is what you put in the prompt and what tools you give the agent.

Claude Code supports MCP servers, which means you can wire up external tools and the agent can use them during its run. Baton passes MCP server config through to each worker:

agent:
  mcp_servers:
    - name: playwright
      command: npx @playwright/mcp@latest

That means the agent has access to a headless browser while it works. It can open a page, click around, take screenshots, verify that the UI renders correctly. You don’t have to build that into Baton. You just declare which MCP servers you want and the agent figures out when to use them.

Same idea with CLI tools. If agent-browser is installed on the machine, you can tell the agent to use it in the prompt template. “Before creating a PR, open the app with agent-browser and verify the acceptance criteria.” The agent spins up a local server, opens the page, clicks buttons, fills inputs, takes snapshots. All from instructions in WORKFLOW.md, nothing hardcoded in the orchestrator.

Claude Code also has skills, which are reusable prompt fragments that teach the agent specific capabilities. If you have a code review skill or a testing skill installed, the agent can use them during its run. Baton’s config supports a skills list for this:

agent:
  skills:
    - code-reviewer
    - accessibility-checker

You can also override skills per issue by adding a ## Skills section to the issue body. If one issue needs Playwright but the others don’t, just add it to that issue.

The point is that Baton doesn’t need to know about browsers or test runners or linters. It just needs to dispatch agents with the right config. The prompt and the tools do the rest.

Putting it together: a todo app from scratch

To see all of this working end to end, I had Baton build a todo app. Fresh repo, no code. I created three GitHub issues labeled baton:

1. Create basic HTML structure
2. Add JavaScript for create/delete
3. Add localStorage persistence

The WORKFLOW.md prompt told the agent to use agent-browser for verification before opening PRs. I ran baton start and went to make coffee.

Baton picked up issue #1, created a worktree on baton/create-basic-todo-app-html-structure-1, and dispatched Claude Code. The agent wrote index.html, spun up a local server with npx serve, opened it with agent-browser, confirmed the layout rendered, then committed, pushed, and opened a PR. The PR description included what agent-browser found:

Opened http://localhost:3456 and confirmed the page renders correctly. Ran agent-browser snapshot -i confirming interactive elements: textbox and button.

I merged it. The issue auto-closed (the PR had Closes #1). Baton saw the issue was gone on the next poll, released the slot, and picked up issue #2. Same cycle. Then #3.

Three issues, three PRs, three merges. I didn’t write a line of the todo app. The agent-browser verification wasn’t built into Baton. It was just instructions in the prompt and a CLI tool on my machine.

Getting started

pip install -e .
cp WORKFLOW.md.example WORKFLOW.md
# Edit WORKFLOW.md: set your labels, tweak the prompt
baton start

You need Python 3.11+, Claude Code CLI (claude), GitHub CLI (gh) authenticated, and Git.

The code is at github.com/mraza007/baton. MIT licensed. About 10 Python modules, no external services, no databases. State lives in memory with JSON persistence for the status command.

What I want to add next

• A proper TUI instead of baton status reading a JSON file
• Issue dependency ordering so issue 3 waits for issue 2 if it needs to
• Cost tracking per issue, so I can see what automating the backlog actually costs in tokens
• More trackers besides GitHub Issues (Linear, Jira, GitLab)

If you’ve got a repo with a pile of issues sitting there, try pointing Baton at it. Start with one label and max_concurrent: 1. See what it does. The setup takes about five minutes and the worst case is you get a bad PR that you close. The code is MIT licensed, the whole thing is ten files, and there’s nothing weird in it. Fork it, break it, rip out the parts you don’t like.

If you try it, I want to hear what breaks.

I write a newsletter called Dev Console where I cover what’s actually happening in AI, minus the hype. New tools, real use cases, stuff I’m building. If this post was interesting, you’ll probably like it.

Harness engineering. It’s the practice of building everything around the LLM — the execution environment, tool definitions, safety boundaries, observability, and lifecycle management. The stuff that turns a chatbot into a production system. (If you want the full definition of what an agent harness is and where the term came from, I’ve written a dedicated post on that.)

If you work in DevOps, you’ve been doing this for years. You just called it something else.

Why Harnesses Matter More Than Models

Pick any AI agent demo. Strip out the model. What’s left?

A container or sandbox. A set of callable tools. A loop that reads output and decides what happens next. Logging. Timeouts. Cleanup.

That’s the harness. And it’s where agents succeed or fail. A great model in a bad harness hallucinates, loops forever, leaks secrets, or silently does nothing useful. A decent model in a good harness stays bounded, recovers from errors, and produces auditable results.

DevOps engineers already think this way. You don’t just pick a good application — you build the infrastructure that makes it reliable. Same thing here.

The Five Parts of a Harness

Here’s how I break down harness engineering into components. Each one maps directly to something you’ve built before.

1. Execution environment. Where does the agent run? A container, a VM, a temporary directory, a git worktree. You need isolation so the agent can’t corrupt shared state. You need reproducibility so runs are consistent. This is the same problem as CI job runners. Docker, Firecracker, nsjail — pick your isolation boundary.

2. Tool definitions. Tools are the agent’s API surface. Read a file. Run a command. Query a database. Call an endpoint. Each tool needs input validation, output formatting, error handling, and permission scoping. Think of it like designing an API — you wouldn’t expose raw database access through a REST endpoint. Don’t give an agent raw shell access either. The tool layer is your contract.

3. Control loop. Observe, decide, execute, verify. The loop is what makes an agent an agent instead of a one-shot prompt. Your job as a harness engineer is to decide: how many iterations? What’s the timeout per step? What happens when a tool call fails? When does the loop escalate to a human? This is the same logic you put in health check loops and deployment rollback controllers.

4. Guardrails. Cost caps. Token limits. Command allowlists. File path restrictions. Rate limiting on external calls. Without guardrails, an agent can burn through your API budget in minutes or write to paths it shouldn’t touch. Every guardrail is a policy decision — same as IAM policies, network rules, and resource quotas you already manage.

5. Observability. If you can’t see what the agent did, you can’t debug it, audit it, or trust it. Log every tool call, every LLM response, every decision point. Capture diffs, timing, token usage, and cost. This is no different from structured logging in any production system. The difference is that agent traces are longer and less predictable than HTTP request traces, so you need good tooling to navigate them.

Where DevOps Context Overlaps

Here’s where your existing skills plug in directly.

Infrastructure as code. Agent harnesses should be declarative and version-controlled. The tool definitions, policies, and environment specs should live in config files, not hardcoded in application logic. When you change a tool’s behavior, that change should be reviewable in a PR.

Pipeline orchestration. Multi-agent systems look a lot like multi-stage pipelines. One agent does research, passes context to a planning agent, which passes a plan to an implementation agent. You’re managing handoffs, shared artifacts, and failure propagation — the same coordination problem as CI/CD stages.

Incident response. When an agent goes wrong, you need the same muscle memory. Check the logs. Find the failing step. Understand the input that caused it. Roll back if needed. The debugging workflow is identical.

Security boundaries. Least privilege applies to agents just like it applies to services. What tools can this agent access? What files can it read? Can it make network calls? Can it spend money? Every agent needs a security boundary, and DevOps engineers already think in terms of boundaries.

Getting Started

If you want to start building harnesses, you don’t need a new framework. Start with what you have.

Take a simple task — say, analyzing a failed CI build. Write a script that collects the logs, sends them to an LLM with a prompt, parses the response, and posts a summary to Slack. That’s a harness. A minimal one, but it has all the components: environment setup, tool use (log collection, Slack posting), a control flow, and output handling.

Then add complexity. Let the LLM decide which logs to fetch. Add a retry loop. Add a cost cap. Add structured logging. Each addition is a harness engineering decision.

You don’t need to learn ML. You don’t need to fine-tune models. You need to build the infrastructure that makes models useful — and that’s the job you already do.

Harness engineering isn’t a new discipline. It’s DevOps applied to a new kind of workload. The sooner you see it that way, the faster you’ll build agents that actually work in production.

You start a new session and it re-discovers the same patterns. Repeats the same mistakes. Asks the same questions. It’s like working with a brilliant colleague who gets amnesia every night.

I got tired of this. So I built EchoVault — a local memory system that gives coding agents persistent memory across sessions. No cloud. No API keys. No cost. Just a SQLite database and some Markdown files on your machine.

The Problem Is Real

I use coding agents daily across multiple client projects. Claude Code, Cursor, Codex — I switch between them depending on the task. Every time I start a session, I’m repeating context that the agent should already know.

“We chose FastAPI over Flask because of async support.” “The deploy script needs –no-cache or the CSS breaks.” “Don’t touch the legacy auth module — it’s being replaced next sprint.”

I was copy-pasting this stuff into every session. That’s not how tools should work.

I tried existing solutions. Supermemory announced their MCP and I was tempted, but it saves everything in the cloud. I work with multiple companies as a consultant — I don’t want codebase decisions stored on someone else’s servers. Claude Mem was the first tool I tried, but it was eating too much memory in my sessions and became a bottleneck when running multiple agents at the same time.

So I built my own.

How EchoVault Works

EchoVault runs as an MCP server. When your agent starts a session, it has three tools available:

• memory_context — load prior decisions, bugs, and context for the current project
• memory_search — find specific memories by keyword or semantic similarity
• memory_save — persist a decision, bug fix, pattern, or learning

The agent calls these tools like it calls any other tool. No hooks. No shell scripts. No prompt injection. The MCP protocol handles everything.

Here’s what happens in practice:

Session start. The agent sees memory_context in its available tools. The tool description says “You MUST call this at session start.” The agent calls it and gets back a list of prior memories for the project. Now it knows what happened yesterday.

During work. You ask about authentication. The agent calls memory_search with “authentication” and gets back the decision to use JWT, the bug with token refresh, and the migration plan. It has context before writing a single line of code.

Session end. The agent just fixed a tricky race condition. The tool description says “You MUST call memory_save before ending any session where you made changes.” It saves the root cause, the fix, and what to watch for.

Next session, that knowledge is there. Every session builds on the last one.

The Architecture

I kept it simple. The whole system is four things:

~/.memory/
├── vault/                    # Obsidian-compatible Markdown
│   └── my-project/
│       └── 2026-02-01-session.md
├── index.db                  # SQLite: FTS5 + sqlite-vec
└── config.yaml               # Optional embedding config

Markdown vault. Every memory gets written to a session file — one file per day per project. These are valid Markdown with YAML frontmatter. You can point Obsidian at ~/.memory/vault/ and browse your agent’s memory visually. You can read them in any editor. They’re not locked in a proprietary format.

SQLite index. This is where search happens. FTS5 handles keyword search out of the box — no configuration needed. If you want semantic search (where “authentication” matches a memory titled “JWT token setup”), add an embedding provider. I use Ollama with nomic-embed-text locally. You can also use OpenAI or OpenRouter if you prefer cloud.

MCP server. The agent talks to EchoVault through the Model Context Protocol. Three tools, stdio transport, nothing fancy. The server starts when the agent needs it and stops when the session ends. Zero idle cost.

Secret redaction. Three layers. Explicit <redacted> tags for things you mark yourself. Pattern detection that catches API keys, passwords, and credentials automatically. And .memoryignore rules for custom patterns. Nothing sensitive hits disk.

Making Agents Actually Save

Here’s the thing about MCP tools — the agent can call them, but will it? Retrieval works well because agents tend to grab context at the start. Saving is the hard part. The agent finishes its work and moves on. It doesn’t naturally think “I should save what I learned.”

The trick is the tool descriptions. When you register an MCP tool, you include a description. Agents read these descriptions and treat them as instructions. So instead of:

"Save a memory for future sessions. Call this when you make decisions."

I wrote:

"Save a memory for future sessions. You MUST call this before ending
any session where you made changes, fixed bugs, made decisions, or
learned something. This is not optional — failing to save means the
next session starts from zero."

That “MUST” language makes a real difference. It’s not 100% reliable — nothing with LLMs is — but agents follow strong tool descriptions much more consistently than passive ones.

Cross-Agent Memory

One of the things I wanted was a single vault for all my agents. A memory saved by Claude Code should be searchable from Cursor or Codex. They’re all working on the same codebase. Why should they have separate memories?

EchoVault stores everything in one place. The MCP server is the same regardless of which agent connects to it. Setup is one command per agent:

memory setup claude-code   # writes ~/.claude.json
memory setup cursor        # writes .cursor/mcp.json
memory setup codex         # writes .codex/config.toml + AGENTS.md
memory setup opencode      # writes opencode.json

Each agent has its own config format and conventions. Claude Code uses JSON with mcpServers. Cursor uses the same schema but different file paths. Codex uses TOML with [mcp_servers]. OpenCode uses JSON with a mcp key and a different command format (command as an array instead of separate command + args).

I wrote shared helpers so each agent’s setup is just a thin wrapper around _install_mcp_servers() or _install_toml_mcp(). Adding a new agent takes maybe 20 lines of code.

What Gets Saved

Not everything should be a memory. Trivial changes don’t need to be persisted. Information that’s obvious from reading the code doesn’t need a memory. The goal is to capture what a future agent wouldn’t know from just looking at the codebase.

Good memories:

• Decisions. “Chose JWT over sessions because the API needs to be stateless.” A future agent reading the code sees JWT but doesn’t know why.
• Bugs. “The ORM lazy-loads relationships by default, causing N+1 queries in the user list endpoint. Fixed by adding .options(joinedload(...)). Root cause: SQLAlchemy default behavior.” A future agent won’t hit the same bug.
• Patterns. “All API endpoints follow the pattern: validate input, check permissions, execute, return response. Don’t add business logic in the route handler.” A future agent follows the existing patterns instead of inventing new ones.
• Context. “The legacy auth module is being replaced. Don’t modify it — changes go into the new auth service at src/auth/v2/.” A future agent doesn’t waste time on dead code.

Each memory has a title, a “what happened” summary, optional “why” and “impact” fields, tags, and a category. Search returns compact ~50-token summaries. Full details are fetched on demand so context windows don’t get bloated.

The Technical Bits

A few implementation details that might be useful if you’re building something similar.

FTS5 for keyword search. SQLite’s FTS5 extension is fast and works with zero configuration. No external service needed. It handles stemming, phrase matching, and ranking. For most use cases, this is all you need.

sqlite-vec for semantic search. When you want “authentication” to match “JWT token rotation”, you need vectors. I use sqlite-vec to store embeddings right in the same SQLite database. No vector database needed. Embedding providers are pluggable — Ollama for local, OpenAI or OpenRouter for cloud.

Hybrid search. The search pipeline runs FTS5 first (fast, precise), then semantic search (slower, fuzzy), and merges the results. This gives you the best of both worlds — exact keyword matches and semantic similarity.

TOML parsing with fallbacks. Codex writes some non-standard TOML — unquoted filesystem paths as table keys, dotted version strings as key names. Standard tomllib chokes on these. I added a fallback that appends the MCP section directly via string operations when parsing fails. It’s not pretty but it handles real-world config files.

Symlink handling. Some agents create symlinks in their skill directories. shutil.rmtree() crashes on symlinks. Small thing but it bit me in production.

Setting It Up

pip install git+https://github.com/mraza007/echovault.git
memory init
memory setup claude-code

That’s it. Three commands. The agent has memory now.

If you want semantic search, configure an embedding provider:

memory config init
# Edit ~/.memory/config.yaml to set your provider
memory reindex

For fully local operation with no external API calls, use Ollama:

embedding:
  provider: ollama
  model: nomic-embed-text

What I’ve Learned

Building this taught me a few things about agent tooling.

Tool descriptions are instructions. Agents read them and follow them. Strong, directive language in tool descriptions is more effective than passive documentation. “You MUST” works better than “You can.”

Local-first matters. Not because of ideology, but because of practical constraints. Consultants work with multiple clients. Sensitive decisions shouldn’t leave the machine. And when your internet goes out, local tools still work.

MCP is the right abstraction. Instead of writing agent-specific hooks, skills, and config formats, I write one MCP server and each agent connects to it. When a new agent comes along, I add a setup function for its config format. The memory logic doesn’t change.

Simple storage wins. Markdown files you can read in any editor. SQLite you can query with any tool. No custom binary formats. No daemon to keep running. The system is completely inspectable and debuggable.

The code is at github.com/mraza007/echovault. It’s MIT licensed. If you’re tired of your agents forgetting everything, give it a shot.

Now swap out the hardcoded logic for an LLM. That’s it. That’s an AI agent in simpler terms. The fancy demos want you to think it’s magic. Some brand new thing you need to learn from scratch. It’s not. When you take away the hype, an agent is just a controlled automation loop. The LLM handles the reasoning and everything else is infrastructure you’ve built a hundred times.

Here’s what matters, the agent itself isn’t the hard part but The harness is, the execution environment, tooling, guardrails, and observability. It’s all the important stuff that makes automation work in production.

DevOps engineers have been building harnesses forever. CI runners. Deployment pipelines. Infrastructure automation. The patterns are the same. The skills transfer directly.

So if you’re wondering whether AI agents are worth learning, here’s the short answer. You’re already halfway there.

What an Agent Actually Looks Like

Let’s forget the marketing hype around AI agents and understand from a DevOps engineer’s point of view, what an agent actually looks like. An AI agent has six parts.

1. An LLM: Now LLM is the most important part of an agent as this acts as a brain. It reads context and decides what to do next. It doesn’t touch anything directly.

2. A workspace: Think of it as a sandboxed environment. A cloned repo. A container. A temp directory. Same as any CI job.

3. A set of tools: These are the actions it can request. Read a file. Run a command. Call an API. Query logs. The agent doesn’t run these itself. It asks for them.

4. A control loop: This is the core pattern. Observe the current state. Decide an action. Execute it. Check the result. Keep going until you’re done.

5. Policies and limits: Timeouts. Permission boundaries. Rate limits. Cost caps. Without these, agents can spin forever or do things they shouldn’t.

6. A termination condition: The agent needs to know when to stop. Task complete. Error threshold hit. Human review needed. Something has to end the loop.

Now none of this is new as you’ve built systems with all these components. The only difference is the LLM sitting in the decision seat.

The Harness Does the Heavy Lifting

Everyone focuses on the LLM. They miss the important part. The harness is what makes an agent actually work.

The harness is everything around the model. It spins up the environment. Exposes tools. Executes commands on the agent’s behalf. Captures logs and diffs. Enforces limits. Decides when the loop should stop.

Sound familiar? It should. This is what CI runners do.

GitHub Actions. GitLab runners. Jenkins agents. They all follow the same pattern. Spin up an isolated environment. Run steps. Capture output. Handle success and failure. Clean up.

An agent harness does the exact same thing. The only twist is the steps aren’t hardcoded in YAML. They come from the LLM at runtime.

This is why DevOps engineers are perfect for this work. You already think about isolation, execution, logging, and cleanup. You already build systems that run untrusted code safely. Agent harnesses are the same problem with a new input source.

Tool Use Is the Safety Mechanism

Agents don’t touch systems directly. This matters. The LLM never runs a command itself. Never writes a file itself. It requests actions through tools.

The harness gets the request. Validates it. Executes it in a controlled way. Returns a structured result.

This is how you keep agents safe.

Say the agent wants to run a shell command. The harness can check it against an allowlist. Run it in a sandbox. Set a timeout. Capture stderr. The agent never gets raw shell access.

Same thing for file operations. The agent requests a file write. The harness checks the path. Validates the content. Writes the file and returns confirmation.

You control what tools exist. You control how they behave. You control what the agent can even ask for.

This is the same idea behind least privilege. The agent only gets access to what it needs. The harness enforces the boundary.

The Control Loop in Practice

The core of any agent is the control loop. It looks like this.

1. Observe: The agent reads the current state. Test output. Log files. Diffs. Error messages. Whatever context it needs.

2. Decide: The LLM looks at the state and picks an action. Run another test. Edit a file. Ask for more information. Give up.

3. Execute: The harness runs the requested action and returns the result.

4. Verify: The agent checks if the action worked. Did the test pass? Did the error go away? Is the task done?

5. Repeat: If the task isn’t complete, go back to observe.

This loop keeps running until a termination condition hits—success, failure, timeout, max iterations, or human intervention.

You’ve seen this before: build, test, fix, rebuild. CI pipelines do this, deployment rollbacks do this, and health check loops do this.

Agents just make the “decide” step dynamic instead of scripted, and here’s where they actually help in DevOps work.

CI failure analysis. When a test fails, the agent reads the logs, checks the diff, identifies the cause, and suggests a fix—maybe even applying it and rerunning the test.

Terraform drift detection. The agent compares actual state to declared state, flags the drift, and proposes a remediation plan while a human approves before anything changes.

Kubernetes manifest review. The agent checks YAML against best practices (missing resource limits, no liveness probes, exposed secrets) catching the stuff humans miss in review.

Cost anomaly investigation. When spending spikes, the agent queries cost explorer, correlates with recent deployments, and surfaces the likely cause, saving an hour of digging.

Incident log triage. Faced with pages of logs, the agent reads them, extracts the relevant lines, and summarizes what went wrong (not replacing the engineer, but getting them to the answer faster).

Notice the pattern: the agent assists and handles the tedious parts while the human stays in control of decisions that matter.

AI agents sound complicated with their new frameworks, new terminology, and new paradigms.

But look past the hype and you’ll see something familiar.

An agent is an automation loop where the LLM picks the next step, the harness executes it safely, tools provide controlled access to systems, and policies keep things bounded.

This is CI/CD architecture, infrastructure thinking, the stuff you already do.

When you read about agent frameworks or watch demos of coding assistants, you now have a lens to see the harness underneath, spot the control loop, and ask the right questions: what tools does it expose, what limits exist, and how does it handle failure?

You don’t need to become an ML engineer to understand agents—you just need to recognize the infrastructure patterns you’ve been using all along.

The LLM is the new part. Everything else is your domain.