harness engineering
Your agents write code. Nobody trusts it yet.
Fleets of coding agents burn money in ways a single Copilot seat never did. Output varies run to run. PRs pile up faster than humans can review them. The spend is visible on the invoice; the value is stuck in the queue. I build the harness that fixes that.
The problems I fix
- Inconsistent output. The same prompt produces a clean patch on Monday and a mess on Tuesday. Without evals you cannot tell whether a model change, prompt tweak, or tooling update made things better or worse.
- Unreviewed PR backlogs. Agents open pull requests all day. Your seniors were the bottleneck before; now they are drowning. Merging blind is how agent code becomes an incident.
- Invisible spend. Token costs climb and nobody can say which workflows earn their keep. Agents retry, loop, and rabbit-hole on your dime.
What I build
- Observability. Traces, cost attribution, and failure taxonomies for every agent run. You see what each agent did, what it cost, and where it went off the rails.
- Evals. Task-level evaluation suites that gate changes to prompts, models, and harness code. Regressions get caught before they hit the fleet.
- Automated review pipelines. Layered review for AI-generated PRs: static checks, test gates, and agent-assisted review tuned to your codebase, so human reviewers see only what deserves their attention.
- Consistency engineering. Prompt and context discipline, guardrails, and deterministic scaffolding that narrow the variance between runs.
Why me
I run agent fleets on my own projects and have spent 20+ years building the CI/CD and testing infrastructure this work descends from. Agent reliability is a systems problem. That is my home turf.