The Rise of the Harness Engineer

We're in an intense moment as software engineers. On one hand, coding agents have become genuinely amazing and we’re using them all the time with higher and higher throughput. On the other, they build in sloppy, bug-accumulating ways, and we’re under unrelenting pressure to keep them in check.

I'm a researcher and engineer at Tessl, where I've spent a lot of the last while helping build our internal software factory and evaluating what agents actually do when you point them at real work. The conclusion I keep arriving at is this: as agents write more and more of our code, engineering doesn't matter less. It matters more. It just looks different from the engineering many of us grew up doing. I've started calling the person who does this new kind of work a harness engineer, and I think it's a role a lot of us are about to grow into, some as a full-time specialism, many more as a growing slice of the job alongside product engineering.

Let me walk through how I got there, and what new skills are emerging as crucial to staying in control.

The agents got great, but cracks show over time

Start with the good news, because it's genuinely remarkable. The adoption curve for AI coding tools has gone close to vertical. A year ago adoption was spiky and experimental; now, across a lot of organisations, the vast majority of code is AI-authored [see Faros AI, AI whiplash report]. But, hand-in-hand, the number of bugs and production incidents are increasing with it, as more PRs slip through with no human code review.

The benchmarks tell the same story from the other side. Task based benchmarks are largely saturated – if you can define a task cleanly and measure if it was done, an agent can complete it. But if you ask agents to refactor large codebases, or build on their work over time, then we see deterioration, and much lower levels of success.

Better agents will help over time, but also come at a cost

The natural hope is that better models will simply dissolve all of this. But there will be heavy trade offs – much of the issue is reasoning about overall design decisions and reviewing existing code to follow patterns and stay consistent – all of which take time and reasoning effort to do. Expecting agents to do this well from first principals in every session would be incredibly expensive even if they can do it.

If you plot how much it costs to run an agent against its intelligence, the intelligence axis moves roughly linearly while price moves on a log scale. [see Artificial Analysis’s pareto frontier charts] Across the range that's something like a 100× spread, and the difference between a top-tier model and a perfectly capable cheaper one can be around 10× for the same task. Even when the very best model can do the job, that can be an extraordinarily expensive way to solve a problem. And when you start running agents at real throughput, this stops being academic. We closed something like 600 PRs in a single week recently, and we're a small company. When that much is flowing through, cost and efficiency become first-class engineering concerns.

The steering gap

We think a lot about skills at Tessl, so we built a benchmark to understand them properly. We took around a thousand skills from open-source repos, built tasks that would exercise those skills, and then ran agents through the tasks both with and without the skill so we could compare. We graded two different things: did the agent complete the task, and did it actually follow the instructions the skill laid out, not just "did it finish," but "did it do it the way it was told." [see: https://arxiv.org/abs/2606.17819]

Zoom

What did we learn?

First, well defined skills reduce the difficulty of tasks and let cheaper, smaller agents reach the same level of success as much larger, more expensive ones. If we can make it easy to find the right guidance and break tasks apart well, we’ll win in both consistency and efficiency.

Second, and less comfortably: instructions don't get followed thoroughly. Task completion rates were high across almost everything we studied, but only about 70% of the instructions in a skill were followed on average. When you break a skill down into its individual instructions and check them one by one, even the best models are quietly ignoring a big chunk of what you told them. Skills aren’t enough, they need to be paired with checks that the rules are followed consistently.

There’s an enormous amount of value in designing how we use agents, rather than just using them as they are and hoping they make the right choices.

Engineering matters more, not less

Here’s the shift. When we hand-wrote code, we could review and discuss it, and many of the key decisions we were making could emerge and be shared implicitly as we worked. We’d know to go sketch an architecture diagram on a board when it stopped scaling the way we wanted, or we’d spike on which library to choose at the moment we realised it was important, and we could feed those choices back in. It was easier to mix taking time on the overall approach and the specific task.

Now, the agents are going so fast, it’s easy to fall into only being able to react – being pinged to review the latest PR when it’s ready, and feeling frustrated with the same errors being repeated.

If we want to go really fast, we need to be proactively spending our time designing the system in which the agents run, making our choices explicit and measuring whether the system that is emerging is what we want.

I see three skill sets that matter more now than they used to, and that are a little different from the ones many of us spent years developing.

1. Systems thinking: capturing invariants

Some of what you want from a change is specific to the task in front of you. But a lot of it is really an expression of a general principle you want to stay true across the whole system. If you can identify those invariants, the things you want to keep being true, and then capture and enforce them, you have a real shot at keeping consistency and holding the quality bar high no matter who, or what, is doing the writing.

What do invariants look like in practice? Design systems are a great example, you describe the types of layouts you want, the brand voice, the components, what a button looks like. Individual pages are then built from those reusable components. They show up in architecture, how you want code structured, which libraries you've chosen, opinionated views on how they should be called. Each new feature follows the patterns and choices consistently. And they show up in best practices, how errors should surface through the stack, how things should be named, how data should flow.
In the past, these lived in a code owner's instinct; you could ask them to sketch an architecture diagram, or sit with a designer over the Figma files, and then have them check PR by PR whether things matched.

Now we want to make these choices explicit, and when we do we can enforce them consistently much more cheaply than we could before, effectively steering development from these patterns. Why is it easier? Because just as agents are better at writing code, they are also better at reviewing code – especially when the rules are clear and thoroughly described.

How do we enforce them:

• We can write skills that describe exactly what we care about and load them at the right moment.
• We can put deterministic checks into CI, linters, and tools like ast-grep, which I'd barely heard of a few months ago and now talk about constantly. (Agents are excellent at writing these, so the thing that used to be a pain is now easy.)
• We can add narrow verifiers, tight rules measured on every single file by a quick agent, to enforce the kinds of constraints that are hard to pin down deterministically.
• And we can do agentic code review with broader prompts describing the principals we care about.

Between them these form a system of guidance and guardrails that steer the agents when they are working, and review if they did things correctly. They leave a lot less on the table to be caught in ad hoc review, or to slip through and cause problems later.

2. Analytics: reason about the system with data

This one surprised me, because I didn't used to think of analytics as an engineering skill. But as we automate more of the code-generation and review process, we generate more and more data we can actually look at, and there's a wealth of signal in it. This is what enables us to find invariants and measure what’s important.

New signals that we can study:

• Agent logs are fascinating. They show where agents are spinning and getting confused. You can see where it wastes time. For example, are agents struggling with a particular API, are they reading files they should, are they puzzled about what we want?
• PR comments are another signal. Do comments generalise into invariants that could be applied widely? What types of errors are being caught?
• Analysis of the code base itself. Is the complexity of the codebase worsening with spaghetti code importing from everything and god files growing to thousands of lines? Does mutation testing find tests that aren't earning their keep, and gaps in coverage that matter? Are there multiple ways of doing the same thing that could be converged?

Then you turn all of that into hypotheses about where agents are going wrong, and then look for how you could address that – giving the agents better tools, refactoring code, adding skills, defining new invariants and enforcing them so you never have to worry about that failure mode again.

Sometimes that's a human grabbing the data and staring at it; sometimes it's an automated loop where an agent regularly runs the analysis and finds incremental improvements. Either way, it’s a seed for discovering how well the system is working and where improvements could be best made.

3. Risk and operations: grade the blast radius

As we build trust in review mechanisms, then we need to reduce the human review burden. We need to reason about the risk we’re introducing by relying on our processes and how that aligns with the sensitivity of the code base.

At Tessl we use a kind of ladder. Our research codebase is a genuine free-for-all, merge anything you like, no rules. Some of our internal tooling is auto-merged; hundreds of changes flow through, with purely automated review, and nobody looks at any of them. Other parts of the codebase absolutely need an engineer, the person who owns the feature has to take it through and do the final merge themselves. And a few very high-leverage parts should make you stop and go ask someone who's thought hard about them, rather than ramming a change in.

The work is figuring out which parts of the codebase sit at which level of risk, and then codifying that into the approvals flow.

The harness engineer

Put those three together, invariants, analytics, risk, and you get harness engineering. For a lot of organisations this will live in a few specialists who really embody it. But I suspect that for many of us working in engineering, it simply becomes more and more of where we spend our time. That's a real shift in identity: we become tenders and gardeners of our code and our systems, so that when we build features we can focus on whether the behaviour is what we want and trust that the implementation will be consistent and scalable.

Spending your hours on invariants and analytics and risk policy, rather than only on the next feature, feels different, but it has tremendous leverage.

We've leaned all the way into this. Around 90% of our code went through the factory this week, and we've been over 80% for the last month. The shape is consistent: more of the coding moves into remote execution loaded up with skills and context; a layer of guardrails (deterministic checks, LLM verifiers, CI) run through a detailed checklist of rules; automated code review handles new problems and feature specific issues; a triage/gating step reasons about how much risk and care a given change needs; and update loops monitor what's happening and feed learnings back into every layer of the stack.

Where to start

If this isn't your full-time job yet, here's where I'd begin, three small exercises you can try this week.

1. Build a dataset. Agents are great at this, tell one to grab your last 50 PRs and summarise what kept going wrong: what people repeatedly said in comments, what errors recurred, where things tripped CI. Identify issues that generalise and might come up again.
2. Distill three invariants for code you work on. Work out what rule would clarify the behaviour you want - in architecture, design, behaviour, or library usage for example.
3. Turn one invariant into a CI gate, and then make sure you have a way to track what happens with it and watch it. Did it start catching the problem you cared about? How often is it firing? Try implementing it using AST-grep (if it can be expressed deterministically) or try out tessl verifiers (if you want something broader).

For any of these, we’ve loaded guidance into tessl agent, so you can ask it to help you work out where to start and see if you agree with it. Install tessl and run tessl agent to get going. The free tier should be plenty for building out your first invariants.

Here are some examples of rules we have implemented at Tessl to help you get started.

Agents gave us rocket boots. Harness engineering, systems thinking, analytics, and risk, is how we keep from flying straight into a wall. And I think it's about to be some of the most valuable engineering work any of us can do.