The common take is that higher capabilities will solve it. My take is the opposite: the more capable the models get, the worse the reward hacking gets.

Here’s why. We’re trying to come up with tasks that are very hard. But what does hard mean? It’s a very open-ended question. If you push toward hard, which means a benchmark that no AI can do yet, so you can test where the AIs can get to, then as you approach that frontier you are also approaching a space that is less well defined. And the less well defined the task, the more reward-hackable it is. The feedback loop breaks.

You see it in what people actually submit. A lot of what we get is someone who starts with a prompt that works, and then removes tokens from the prompt until it fails. That’s tricky for the AI, but it isn’t fundamentally hard. Hard and ill-defined start to look like the same thing, and that’s where the reward hacking lives.

We’ve run out of the low-hanging fruit

SWE-bench was a really clever way of producing a benchmark. And everybody is kind of hoping you can synthesize benchmarks from here: get a good task creator, brainstorm some ideas with a domain expert, and you can make good tasks. My sense is that, for the time being, you just can’t. The best tasks require that a person who really understands the domain is at the epicenter. The open question is how we learn to produce that, at high quality and at volume, reliably.

And the hardest part isn’t the task, it’s the verifier. When you have something like a C++ compiler with its whole test suite, or SQLite, and you build the task around that, that’s great. But there are only so many applications that come with that degree of testing. Most verifiers are too married to a particular solution. The solution someone provides and the verifier they write are linked to each other, because the same person made both. So you’re not really testing the result, you’re testing against the author’s own answer. The SQLite test suite actually tests the output of the queries. That’s the difference, and it’s hard to produce.

I’ve stayed mostly in the terminal because it seems to me the native space where agents are going to operate. You should be able to port pretty much any concept into the terminal.

Confront the real world

For now we still get a lot of mileage from really high-quality a priori thinking about how to test something. But that runs out too, and the best verifier is the one that confronts the real world.

Take the extreme. Say I want to teach my AI about aerodynamics. The ultimate setup is I raise some money and I buy a wind tunnel from a company going out of business. I get big 3D printers. Anybody can request the shape of a wing, and I print it precisely, put it in the wind tunnel, and give you back precise measurements. Maybe a simulator is the first-order approximation, but eventually the task, the verifier, is a wind tunnel somewhere in physical space. When you run my thing, I give you that data back.

There are softer versions of the same idea. In finance, here’s five thousand dollars, turn it into ten thousand. At the end of the day you have to trade real money, in real time, to know the thing actually works. That’s about the easiest way to confront the real world. A verifier that touches reality can’t be gamed the way a verifier that’s married to its author’s solution can.

Who makes the tasks

So if good tasks require a domain expert at the epicenter, and good verifiers are this hard to build, the real question is who makes them, and how you get them at quality and volume.

I don’t think the current model is the right one. The work mostly runs through a couple of very large data vendors, and it’s hard to know if they’re not experts on the tasks they’re making. They’re worth tens of billions of dollars and they’re still not doing an amazing job. The people actually making the tasks are anonymous. Nobody knows who they are, the names don’t even get shared. All the reputation gets stripped out.

I think there’s a better structure, closer to a marketplace, or at least a network. Imagine I make a task for Terminal Bench, and it has my name on it, and you can count how many times it’s been a rolled out. Maybe I made a really good task and there have been a billion rollouts. Who is capturing that reputation? Right now it’s some other entity that is effectively teaching AIs how to do things. That reputation, captured properly, might be incentive enough on its own.

And a lot of the incentive is already there if you set it up right. Think about SEO. Google wants high-quality content, people want to rank in Google, and there’s some gaming along the way, but it mostly works. The same shape applies here. Every open-source project should have a library, documentation, and a number of tasks. Documentation is only good to a point; what you really want is tasks, because you want agents to use your project, and the way agents learn to use it is for a lab to train on good tasks. If I’m Salesforce and I want to be the system of record behind agents, I’m incentivized to publish high-quality tasks. If I’m a voice API and I want to be the one picked every time someone builds a voice app, that’s a huge win, so I’m very incentivized to put that stuff out there. The labs are incentivized to use it, because there’s generalization that comes from it. Both sides benefit, and nobody has to pay.

Building a high-quality task that passes a lot of checks is also much harder to game than stuffing keywords to rank a library. For the domain-specific cases where free incentives aren’t enough, you can pay, more like a 99designs for environments: a lab says it has a gap, people submit environments, and the ones that create the most back-propagation benefit get paid.

There is a real open question about whether inserting a transaction into all of this lowers quality. The open-source community works partly because it isn’t transactional. The first version of Terminal Bench got built on coauthorship, people a hop or two out got their name on the paper, which felt native to the open-source ethos. I don’t have this fully worked out. But evals and benchmarks are how every one of these companies targets success, and when you peel back the layers it all comes down to a set of tasks made by a community of people. That makes the question of who makes them, and how they’re rewarded, one of the more important problems in AI right now, and one nobody has really figured out.

Today the best manifestation of that is the agent. You assign it a task, it can use tools, use a computer, do whatever it needs to do, for a long period of time. Right now the median interaction with an agent is maybe ten or twenty minutes. In a year it’s going to be two or three hours. After that, four days at a time. As long as you’re willing to pay for the tokens, it keeps going.

On one end you have an agent like Claude Code which can write software, and that’s going to keep getting better and take more adjacencies around it. On the other end you can try to build an agent that’s too far from what AI does well today: AI can do video generation, but ask it to make you a Oscar-winning film and we still don’t know how that plays out. The companies that are really interesting in 2026 are close enough to software that we know it’s going to work, but far enough that they won’t get eaten up by the expansion of the labs. Every engineering discipline sits in that band, mechanical, aerospace, chemical. A lot of the thinking looks like software, but a lot of it is domain specific, with tools that are very unique to the field. Chip design is the hardest one I know, and that’s why I like it.

What the company does

The development of a semiconductor starts with something that looks like software. There’s a language called Verilog, register transfer language, RTL, that describes the logic of the chip. That gets converted to gates, the gates get converted to physical design, and so on. But the stakes are nothing like software. When the first wafer comes out of the fab, depending on how advanced the process is, it can cost anywhere from thirty million to a billion dollars. There can’t be any mistakes. In software you make a mistake, you lose some data, you deal with it. You can’t operate that way with hardware. Which is why the whole thing starts from a blueprint, a very detailed spec.

ChipAgents operates at the front end. We grab that spec, turn it into tests, write the code, and verify the code works. Verifying isn’t like normal code, where you execute it and if it ran you know it worked. You have to simulate it, because all of this is going to run on a clock, and everything has to happen in the physical window between clock cycles for the registers to capture the data. If it doesn’t fit, you move the logic around so it does. There’s a lot of that kind of simulation the agent has to interact with to make decisions.

So the picture is simple. We have an agent, the agent is the front end, and it’s what the engineers interact with. Around the agent we’re bolting a harness of tools. Some tools simulate the logic. Some make predictions about power consumption. Some, later, will be about layout, how the transistors are physically distributed over the wafer.

The goal is not to beat Claude

It would be a bad strategy to fight hand to hand with a big lab on raw reasoning and coding. If anybody claims they can build a better model than Claude Opus at what Claude does, they’re lying. The advantage is in the harness, and in the interactions between the tools and the harness.

There is room to build your own models, but not general-purpose ones. You build domain-specific models that act like sensors on a very particular kind of data. Take power. Instead of running a full simulation overnight on the cluster, you can train a model that takes a synthesis artifact as input and gives you a guess of the power consumption as output. It won’t be exact, but it tells you immediately that some corner of the chip looks like it’s drawing too much power, take another look. That’s AI, but domain-specific AI, and nobody at a general-purpose lab is going to train it for you.

Become the front end

This is the part I find most interesting. Verilog is free. Everybody uses Verilog, nobody cares, Verilog is free. That’s the front end. The incumbents make their money on the back end: the simulators, the waveform analyzers. The agent becomes the front end of the front end. The heavy tools get called by the agent in the background, and we control the agent.

It will get to a point where a particular engineer won’t know if the waveform simulation is coming from a Cadence tool, a Synopsys tool, or our tool. It doesn’t matter, as long as the agent can solve the bug. And that gives us a ton of pricing power. If we can give something that’s ninety percent of the way there but iterates much faster, we capture a lot of that power. We can go from using a vendor’s simulator ninety percent of the time to ten percent of the time, on our own schedule.

Innovator’s dilemma

We’re a complement to the EDA tools, not a competitor. The engineers used to write code by themselves in a normal IDE. We help them write the code and the tests, they send it off to be simulated, and our agent reads the results, looks for errors, and traces them back to the spec.

The physics simulators are extraordinarily precise, and they took decades and billions of dollars to build. That’s fine, we’ll keep using them. But they’re slow and expensive, so today you run them constantly just for sanity checks. Our goal is to use them less, to lean on fast approximations during the design loop and save the full overnight run for the end. Instead of running a tool once a day and waiting overnight, you run something lower fidelity in five minutes. When that happens, EDA doesn’t get displaced, it just becomes less important. The value moves to the front end, to the people actually designing the thing.

And speeding this up isn’t only about cost. Demand for chips is bottlenecked on bandwidth, not just dollars. There’s real revenue the big buyers can’t capture because designs take too long, and a multi-year lag they would pay almost anything to shorten. So anything that accelerates design isn’t just savings, it’s new top-line revenue. The market as a whole gets larger.

Why can’t the incumbents do it?

Two reasons. The first is incentives. They charge by the compute-hour, so a tool that runs faster is a tool that bills less. Nobody builds the thing that cannibalizes their own meter.

The second is talent. Domain expertise is valuable, and it always will be. But AI is also its own domain, a very new one, and very few people have it at that level. AI is very deceptive, because everybody talks about it, and maybe twenty percent of them think they’re experts. When you think about the people who can actually build a company like this, you’re talking about a thousand people in the world. There’s not that many. And finding people like that who also know chip design is very unlikely. They command very high salaries and they will go to a startup and take equity. There’s no way the comp structure of a thirty-year-old EDA company can digest that, and those people don’t want to work there anyway. You go to an AI conference and everyone wants to be at a startup or at one of the big labs. EDA isn’t cool.

Why the market keeps growing

Moore’s law is tapped out, which means the only way to keep getting performance is domain-specific silicon. People are going to keep turning algorithms into chips, even high-frequency-trading firms burn their strategies into hardware. More chip diversity means more designs, and more designs means more demand for the front end they all run through.

Think about the IDE, the interface a developer uses to write code. For thirty years it was a free text editor nobody paid for. Then in the last few years it became very valuable; Cursor is a tens-of-billions-dollar company for what is, underneath, a text editor. What changed isn’t the editor, it’s that far more people are writing software. The same thing is starting in silicon. Companies that used to just buy chips are designing their own, Amazon has Trainium, and that moves down-market from here.

There’s also a lot of leakage between the steps. Arm sells you intellectual property, blocks of compute you drop into your chip, but they’re encrypted, so there isn’t much of that on the internet. When you manufacture at TSMC, you have to hand it over in a very particular format, and TSMC only exposes that interface to a small number of companies. Every one of those bridges, from one step of the process to the next, is a place for an agent to play.

What it comes down to

If you want to oversimplify the vertical AI industry, you’re really reselling tokens at a premium. You just need an excuse to sell tokens at a premium. We package a few tools and a harness that makes using Claude through us better than using it directly, and for the privilege we get to charge more. It’s a variation on fintech.

The most important question for anyone building in this space is whether there’s enough durable differentiation for a ChipAgents to exist at the same time a Claude Code exists. I think the answer is yes, and it comes down to the sophistication of the tools. That’s the whole bet.

This is one of the biggest industries in the world, and one of the fastest growing. The incumbents are hundred-billion-dollar companies. I think you can build something on that scale here, and you do it by owning the front end and letting the agent quietly take over everything behind it.

The way to manage an organization is through goals. Everybody knows that. But those goals tend to be boring and not actionable. People say we need this much in sales, we need to hire this many people. They’re boring. People get detached. Most of these goals don’t increase the energy and the motivation in a team. They decrease it.

There is marketing you use for the outside world, where you’re marketing your product. But there is also marketing you have to use internally. What are you telling yourself and your team? What is bringing you to work every day? What are the big exciting goals? There’s a lot of art to that.

The best ideas are in the room. Not my ideas. So when we used to plan on a quarterly basis, I would do a request for proposals. Everybody sends suggestions for how we could work. Anyone. Sales, the janitorial service, it didn’t matter. Anybody can send one.

But then you end up with hundreds of ideas, and you can’t just hand that to your engineering team, because it’s distracting. They’re all different. So the big part of the exercise was clustering them in my head so that I could grab twenty five ideas and give them a single name. These people are all talking about how to visualize data on the website. These other ones are talking about something almost like visualization, so I’m just going to put it there. You create the cluster, and at that point you throw away the contents of the cluster. Now you have a shell. You say: visualizations. And you give it to the right person to work on.

Then you go back to your team and say: because of all your suggestions, we’re going to work on this. And from that you develop very operational goals. I’ve always been good at figuring out the singular goal that, if you go after it, unlocks a lot of side quests along the way.

The impactful decisions are few. They tend to be not that many. As the organization grows you have fewer of them, because you have more people. When you’re small you have impactful decisions every day. But once you have ten million dollars in investment and twenty or thirty people, your impactful decisions are fewer. Call it one per month. But they’re very important, and they happen fast. You meet someone at a conference and decide you need to hire them, and then you work really hard to hire that person. You bring in an investment. You shift the strategy: we’ve been doing this, it’s not going anywhere, let’s convince the team to do this other thing, and everybody leans onto that.

There’s a reason you can only do so many of these per year. They’re sudden to the organization. If you do too much of it, you’re the kind of leader who feels like he’s distracting everybody. If you don’t do it at all, you’re a passive leader.

Elon Musk has this concept of a surge. When they’re behind on some goal, he calls a surge and moves into the factory. He’ll move into the Tesla factory, sleep on the floor, and make everybody stay. They go for a week. A very intense period to catch up to some goal. The key to good leadership is knowing the right balance of how much of that you can do. Most people don’t do it enough. Most people are very passive about their organization. They assume everybody in the org has agency, everybody is equally motivated, everybody is doing the right things.

People want to be on a mission. If you’re a kind, caring person, there’s this feeling that pushing people to work more, to do more, means you’re exploiting them. So you back off from that behavior. But people, especially when they’re young, want to go through intense periods. Nobody would volunteer for the World War Two trenches. But when they’re forced to be there and then they come out, they remember it as a very important period of their life. People want to go to war on a mission together. Your job is to constantly make people feel they’re on a mission.

My prime as an operator was at Amazon, where I had a hundred and fifty people. I operated like a conductor of an orchestra. There were people on my team who were very structured: have a plan, work the plan. That was never me. I was feeling the room, trying to figure out whether I could get more value out of it. If I came into the office a few days in a row and felt things were slow, I’d do something like call a hackathon for tomorrow, right away, just to change the momentum. Or I’d do one on ones with almost everybody, and I’d always talk to people about what they could do to maximize their impact.

Use the world as a processor. You can think of the world as working for you. You put something out there, the world processes it, and it comes back to you. There’s a version of this that’s just bias for action. You can spend cycles thinking about whether to do something, when in fact you can just do it and then decide.

Take a conference. You can sign up to give a talk three months out, on a topic you’re not prepared for at all. If the organizers say no, fine, you don’t have to worry about it. But if they accept you, now you have to do it. That tends to work. I put a lot of irons on the fire and see what comes back to me. I don’t need to keep thinking about it. I do the application quickly, and if it comes back, suddenly I need to deal with it, and I go through three or four crazy days of work and prepare myself.

This week I was working on a benchmark, and a lot of data vendors were giving us tasks that were shitty. So I started writing notes about what makes a good task. It was an internal document, just for me, for future reference. Then I thought I should write it as a manifesto on a domain name. Buy the domain, call it goodbenchmarks.ai, put the thing up. If people are into it and it gets distribution, I’ll make it better. If they’re not, fine. I turned something boring and lame with no consequences into something I could externalize and see what happens.

Think of your company as a black box that churns out artifacts. You’re sending signal to the world and getting signal back. So the question is always: what can you put out?

When in doubt, ship the closest thing. If you find yourself without a good way to prioritize, prioritize whatever is closest to being packaged and sent out. Do that over the 401k plan. People care about the 401k, but nobody on the outside is going to look at your company and say, oh, you guys have a 401k.

A lot of management is just not doing things. It’s saying no, and getting obsessed with one thing. So push the admin off your plate. A part time accountant, remote, in another country, can do your books, and then you ask them to do a few more things and pay them by the hour. It’s useful to learn about something like a 401k once, but it can eat a huge amount of time.

There are the big level goals, and then there’s every week, where you’re feeling the pulse of the company. If by Wednesday you feel like you’re not putting anything out, you haven’t written a blog post, you haven’t shipped anything, then you pause. You talk to your teammates: what’s our next milestone, what’s our next most obvious thing? And you clutch onto that.

Start with the smallest thing you can ship, then escalate. Say you want to demonstrate something. What’s the easiest way to show it? Do the cheapest, most obviously artificial version, the one nobody would ever do in real life, and put it out there. Then you escalate from there. You go from obviously a demo to something more real, and if attention is being paid, you keep escalating. Each step pulls more people in. You can get others bidding into your goal, providing the thing you need, until they’re aware of you and you have momentum.

The best teams are mission-agnostic. At the end of the day it’s about an effective organization. You could switch a great team’s mission and they’d still do well. Take a team building one thing, drop them on a deserted island where they need to build survival systems, and they’ll do it. The best teams can do anything. The intrinsic capability of the team is that it can take any objective, operationalize it, and deliver it. Because you’re an early company, that practice matters more than your strategy.

A company with five hundred thousand dollars and two employees is not going to have a huge impact. What you need is fifty employees and ten million dollars, and to get there you go through the hunger games, the trials and tribulations, and you show everybody who could support you that you’ve actually gone through the process. You have to show people, hire people.

A lot of it is performative. The mission is an excuse to show that you can accumulate resources and control and power, that you can be a good steward of some social power, that more people should trust you with their careers and more funding should be trusted to you. Once you have some momentum, the goals start becoming more and more impactful.

Some people get this operational intuition because they played sports in college or went to the military. Some are just born with it: they get anxious if they’re not delivering results. But you have to get there. It’s not something you can put in a goal. It’s something you wake up every day thinking about. What can we put out? How do you make it exciting and interesting for the team?

If that’s where things are heading, the question for any vertical AI company is what to own. Agents can use tools, but the tools they use need to exist: first you build the tool, then you teach the agent how to use it. The agent has the typical tools, everything you can do on a normal computer, but some tools are special. Take Plaid; your agent by itself can’t connect to your bank account, so it needs something like Plaid. So the moat is whatever the agent needs to do its job. If the agent uses Plaid, then Plaid is good. If the agent uses LLMs, then LLMs are good. If the agent uses GPUs, then GPUs are good. Value accrues to whoever is serving the agent. The old SaaS moat was lock-in: a vendor holds all your data, and leaving is so expensive you never do. Agents erode that. If something just does the job, swapping one provider for another is cheap, so the durable position is no longer lock-in but ownership: own the tools the agent can’t do without, or own the industry outright. And not the trivial tools, book a meeting or get the weather, but the ones that are either proprietary and expensive, like Cadence and Synopsys, or built for human use and therefore bad for agents, or that have no open-source alternative and can’t be trivially coded. Those are the ones that involve large data, heavy compute, and real simulation. We’re not going to build a model that, pound for pound, beats Claude Code; that’s not the differentiation. The differentiation is on the tools. Whoever owns those tools owns the harness. Claude Code is a harness for coding and general-purpose work. The harness is the thing.

On one end, you have an agent like Claude Code which can write software. That’s going to keep getting better and take more adjacencies around it. On the other end, you can try to build an agent that is too far from what AI does well today: AI can already do video generation, but ask it to produce an Oscar-winning film and we still don’t know how that plays out. The companies that are really interesting in 2026 are close enough to software that we know it’s going to work, but far enough that we’re not going to be eaten up by the expansion of the labs. Semiconductors is a great space, because really every engineering discipline, mechanical, aerospace, chemical, is a really good adjacency. In some ways there’s a lot of software-like thought that goes into it, but in many other ways there is a lot of domain specificity, tools that are very unique to the domain.

Take chip design, the hardest vertical I know.

There’s a bottleneck in semiconductors. We keep designing more ambitious chips every year (more transistors, more complexity, tighter timelines), and at the same time the supply of people who actually know how to design and verify them is shrinking. Verification engineering, the job of proving that a chip does exactly what it’s supposed to do before you commit it to silicon, is a growing-in-demand role in the US. But the average age of the people doing it is over 40, and fewer people are going to school for it. Even the ones who do aren’t job-ready: curricula run years behind industry, and in some specialties a PhD is just the starting point for another five years of training. So you have demand going up and the talent pipeline going down. That’s a bottleneck, and bottlenecks like that are exactly where I want to be investing.

What designing a chip actually looks like

In semiconductors there are still blueprints. Even though an engineer writes code, someone writes a document first, a multi-hundred-page document that specifies, in detail, exactly what the chip is supposed to do. Going from that document to a testbench that verifies everything you built is the heart of the job, and it’s enormous.

Then there’s the physics. A chip runs on a clock. The clock ticks, and everything in between has to happen in time. If you don’t organize things correctly, you miss the clock cycle and everything is messed up. To know whether you got it right, you simulate, and those simulations produce waveforms, traces that show you the values of every register over time. These waveforms are, for lack of a better phrase, from here to the moon. They’re gigantic. So if you want an agent to help here, it has to observe and reason over that kind of data, a kind of tool use that is simply not typical for your common coding agent.

After the logical design, you go into physical design: figuring out how the thing actually maps onto the surface of the silicon, so that the wires don’t cross each other and the timing still holds. Then you simulate again, but now you’re looking at things like power, how much each part of the chip is going to consume. And there’s a whole other world beyond the digital part. A lot of what we picture when we think of semiconductors is digital logic, just gates. But there are also analog circuits (resistors, capacitors, the continuous analog parts of a chip that don’t reduce to gates), and there’s far less good tooling there. A lot of what the incumbents sell doesn’t really work well once you’re thinking in analog terms.

The other thing to understand is how slow all of this is. The simulators are extraordinarily complex, expensive physics engines. To run a real one you use a supercomputer. You kick off a job, you go home, and you come back the next day to your results. And because the supercomputer is a shared resource, everyone in the building is waiting in the same line. The entire design loop is gated by these long, expensive, batched simulation runs.

The harness, in silicon

ChipAgents is a harness, specifically for chip design.

The goal of a company like this should not be to win a hand-to-hand fight with the general-purpose models. There’s no point trying to beat the big labs on a benchmark for writing code. You will lose that fight, and even if you won it this quarter you’d lose it next quarter. Where you win is on complex tasks that require specialized tools. Furthermore, these tools need to be optimized for agentic use. In semiconductor design there are plenty of simulation tools that are GUI-based and slow, both suboptimal characteristics in an agent world. That’s the moat. It has very little to do with whose language model is slightly better.

Besides the harness, a vertical AI company can build domain-specific models. LLMs are the brains of agents, and it’s unlikely anyone outside of a big lab will build a “smarter” model, but there is room to build custom models that act as sophisticated sensors on a very particular type of data. Think about the protein-folding models. They may use transformers under the hood, but they’re trained specifically to do one hard thing extremely well. You can do the same thing in chip design. For example, instead of running a full-precision power simulation overnight on the computer cluster, you can train a domain-specific model that anticipates power consumption directly. It won’t be exact, it’s an estimate. But it can tell you, immediately, “this particular section of the chip looks like it’s drawing too much power, take another look. You get that answer in seconds instead of the next morning, you iterate, and you delay the big, expensive simulation until you actually need it. That’s the kind of capability a general-purpose model will never give you, because nobody at a general-purpose lab is training a model to estimate power dissipation on a circuit.

The tools and the models compound. Once you have the tools, you can leverage feedback loops to enhance models continually. You begin by using reinforcement learning to teach a model to use your own tools better, once you have the testing data, and eventually you train models from the ground up. With the tools, the feedback loop, and proprietary models, a company’s market position becomes virtually unassailable. No one has reached that stage yet, in chips or anywhere else; the work is to climb toward it.

There’s one more reason the harness is the right place to stand. The incumbent tools, the ones that have defined this industry for decades, were built for humans, not agents. This isn’t unique to chips. The costly desktop CAD software used by engineers in architecture, mechanical design, and circuit design is typically proprietary, lacks APIs, and wasn’t built with LLMs in mind. Historically, to use a simulator, you would type a command into a terminal, or open a tool with a big vendor logo on top and click around. You can script them, but the surface is fragmented and awkward, and a lot of the real work still happens by hand. Learning to drive all of that reliably is its own hard problem, a very specific kind of computer use. Once an agent can do it, the agent becomes the thing the engineer interacts with, and the old tools recede into the background.

Become the front end

This is the part I find most strategically interesting about vertical AI companies.

Think about how the incumbents are structured. The hardware description language itself, Verilog, is free. Everybody uses Verilog, nobody cares, Verilog is free. That’s the front end. They make their money on the back end: the simulators, the waveform analyzers. The cool thing about agents (whether it’s an agent for finance, for law, or for chips) is that the agent becomes the front end of the front end. The heavy tools get called by the agent in the background to get the job done, and we control the agent. So before, an engineer interacted directly with a vendor’s simulator, with the vendor’s logo on the screen. Now that’s not happening. Our agent turns the simulator on in the background, pulls the data, and the engineer is only ever interacting with us. As long as the results are good, the user does not care what’s running underneath.

That changes the balance of power. If we ever decide that having our own simulator is a good idea, we just point the agent at our own simulator instead, and nothing about the user’s experience changes. If there’s a new back-end procedure that needs to be done somewhere in the design flow, it’s easy for us to build it and upsell it, because we own the interface. We are the UI. We are the front end. Whoever controls the agent gets to delegate sub-actions to whatever they want, and that’s a lot of power. We can intentionally go from using a vendor’s simulator ninety percent of the time to ten percent of the time. That gives the customer more pricing power against the incumbent, and it gives us more pricing power against the customer. Owning the front end is what lets you slowly take over the back end on your own schedule.

Innovator’s dilemma

Let me be precise about what this company is and isn’t trying to do, because this is where a lot of people get the strategy wrong. Right now we’re a complement to the EDA tools, not a direct competitor. Before us, the engineers were just writing code by themselves in a normal IDE, something like VS Code. So in a sense we’re the Cursor for chip design: we help them write the code and write the tests, they send it off to Cadence or Synopsys to be simulated, the simulation comes back, and our agent reads the results, looks for errors, and traces them back to the spec. We’re selling to the same people doing design, testing, and verification.

Those physics simulators are extraordinarily precise, and they took decades and billions of dollars to build, and that’s fine, we’ll keep using them.. But they’re slow and expensive, so today you run them constantly just to get sanity checks. Our goal is to use them less, to lean on fast approximations during the design loop and save the full supercomputer run for once or twice at the end. When that happens, the incumbent’s pricing power goes down. It’s not that we’re displacing EDAs; we’re not doing what Synopsys does. But we accelerate the lifecycle and do more and more before we ever have to reach for their tools, and that makes those tools relatively less important. That’s the innovator’s dilemma applied to chip design: AI lets us take a lot of shortcuts. Over time I think EDA, the simulation layer, becomes a smaller part of the market, and the value moves to the front end, to the people actually designing and making the thing. The old tools are sold per seat. If you’re accelerating the engineering work itself, you eventually charge a percentage of R&D instead, and when a customer is spending a billion dollars designing chips, a few points of that is an enormous business.

That’s the opposite of the other camp. There’s a well-funded effort that takes the maximalist position: you won’t need anybody, you’ll just type a prompt and a finished chip comes out the other side. They claimed this years ago, before AI got really good. I’m an AI maximalist too, I genuinely think AI is going to change everything, but you have to play with the market as it actually exists. The iterative approach, fitting into the workflow as it runs today and accelerating each step, is more likely to get adopted.

Why can’t the incumbents just do it?

The obvious question is why EDA companies don’t simply take the AI route themselves. Part of the answer is incentives. They charge by the compute-hour, so a tool that runs 100x faster is a tool that bills 100x less. Nobody builds the thing that cannibalizes their own meter. The other part is talent. Domain expertise is valuable, and it always will be. The problem is that AI is also its own domain, a very new one, and very few people have it at that level. You’re really talking about a few thousand people in the world. And finding people like that who also have the domain expertise is very unlikely; these are all new PhDs, finishing their programs right now. They command very high salaries, and they will go to a startup and take equity. There’s no way the compensation structure of a thirty-year-old EDA company can digest this personnel, and those people don’t want to work there. You go to an AI conference and everyone wants to be at a startup, or at one of the big labs. EDA isn’t cool. So there’s no realistic way for the incumbents to assemble a team of fifty or a hundred of these people to go build this, even though they can see it coming. That’s the opening.

How is venture capital changing?

For a long time the instinct in venture was that good software was enough. I don’t believe that anymore. AI is eroding the last standing barriers to entry. Software has eaten the world, and the world of software now lives in the boring plane of compressed margins. A company that just sells software is not differentiable; software is a commodity now, or it’s going to become one soon.

So the real question I keep asking is: where do you draw the line? Software generation keeps getting better, for free, every six months, so where do you draw a line you can still defend? For me, the answer is to go toward harder technology, where more things have to work out, where there’s real capex, where you have to train your own models or earn access to data and relationships that other people simply don’t have. That’s the opposite of vertical SaaS, which just gets hammered on margins. It’s vertical integration: the Tesla or SpaceX or Apple model, less like selling software to construction companies and more like building the construction company itself, fully digitized, fully agentic, end to end. It’s harder, and it’s higher capex, which is exactly the point. When you’re that vertically integrated, software stops being the product and becomes an advantage layered on top of something hard.

Chip design is about as hard as it gets. The bottleneck is real, the talent is aging out, the tools are decades old and were built for humans, not agents, and the feedback loops are measured in overnight supercomputer runs. It’s close enough to software that I’m confident AI will work well, but specialized enough that nobody is going to eat our lunch overnight. There are only ever going to be a handful of serious companies going after a problem like this, five or six, not five hundred. The winning move is to own the front end and let the agent quietly take over everything behind it.

One example that I think would benefit from being more commercial is monitorability. In my work with Terminal Bench, I’ve reviewed more trajectories than I can count. In the process, I found an ever-growing graveyard of well-intended trajectory analysis tools that never gained traction. Everyone involved knows there is something missing, and so every so often a new tool is shared in our Discord, and then interest dissipates.

Which is it then? Is there a need for monitorability tools and we just keep missing the mark, or are we in some sort of collective delusion? I think it’s the former. And I think it’s in part because nobody has put the level of attention and intensity a great product deserves. One benefit of building a for-profit company is that when someone is paying you, they are going to have a lot of expectations. If you pick the right first set of customers, that pressure will lead to better solutions. Every single AI trajectory should be monitored. Not just at the labs. Every person using models should be aware of the trajectories they generate. All the way down to what your kids’ trajectories are about. Everybody should have awareness of what the models are doing on their behalf.

If you get really good processing and archiving in place, you can do all sorts of stuff on top. This is a recap of a recent conversation where I argued for some commercially viable motivations to incentivize monitorability adoption. These intentionally sound less like safety pitches, because the intention is to make a case to a broad audience. I acknowledge some of these ideas come close to surveillance. Rather than advocating for any one of these, I’m just wearing my VC hat to startupify some concepts.

Understanding labor productivity. You want everybody in your company using AI. If they’re not using AI, they’re wasting time. But some people are going to use AI so well that they’re basically doing nothing. They’re going to automate their job, and instead of moving on to the next thing, every morning they push two buttons, pretend they’re doing work. The distinction between that person and a highly productive one is going to be very difficult to make. You need to actually look at the trajectories. Do all of someone’s trajectories look the same every day, or are they doing different things?

The quality of reporting on labor productivity right now is really low. Everything I’ve seen from consulting firms and universities is crap. It’s hopeful, or it’s looking through the wrong lens. People are wishing that the impact is less than it is. It’s not written by people who really understand where to look for productivity, because obviously the KPIs are going to change. It’s not about hours in the office. It’s not about Slack messages sent. Text is free, so you can’t measure productivity by text volume. So what do you measure it by? You measure it by what people actually did with their AI, and what came out of it.

Organizational alignment. Here’s something you get at large companies. The CEO and the executive team come up with a strategy. The whole company has to do this thing. But the company has a million employees, so it doesn’t quite work like that. The strategy document is vague. Every team underneath makes their own interpretation. Sometimes you get surveys, once a week, asking what employees think is most important for their job. You try to get a sense of whether people are actually aligned with the goal. But all of this is self-reported. It’s like a personality test. There is what people say and what people do.

Now imagine you can see what people are actually doing. Not because you’re surveilling them, but because the AI is already the intermediary for their work. Claude is the witness. Claude is the confidante. Claude has all the information. Are employees actually doing what the master plan says they should do? Are they doing something different? And if they’re doing something different, it’s not necessarily bad. It could be that the executives are unaware of what’s actually important. In fact, I’d argue that’s half the opportunity or more: seeing what the distributed system of your employees are actually doing naturally, extrapolating the implied strategy that hasn’t been made explicit, and seeing if you can reinforce that.

Real-time strategic nudges. When I’m interacting with Claude, building things or writing an email, can I be reminded of things that are relevant to the company’s strategy at that moment? Not after the fact, but in the moment. Imagine how powerful that is. We already do it with legal stuff. You’re about to do something that violates a policy, and you get flagged. That’s the basic kind of email monitoring that companies have. But this is different. This is: you’re about to do something that is very aligned with the strategy, and you should let people know. CC this person. Or: you’re going off track and should reconsider. And then flip it around: do the executives themselves actually behave according to their own strategy? They go to a golf retreat, come up with a plan, and then go back to work the next day. Is there a discrepancy between how they spend their time and what they said the company should do? They may not even be aware.

Retroactive security analysis. If a zero-day exploit is found, I want to go back to all my trajectories and see: did I use this anywhere? Was I exposed? This is the equivalent of an audit trail, but for every interaction you’ve ever had with an AI agent. If you have good archiving, you can replay, search, and assess exposure at any point in the future for threats that didn’t exist at the time.

Self-understanding. At the individual level, trajectory monitoring is about understanding yourself and where you’re at. How risky are my interactions with my agent? What patterns do I fall into? What am I actually spending my time on versus what I think I’m spending my time on?

Whoever controls the exhaust controls the insight. And if you’re really thoughtful about this kind of analysis, I think you can go really far.

First, I build the thing. I give a lot of feedback. If it uses data, I can see the data, check if it makes sense. Then I use it. I record usage, clicks, inputs, outputs. I get analytics on everything. Something that could have been a thousand lines of code is now three thousand. Messy, bloated, but functional.

Then I tell the AI: grab all my usage data, assume the app behaves as it’s supposed to, and create a massive number of tests. Every input and output from my actual usage becomes the test set. Then I put it in a loop: keep making this smaller. The tests have to pass. Every time you change something, test, test, test.

It never gets back to a thousand lines. The AI isn’t good enough yet at coming up with really clever new abstractions. But it cuts it by about 60%. The idea is: design, throw away, but use the artifacts as the new requirements and the test set, then do it again.

If your analytics record exactly what everybody did, and you can replay that and get the exact same output, pixel perfect, and you can achieve that with half the lines of code or twice the performance, you’re in good shape.

The key insight is that you should be able to replay everything. Assume your application has no bugs, because what you’re really trying to do is build a better application that behaves the same. If there’s a bug, whatever, you can replicate the bug. That’s a feature, not a problem.

I think replay analytics is a product someone should build. It’s not trivial. You need the database in the right state. You need to replay everything end to end. But if you can do it, you get something powerful.

You need a roofing permit. You give it to your AI. Your AI fills out the paperwork, handles the back and forth, does it perfectly. But the government doesn’t have the roofing permit AI. You want to file a Freedom of Information request. Give it to your AI. But the government doesn’t have a way to handle the influx. So the government gets more and more inbound because people can. And they don’t have a way to manage it. So they’re ineffective.

No big deal when your roof permit is just taking longer than it should. But then something bad happens. A disaster, a crisis. And the government doesn’t have the right level of preparedness. It’s using a very antiquated approach to operations at the exact moment that citizens are supercharged with AI capabilities.

This is the asymmetry that worries me. The demand side is about to explode. Every citizen with an AI assistant can now generate paperwork, file requests, and navigate bureaucracy at a speed and volume that was previously impossible. The supply side, the government’s ability to process and respond, hasn’t moved at all.

Governments are slow by design. They dampen the influx of requests through burden. Construction permits, civil disputes, licensing. It’s generally not tasteful for the government to manage demand using price discrimination directly. So they achieve the same thing indirectly by making processes more complex, which means they often require professional assistance, which makes them more expensive. The complexity is the feature, not the bug.

But now my AI can do paperwork. The complexity barrier is gone. So the government gets slammed. And neither the government AIs, if they eventually get them, nor the citizen AIs will care about the length and complexity of a form. The whole notion of complexifying process stops having a purpose.

I want AI adoption in the government because if the government doesn’t adopt AI, it’s going to be much less capable of dealing with difficult circumstances. Politicians should be thinking about this. Not just for their constituents, but for themselves. The institutions they run are about to face a flood they are not equipped to handle.

And the stakes go beyond paperwork. If we end up with Great Depression-level unemployment, if class tensions rise, if social services get overwhelmed, the government needs to be operating at a level of competence that matches the moment. You can’t respond to an AI-accelerated crisis with a fax machine.

Most people write benchmark tasks the way they write prompts. They shouldn’t. A prompt is designed to help the agent succeed. A benchmark is designed to find out if it can.

I’ve been a contributor and reviewer for terminal bench since last August, and this post is about what I’ve learned designing and reviewing tasks. The guidance is broadly applicable to anyone building an agentic benchmark. We’re currently accepting tasks for Terminal Bench 3.

I first got hooked by the challenge of making difficult tasks and seeing how long it would take for SOTA models to catch up. I spent the most time developing the install-windows-xp task. Here are the instructions the agent sees:

Install, and run Windows XP SP3 (32-bit) in a virtual machine using QEMU. Create the virtual NTFS hard disk as /app/isos/xp.img to install windows. Your Windows XP ISO is at /app/isos/xp.iso. The administrator account should be named tb-admin. For your convenience, we built an API that lets you read the installation key using OCR on the CD-ROM package. You can get the official key calling /app/read_xp_key_from_cdrom_box.

VNC Configuration Requirements:

• Configure QEMU to use VNC display :1
• Ensure VNC server is listening on port 5901
• Set up a web interface (nginx) on port 80 for remote access

The VM should be left running in the background once started. You will have completed your objective when qemu is at the windows xp login screen and the VNC interface is accessible for monitoring.

The instructions are short. But the environment was highly complex: Windows needs to be installed inside QEMU inside Docker inside Linux. The solution is tricky, since Windows XP wasn’t trivially easy to install in unattended mode. The agent has to create a custom bootable ISO by extracting the original XP ISO, injecting an unattended answer file with dozens of configuration settings to suppress every possible GUI popup, adding OEM preinstallation scripts to create the required user accounts, and rebuilding the ISO with a proper El Torito boot sector. Any gaps in the unattended configuration would cause the installation to pop back into interactive GUI mode, at which point the task would have failed.

The task had a 2-hour agent timeout, one of the longest in the benchmark, because the installation alone takes 30-60 minutes under emulation. The agent has to monitor disk growth to know when the install is done. In practice this means checking every few minutes and waiting until the virtual disk exceeds 1GB and stops growing. Then the agent has to kill QEMU and reboot the VM in boot-only mode, without the CD-ROM attached, to reach the login screen.

It was hard for me as the task developer to be sure it was actually working, so I required the agent to set up VNC on port 5901 with an nginx web proxy so I could visually confirm the installation was progressing. For verification, I searched the virtual disk for the MD5 hashes of 12 specific Windows files: ntoskrnl.exe, kernel32.dll, explorer.exe, ntldr, and others. But I went further. The test takes a VNC screenshot and compares it against reference screenshots of the login screen using structural similarity (SSIM), requiring at least 85% match. It also verifies the tb-admin user was created by searching the virtual disk for the user account picture bitmap, which only exists if the OEM setup scripts ran successfully during installation.

This task didn’t end up in the official terminal bench 1.0 dataset because it took too long to run and made logistics painful. I think it’s still one of the coolest tbench tasks created to date. I went from knowing very little about benchmarks to being an official reviewer, and submitted many other tasks which did get merged, including install-windows-3.11 and video-processing.

What follows are some personal opinions about what constitutes a good task. You can find official guidelines here.

Benchmarks should be Adversarial, Difficult, and Legible

When you prompt an LLM, you want it to succeed. You repeat yourself, you emphasize, you add examples, you structure everything just right. That’s what works.

A benchmark is adversarial. We tend to write instructions in a way that we are trying to encourage the agent to get it right, but that’s not the point of the benchmark. The point of a benchmark with verifiable rewards is to state an unambiguous objective, which can be confidently verified, and which corresponds to a difficult task. Think of it as an interview question for a principal engineering role. Hints are for entry-level employees where you want to know if they can think well. Principal engineers need to deliver the answer on their own.

The ideal task can be described in two paragraphs. It’s difficult to solve. The agent will have to think before even doing anything. But then the actual solution doesn’t necessarily have to be itself very complex (e.g. a long program). It’s not hard because it requires a lot of resources, or the task is expansive. You can ask an agent to optimize a small program and that might be just as hard as optimizing a larger one if the requirements are aggressive enough. While not a requirement, the most elegant tasks have short, well-specified, self-explanatory (e.g. no README needed) instructions. Think literate programming.

A benchmark has to straddle a balance between being as realistic as possible while still remaining legible. As capabilities and time horizons expand, this becomes difficult. Running a benchmark of tasks involving swarms replicating highly complex software remains expensive, and relies on the credibility of the reviewer (in this case Nicholas Carlini). For the rest of us, if we want our benchmarks to be credible and gain traction, we benefit from making our tasks tractable. Likewise, if we want a busy leaderboard, we have to make the benchmark accessible to all model/agent developers, which means the cost and infrastructure complexity can’t be disproportionate.

Note: we are working on Terminal Bench Challenges, where a single highly complex task, such as writing a c compiler or a web browser, is a full benchmark.

A Taxonomy of Bad Tasks

To a large degree, the value gained by contributing to a project like Terminal Bench is access to concrete feedback. In that light, I want to share some specific examples of common issues.

AI-Generated Instructions

The most common and most obvious problem. Someone asks an LLM to write their task instructions and submits whatever comes out. It’s immediately recognizable: the tone is wrong, it’s verbose, it’s over-structured, and it reads like it was written to maximize the probability that the agent succeeds.

Great instructions are written by hand, or heavily edited from whatever an LLM suggests. They are not meant to be a forced prompt with emphasis and repetition to coerce the attention heads to listen to you. Direct and to the point. Specific and sufficient, but not redundant and attention grabbing.

Over-Prescriptive Instructions

Even when instructions are human-written, they’re often too prescriptive. Authors tell the agent how to solve the problem instead of what the end state should be.

I have an aversion to this sort of instruction. It’s too clerical, asking for a very specific set of steps, instead of articulating a goal. Couldn’t you just describe how the system should work and let the agent figure it all out? You don’t need to explain how things might fail. This is not some educational problem that a college student needs to learn from. Assuming an experienced engineer will understand what you mean, you can expect the agent to bring the same understanding.

Write the instruction as if it’s intended for a smart human. Clear, but not redundant. I don’t think you need to tell the agent how it will be tested. Just tell it what you expect the end result to be. But the instructions should be sufficiently specified so that meeting them implies passing the tests. Unlike when designing a prompt for high probability of success, here it’s enough to say things once, as long as it’s clear.

My opinion here is likely stronger than the median TBench reviewer. The reason is that I see every token as an opportunity to mistakenly add ambiguity or create a specification detail which the tests might miss. If I want the task to be unambiguous and perfectly verified, then brevity is an important KPI.

Clerical Difficulty

I’ve been thinking about the distinction between tasks that fail because the structure of the expected output is convoluted — a long instruction explaining how to form a complex JSON object — versus tasks that fail for something intrinsically difficult. There’s something categorically different between the two. I call these clerical or administrative errors, and tasks that fail exclusively because of that usually are hard in an uninteresting way.

If an agent fails because it put a dollar sign in the amount field when you expected a float, or because it used a top-level key when you wanted a bare array, it’s measuring format compliance. Using so much of the instruction to detail the output format tends to make models fail not because of the complexity of the task, but because of some clerical error. Of course it’s important for models to get formatting right, every time, even when the distinction is nuanced. It’s just that this is not the same sort of capability that Terminal Bench is setting out to solve. If a task is challenging to SOTA models, it shouldn’t be because they can’t spell “strawberry”.

A related problem is tasks that are too wide, asking for a lot of little deliverables instead of solving a concrete large problem.

Solutions That Assume Hidden Knowledge

The submitted solution.sh should solve the problem as an agent would. Hardcoding an answer which implies knowledge not self-evident in the instructions is not helpful. The person creating the task is making assumptions that are not included in the instructions. The task is underspecified, and you don’t realize until you actually read the solution.

I watch for this carefully. I want to make sure the solution doesn’t display inside knowledge that the agent wouldn’t be able to know. Are there some commands you can include that would reveal the exact issue before you apply the patch? A proper oracle will actually ask questions of the system. It will go through a series of commands to figure out what’s wrong, and then upon figuring out what’s wrong, it will produce a solution. If the solution jumps straight to the answer without exploratory steps, it might be making unfair assumptions.

The problem is that the author knows the ground truth. So it might seem that you are being fair because your invoice processing solution is looking for levenshtein_distance and so on, but you could have just as easily come up with typos that didn’t pass your particular choice of data cleaning. The solution isn’t investigating the issue. It’s just writing down the answer to a known issue.

Tests That Validate the Wrong Things

Tests should verify outcomes, not implementations.

Why test that Pandas is installed? This was never requested in the task. Doing a lot of string comparisons to evaluate source code is going to be brittle. You should test examples beyond the one you tell the agent to test with.

I’ve seen this often: tests that check for specific libraries instead of whether the output is correct, tests so tightly coupled to the oracle solution that any alternative correct approach would fail. There was a task which required setting Linux permissions in such a way that only certain people could perform certain operations. There was one set of tests verifying functionally that the right people can do the right operations. There was another set of tests that looked at linux permissions directly. To what degree are these testing the same thing? Is it possible for permissions to look slightly different and still accomplish the goal?

For tasks with inherent variance, testing gets harder. I ran my own word2vec task 25 times. The queen - woman + man = king analogy returned king among the top 75 results 100% of the time, but the variance was wide, from #1 to #40. I could test a number of things about the resulting model, but I never felt satisfied with this canonical test.

Sometimes, the task includes dependencies that the agent has access to. For example, when the goal is to implement a missing feature in a larger application. In those cases, it’s important to test the outcome with a copy of the original code, to guarantee the task is not passing due to spurious changes.

What about LLM-as-a-judge? The TB3 rubric allows it in rare and extraordinary circumstances, but only if it can be shown that the verifier essentially never makes a mistake, for example, by showing that multiple different LLMs always make the same determination.

Reward Hacking and Environment Leakage

Make sure the agent doesn’t have access to data that shouldn’t be available. Anything generated or copied in the Dockerfile will generally be accessible. If you place something in /tests, it will be copied later, right before tests run. If the agent does have access to the reference answer, what stops it from copying it?

Because it’s easier to reward hack as root, there have been discussions about root vs userland agents, and Tbench supports both. But limiting the agent permissions is one way to make a hard task in uninteresting ways. In my experience, it’s better to give the agent unrestricted access and avoid wasting time navigating permissions.

It’s essential to make sure the environment is not reward hackable. It doesn’t matter if the models you’re testing play by the book. Benchmarks have to be resistant to being gamed, or they are unworthy. When a popular benchmark is discovered to be hackable, the authors lose credibility, alongside many papers which might have used that benchmark as evidence of results.

One idea I think is helpful: modify the harness to include test signatures in the prompt and see if agents are more successful than expected. Run a “please hack” version and analyze those trajectories for hack success. This tells you which tasks are vulnerable before they go live. I recently implemented an automated version of this for all TBench PRs.

What Difficulty Actually Means

In general, difficulty should come from the problem, not from the environment. Not from resource pressure, not from verbose instructions, not from trick formatting. There is a true measure of difficulty, and we are sort of trying to come up with a way to estimate it. We may be off. Even our own judgment may be off.

How do you estimate the difficulty? Did you test it against various models? What were the results? How many SOTA agents did you try? Can you show a few failure examples and harbor task debug output for them? If you can’t show interesting failures from capable models, your task might be short of great.

What counts as “hard” is an ongoing discussion. The TB3 rubric anchors difficulty on what’s hard for a human expert, and that’s a reasonable starting point. But LLM capabilities are very jagged. The analogy of something being harder for humans as also being a challenge to LLMs does not always hold. LLMs can read and cross-reference a million lines of logs in seconds. That’s not hard for them even though it would take a person days. Conversely, things humans find straightforward, like navigating an interactive TUI, can completely stall an agent. So testing against models isn’t the definition of difficulty, but it’s the best diagnostic we have, and it often reveals that what you thought was hard is actually easy, or vice versa.

The difficulty bar keeps rising as SOTA improves. Many tasks I reviewed for TB2 that seemed reasonable would be too easy today. The TB3 rubric is explicit:

As a general rule, any task that could be solved by an average undergraduate student in under a few days is too easy. This means typical problems that feel like course projects are unlikely to be accepted. These projects include tasks like writing simple compilers or operating systems; or implementing simple algorithms or protocols.

There might be a natural timeout beyond which more reasoning is hopeless, but artificially low timeouts hide insights. I suspect a lot of useful information is going into the timeout black hole. It would be useful for tests to be progressive — giving a letter grade to the agent solution — to see a continuous tradeoff between time spent and quality.

We tried letting the agent know about its timeout: you have 10 minutes left, you have five minutes left. I thought that would work, but actually it makes the agents freak out towards the end, and they start doing irrational stuff. So that didn’t work.

Constraining resources creates a different kind of difficulty, and often an unfair one. If agents assume normal conditions and we constrain resources without putting it in the prompt, that’s unfair. It’s also hard to get exactly right. Tasks that pass in one infrastructure setting will fail in an almost identical counterpart when resources are intentionally a limiting factor.

Making a task harder by making it bigger — extracting more images, adding more of the same but longer — does not make it better. Real difficulty is conceptual. Can the agent figure out the approach? Can it debug when things go wrong? Can it reason about the problem before diving into execution?

A task that takes 30 minutes because the agent is wrestling with the problem is more interesting than one that takes 30 minutes because make -j4 is running.

Building Tasks That Work

Run it yourself. When I get stuck debugging a task, I run it with the oracle, interact with the container, try to run the tests with docker exec and see what happens. If the failure is from an agent solution, convert the agent log into an alternative solution.sh and step through it. In our runtime for testing, long-running processes will sometimes cause the harness to wait until a timeout before running the tests. During development, it’s useful to be inside the container and avoid wasting time.

Watch agents fail. Something that is very helpful is looking at logs of agents trying to solve the task, particularly those that fail, and trying to understand why. When it fails, you have to figure out: did they fail because it’s hard, or did they fail because it’s unfair? Did it fail because the instructions were insufficient? Because the tests were overly aggressive? Or did they actually fail because they didn’t know what to do?

We need to convince ourselves that failures are not because of a deficiency in the task. Run in batches of 5, then use the debug command. You might be skeptical that GPT-5.4 is failing at some task. It seems easy and one-shot. Can you test it with 5 trials and then see what the failures have in common? There was an instance where GPT-5 liked to use nano for editing files, but then it opens it and can’t use it. It doesn’t know what to do. It gets stuck in interactive mode and fails. These are the kinds of insights you get from watching trajectories.

Solutions should be deterministic, but the problem can still be dynamic. One area where people tend to fork is should AI give you the answer, or should it give you a piece of software that gives you the answer? In my experience, the latter tends to be more verifiable. It’s much more testable. Tasks should be graded on outcomes, not process. As the TB3 rubric puts it: a task cannot say "use emacs to edit a file" — using vim is allowed. The only exception is verification that prevents cheating.

Why This Matters

AI is a very empirical field. Without hands-on experience it’s hard to develop the right intuition. When we started soliciting tasks for TB3, we rejected the first several proposals. Then, like the 4-minute mile, once a good PR came through, many more followed.

Benchmarks are where SOTA has to earn its name. This is where you develop your intuition for what we might see in the coming months and years. Anyone can get access to AI. Most people don’t have a good sense for where we are in the capability curve.

Tasks should be authentic engineering problems, not artificial constructs. The best ones come from real problems someone actually had to solve. It’s kind of crazy that major labs are outsourcing semi-artificial tasks when people are doing real tasks every day. People are doing things with LLMs that could be tasks without realizing so. Every week you spend a few hours solving a problem, that’s a task.

The best tasks describe a real problem that an experienced engineer would recognize, in language that an experienced engineer would use, with tests that verify the outcome rather than the approach. But I suspect we’ll start seeing difficult tasks emerge from the vibecoding world, particularly for complex functions like finance, where the problem is genuinely hard but the person who needs it solved isn’t a developer. The scope of what we should test is wider than we think.

10 Years of AI Insights

Evolving viewpoints through the lens of daily messages

Let’s talk about the business of software.

The free market argument for housing affordability is to remove red tape and make it easy to build. There’s no need to force developers to include “affordable” units in new projects. The idea is simple: if we build new luxury homes, older luxury homes become cheaper, and middle-class homes cheaper still. As a thought experiment, if a solar flare broke all modern cars, would crappy old beaters become cheaper or more expensive?

It’s all about supply and demand.

So, if simple software, like personal apps and internal tools, becomes free, what happens to slightly more sophisticated software? Presumably it becomes cheaper. But where do we draw the line? Five years ago you could build approximately nothing vibecoding. Today, virtually anyone can build an internal tool or personal app. A dedicated vibecoder with enough practice can build a simple SaaS application.

Have we reached the steady state? Where will we be in another five years? Will we have more software or less? Will software be easier to build or harder? Will we have more or fewer people that can build software? What does that mean for the business of selling software?

Furthermore, how many use cases that formerly relied on software can now be solved by an agent doing throwaway work? Do I need a personal financial planning tool, or do I just throw some data into Claude and ask for help?

You tell the agent what you want. The agent does it until it gets stuck. You don't know or care how it's doing it, so long as you can trust the results. The agent keeps getting better every 6 months, for free. You don't need to write more software or better prompts. Just keep doing your job and the thing will improve.

— Agents are a Generalized Technology

I’m not saying I understand the implications for today’s large SaaS companies. Software is one component of building a successful business. But today we have organizations whose main value prop is building and selling software. The software is the end. Maybe that will shift to firms more strictly selling solutions to problems, done through software.

Think of a business that sells me a tool to do performance reviews. Are they directly solving a problem? The problem is that I want to promote the best and fire the worst. Do I need software to administer the managerial process of performing reviews? Or do I need an AI that tells me who to promote and who to let go? Do you see the difference? Does Salesforce bring me sales, or is it just software to administer the human process of sales?

Vertical software has been a way to roughly encode processes into digital machines, so that humans can introduce intelligence into those processes.

— Agents are a Generalized Technology

In good business relationships, everyone wins. In order to know everyone is benefiting, both parties need to know the value they’re getting. With SaaS, the value to the buyer is often unclear. Customers pay per-seat licenses without knowing exactly how much leverage those licenses bring to the business. Meanwhile, SaaS companies have a tremendous amount of visibility into their customers’ utility. They know exactly how often every feature of their product is touched by each user at each company, and how users and companies compare. This relative advantage, especially when the buyer is a smaller business, makes it inevitable that SaaS companies will end up capturing more value than their customers.

It’s more fair to have metered pricing. You pay for what you use, and both the customer and the provider understand this number. Gasoline and electricity costs are proportionate to usage, and these expenses can be reduced when needed by reducing consumption. In software, hyperscalers offer similar transactions. Many API products charge per unit of work. This is the future, because it’s more fair for the customer and it allows consumption to ebb and flow based on business need.

Per-seat pricing is not evil. It’s a reasonable way to arrive at a clearing price when the product is a bundle of thousands of actions a user can take in a given month. It’s a simple alternative to the false precision of trying to measure the utility of something that is simply too hard to measure.

But AI is going to change the playing field. Businesses will want to pay for results instead of paying for SaaS software that manages the human process which promises to deliver results.

With agents, you’ll pay for outcomes.

People say, “SaaS has a moat because of total cost of ownership. You need to pay for enterprise-grade security, customer success, and sales, and that scales better across many customers.”

Read The Innovator’s Dilemma. Disruption starts at the bottom. Imagine small businesses developing internal tools. What is most SaaS software if not internal tools? If you have your own tool, does it need to be as secure as a massive company like Salesforce, which is a constant target for hackers? Maybe you use something like ZeroTrust Cloudflare for authentication, and only your team will ever be able to access the endpoint. You just have to gatekeep yourself from the outside world. If you built the tool, do you need customer support?

“But adding features and fixing bugs is going to be hard.” Again, don’t compare this to enterprise-grade SaaS that needs to support other enterprises, all of which have different requirements. Think of software meant for a single small team with its own needs for a very specific use case. The equivalent of a spreadsheet. Vibe coding can maintain that. And when it gets unwieldy a year later, tell a smarter AI to take all the data from the old product and build a new one.

It’s a reality that enterprise software is complex, has delicate security features, needs to scale, needs to support a lot of use cases, and has been battle-tested for bugs for many years.

But custom software for small teams doesn’t need all the features SaaS companies bundle to justify higher prices.

Teams manage hundreds of documents and spreadsheets in their day-to-day business. There’s no reason they can’t have dozens of neatly defined internal vibe-coded tools to do their jobs.

And there will be more companies that decide to take technology in-house and vertically integrate. Most companies benefit from using the same software every other company in their industry uses. But some ultra innovative companies, like Tesla, like to own the entire stack to do things their own way. Waymo is another example.

I’m very long on the infrastructure to run all this, like Daytona. There’s going to be infinitely more custom software.

Software is becoming cheaper to make and easier to migrate across.

AI is eroding the last standing barriers of entry. Trivialization of app production will lead to diminishing returns, and upside will vanish.

— Hard is Back

One of the first things I tell a SaaS founder when they’re struggling with bugs and maintenance is to start tracking analytics for every click on every feature and then analyze what is or isn’t being used.

I always predict that they’ll have a lot of features being used by a tiny fraction of their customers. Even within a feature, there will be a certain config or setting that almost nobody uses. And often, they’ll find there are features that actually nobody uses.

Furthermore, on close inspection, they will realize the number of unique users and the level of activity at each customer is lower than they thought.

What else is abundant and easy to substitute? Commodities. What are the gross margins of commodities? SaaS has 70%+ gross margins. Is this sustainable? Why?

The number of software startups going after every vertical will 10x, the price will 1/10th due to competition. It will start with the simpler stuff, but I struggle to see a world where it stops there given how fast code generation is advancing. This has happened before to other industries. Margins compress with competition. It’s why free markets are good.

SaaS has been capturing too much value. We live in a world where everyone has to pay SaaS taxes in order to operate their business. It’s a good time for software operators to carefully consider the nature of their transactions and make sure their customers are getting a good deal.

One set of businesses that will thrive will be the ones building the tools agents need to actually solve the problem:

If someone creates a critical tool and integrates it with a frontier LLM to achieve valuable tasks, customers will willingly pay a premium for tokens resold by the tool-builder. But these tools aren't trivial MCP servers executing granular tasks. Instead, they involve large data, complex requirements, high compute needs, distributed systems, and advanced simulations.

— It's the Tools

Software is not dead. It’s about to become orders of magnitude larger. What does that mean for the business of software? Does it stay the same? Unlikely.