Writing archive

· building with AI

I Decided to Build While AI Keeps Improving

I have been wondering whether to keep exploring a project or wait for better AI. I chose to continue, with a different test for progress: what will I understand even if the implementation changes?

Low teal, near-black and coral painted planes overlap beneath a broad aqua arc on a textured cream ground.

I have been in a familiar dilemma lately, working on a project I am still exploring. Should I keep going, or wait until better AI makes the work easier?

The question is reasonable. I can imagine spending considerable effort on something that a future model handles with much less assistance. It is uncomfortable to commit time when the tools may change the economics of the work before I finish it.

I decided to proceed with the exploration. That decision has made me think more carefully about what I want to get from the time I spend.

Ethan Mollick gave this dilemma a useful frame in The Lazy Tyranny of the Wait Calculation, published on January 16, 2024. His argument is that waiting can sometimes produce an earlier or cheaper result when technology improves sufficiently quickly. But he also warns that waiting can deprive us of learning and the satisfaction of doing the work. His own experience building educational games helped him use AI to create new ones.

I read that as a reason to examine the purpose of a project before deciding whether to postpone it. The essay's model comparisons and timeline forecasts belong to early 2024; I do not need to carry those predictions forward to find the dilemma relevant today.

What would I actually be waiting for?

“Better AI” is too vague to guide my next decision. It gives me no condition I can test and no moment at which the wait is over.

A more useful version would name a limitation: a task that fails too often, a cost that makes the intended use impractical, or a missing capability that prevents a meaningful experiment. Then I can ask whether I should pause that part of the work, change the experiment, or try a simpler approach.

There is a difference between postponing an expensive commitment and postponing every opportunity to find out more. I want to be able to do the first without drifting into the second.

For example, imagine a team considering an AI tool for reviewing documents. This is an illustrative case. The team might reasonably defer a complex automated workflow until it can meet a reliability threshold. It could still examine representative documents, work through difficult examples with people, and decide what a useful review must contain. Those activities could reveal that the proposed workflow solves the wrong problem, saving considerably more work than a future model upgrade would.

The next model may change how the review is produced. It will not retroactively run the conversations the team chose to postpone.

The understanding has to survive the prototype

There is a research tradition behind the idea that preparation affects our ability to benefit from progress. In their March 1990 paper on absorptive capacity, Wesley Cohen and Daniel Levinthal argue that prior related knowledge helps firms recognize, assimilate, and use new external knowledge.

That is an organizational learning argument, not evidence that every AI prototype pays off. My application of it is narrower: working through a problem may put me in a better position to recognize what a stronger tool makes possible.

Still, “I am learning” can become an excuse for continuing indefinitely. I need something more concrete than time spent or code produced.

The question I want to ask after an experiment is: What can I now explain or decide that I could not before?

A useful answer might be that an apparent model limitation was actually an unclear requirement. It might be a set of examples that separates acceptable results from convincing failures. It might be evidence that an interaction is awkward, that a proposed feature is unnecessary, or that the whole direction deserves to stop.

I would want to preserve those findings outside the implementation: the question, the examples, what happened, and what I changed my mind about. A prototype that gets discarded can still leave a useful record. A polished prototype with no clearer understanding is harder to justify as exploration.

Keep the cost of changing direction low

Choosing to continue also changes how much permanence I want to build into the work.

I would rather make a small experiment answer one question than build a large foundation around assumptions I have barely tested. That means limiting the time and budget, keeping the relevant parts easy to replace, and deciding in advance what result would make further investment worthwhile.

Anthropic's engineering guidance in Building effective agents, originally published in December 2024, recommends starting with the simplest solution and adding complexity when it demonstrably improves the outcome. The page now notes that its tooling landscape has changed. The principle of earning complexity through evidence is the part I find useful here.

For my own exploration, I would use four questions before the next substantial piece of work:

  1. What uncertainty will this help me resolve?
  2. Can I answer it with a smaller experiment?
  3. What will remain useful if the implementation is replaced?
  4. What finding would make me stop or change direction?

A small experiment leads to two outcomes: an implementation that can change and understanding to carry forward.

A practical test for exploration: preserve the findings even when the implementation changes. This is a conceptual diagram, not a measured result.

These questions leave room for waiting. If a task mostly involves maintaining a workaround, teaches me little, and depends on a capability I cannot currently obtain at an acceptable cost, pausing it may be sensible. The same is true if the experiment would create obligations that make changing direction expensive.

But I do not want the possibility of easier implementation to become a general argument against curiosity.

Why I am continuing

Part of my decision is simply that I want to explore the idea. That is an honest reason to spend some of my own time, provided I remain honest about the cost and what I have learned.

I am not treating the decision as a promise that the project will become a product. I am giving myself room to discover whether it should, and to change my approach as the tools change.

A future model may make some of today's work unnecessary. I want to be ready to welcome that. The point of this exploration is to reach that moment with better questions, clearer judgment, and something worth trying next.

Sources