misotofu
中文 / english

When software development becomes infinitely fast, can the real world keep up?

Lately I keep seeing developers share their AI coding workflows. Some claim to run hundreds of coding agents at once and to open, or even merge, thousands of PRs a month.

Setting aside whether those numbers are real and how they’re counted, I don’t think that kind of engineering output is technically unimaginable. As coding agents get more capable, one person can start a large number of tasks and have them build features, fix bugs and change code in parallel.

But one question keeps coming back to me: if software development gets a hundred times faster, can a product, a company, or a whole industry move forward a hundred times faster too?

For a new project starting from scratch, that speed isn’t hard to picture. Developers can stand up a system, build features and rework the architecture quickly. The product hasn’t met real users yet, so there’s no need to think about how existing users will adapt to the changes.

Large refactors are similar. As long as external behavior stays stable, the code inside can change a great deal without users noticing at all.

A product that already has real users and is being run as a business is a different matter.

Even if the team can finish dozens of new requirements a day, do users actually need them? Can they adapt to that much change? The effect of one feature hasn’t been properly observed before the next one ships.

Widen the view from pure internet products to businesses in the physical world, and the problem gets more obvious.

Software can be finished overnight, but supply chains have to be built, hardware has to be developed and manufactured, customer relationships have to be maintained, products have to be deployed, and services have to keep being delivered.

So I want to start with an assumption that is very generous to AI: the code coding agents write is entirely correct, and neither engineering quality nor automated testing is a bottleneck any more.

Even then, how much of the growth in software production turns into progress in the real world?

1. Development throughput is not the speed of product progress

First, it’s fair to admit that a mature product has a lot of work that can soak up new development capacity.

Performance work, compatibility fixes, localization, long-tail cases, third-party integrations, and requirements that piled up for years because there weren’t enough engineers. This work usually doesn’t make users relearn the product, and doesn’t necessarily change the core experience.

If development costs suddenly dropped a hundredfold, much of that backlog could be cleared quickly.

That would be a real release of productivity.

But a backlog is finite. New devices, new platforms, competition and regulation keep producing new requirements, yet they don’t necessarily appear faster just because code can be produced faster.

Once the requirements that are easy to spot and clearly valuable have been met, the team faces another question: what should we actually do next?

This is where the number of PRs finished in a month and how much the product actually improved start to visibly diverge.

We need to tell apart at least three numbers:

  • How many code changes were made.
  • How many of those changes actually reached users.
  • How many of them ended up improving the user’s experience.

The first two can grow quickly with engineering capacity. There’s no reason the third should automatically follow.

And for businesses that depend on supply chains, hardware, offline operations or customer deployments, there’s a fourth question: can the changes that are finished, and genuinely valuable, actually be delivered into the real world?

code changesreach usersimprovement?real deliverysupply chain · hardware · deployment

Finishing the software is just one link in the chain.

2. How much change can users absorb?

One of my first worries was that users can’t cope with a product that changes too often.

On reflection, that judgment needs narrowing.

Most software updates ask very little of users. A slow page gets faster, an occasional error gets fixed, or a feature appears only when the user needs it. Changes like these are usually just good.

The updates with a real cost of adaptation are the ones that change existing habits.

For example, an entry point someone uses every day suddenly moves, a familiar workflow gets redesigned, or the product’s main interaction model changes.

Snapchat’s 2018 redesign is one example. In its filings that year, Snap acknowledged the redesign had hurt daily active users; average DAU in Q2 2018 fell about 2% from the previous quarter.

But the cost of adaptation has another form too: accumulated complexity.

Suppose a product adds five new features a week, each with a clear need behind it and none breaking an existing flow.

A year later the product may have gained a large number of entry points, settings, states and ways of interacting.

Even if each feature makes sense on its own, together they can make the product harder and harder to understand.

0 more, shown when neededweek 04 entry points

Back in 2005, Thompson, Hamilton and Rust’s research on feature fatigue found that consumers tend to overvalue a rich feature set when choosing a product, and care more about ease of use once they actually use it.

Of course, complexity isn’t an unsolvable problem.

Through progressive disclosure, segmentation, personalization, and perhaps natural-language interaction in the future, we can keep a large number of features out of sight until the moment users actually need them.

So I’m now less inclined to think that users’ ability to adapt sets a single, fixed ceiling on how fast software can iterate.

It’s more like a cost. Product design can lower it, but as changes keep piling up, it’s hard to guarantee it will never show up.

3. What’s really hard to speed up may be feedback from the real world

Compared with user adaptation, I care more about another limit: how long does it take to know whether a product change was actually worth it?

Microsoft has shared its experience with large-scale online experiments. In products like Bing, fewer than a third of ideas improved the metrics they targeted, and in highly optimized areas the success rate can be lower still.

That doesn’t mean the rest of the ideas were worthless, nor that every product has the same hit rate. But it does show that what a team believes is worth doing doesn’t necessarily have the effect it expects.

When building a feature takes months, the cost of building it forces the team to be selective.

If it only takes minutes in the future, we can try many more ideas.

That’s a big step forward. Cheap trial and error may give a chance to ideas that used to cost too much to test.

But no matter how fast software gets built, it doesn’t automatically produce more real user behavior.

For changes that can be measured in the short term, such as button click-through or sign-up conversion, a high-traffic product can run a great many experiments at once. As long as they’re well designed, the capacity for feedback really can scale with traffic and experimentation infrastructure.

Low-traffic products have it much harder. Even if a hundred variants can be built in a day, there may not be enough users to reliably tell them apart.

More importantly, many significant product changes can’t be judged from a few days of data.

Did the new recommendation system change users’ long-term content habits? Does a learning product actually improve learning? Can a new kind of social interaction gradually turn into a stable habit?

All of these questions involve real behavior changing over time.

Thirty-day retention needs at least the corresponding thirty days of user behavior to observe. No matter how much compute and development speed grow, future behavior won’t happen any sooner.

Of course, we can run experiments in parallel on different user groups, or start a new one every day, and keep getting long-term data as it matures.

So calendar time doesn’t simply limit us to a few experiments a year.

The real limit shows up in a chain of interdependent product decisions: if the next step has to be based on the long-term result of the previous one, that wait is hard to eliminate entirely.

developobserve 30 daysfirst result: day 40test 1test 2test 3test 4test 5test 60306090120days

Short-term proxy metrics can speed decisions up, but they raise another question: do they really represent long-term value?

A video app may get users to stay longer by optimizing recommendations. But does more time spent necessarily mean users got more value?

Conversely, a tool that cuts a ten-minute task down to one minute lowers time spent, while the value users get may go up.

So we also need to separate the value users actually receive from the revenue, time and engagement a company can capture from it.

Especially as development costs fall sharply and competitors can easily copy features, the commercial value a product’s maker captures and the value users get may drift further apart.

Faster development lets us produce candidate solutions faster, but it doesn’t necessarily speed up our understanding of the real world by the same factor.

4. Knowing what’s worth doing doesn’t mean you can do it right away

So far the discussion has mostly been about purely digital products. But in many real businesses, software is only one part.

Imagine a company that installs automated food-service machines in office buildings.

With coding agents, it can quickly build the ordering pages, payment flow, device management console and inventory system, and even generate different operations interfaces for different clients.

But the software being ready doesn’t mean meals can be reliably served in a hundred office buildings the next day.

The machines need to be manufactured, parts need to be sourced, food needs to be supplied, deliveries need to be arranged, and each site needs to be ready for installation. Clients have to agree to let the company in, and operations staff have to know how to restock, clean and handle problems.

We can even go further and assume the product direction is settled, client needs are confirmed and the software is entirely correct.

All of these things still have to be done.

At that point, what limits how fast the business moves may not be not knowing what’s worth doing at all, but knowing exactly what to do and not yet having all the conditions in place.

Hardware development is the same.

The control software can be done quickly, but a new actuator has to be designed, machined, assembled and tested. Even if the design has no mistakes, the physical thing doesn’t arrive at the client’s site along with the commit.

Customer relationships involve a different kind of time.

Suppose a client is willing to buy a few machines first. They may need a period of real operation, to see that the service is stable and that someone handles problems when they come up, before they’re willing to expand to dozens.

During that time the client isn’t just supplying a batch of feedback. Both sides are building a shared history, forming expectations of each other, and gradually taking on larger commitments.

A better CRM can reduce what slips through the cracks and help with follow-up, but it can’t conjure that history out of nothing.

In their 1989 work on how firms accumulate assets, Dierickx and Cool discussed a phenomenon: putting in more doesn’t always shrink the time it takes to build up an asset in proportion. Customer trust, working relationships and organizational capability may all have this property.

Of course, these processes aren’t forever impossible to speed up. Mature suppliers, standardized parts, existing channels, better automation and more ample resources can all shorten the time it takes to build.

But those need specific conditions.

We can’t assume those conditions already exist just because software development got cheaper.

Some time goes into understanding the real world, and some into changing it. The latter can’t be replaced just by producing information faster.

5. Local speed can’t be extrapolated to the whole business

A simple hypothetical illustrates the problem.

A project has to finish four weeks of software development and then eight weeks of manufacturing and delivery. Assume these steps really do have to happen in sequence, and nothing else changes.

Even if the software were done instantly, the whole project would only shrink from twelve weeks to eight.

That’s real progress, but it’s a long way from a hundredfold.

If the software could already run in parallel with the hardware and was always finished first, speeding up development further might not move the final delivery date at all. It can still save money and add room to adjust, but those gains won’t all show up as a shorter business cycle.

Conversely, if the equipment can’t go in precisely because the software it depends on isn’t finished, faster software could unlock a great deal of value immediately.

The key is whether software is, at that moment, the link that’s really holding up the whole project.

softwaremanufacturinginstantwaitingdelivered in week 1204812weeks

That’s also why I don’t agree with summing up the real world’s limits as “people can’t keep up with AI.”

Some problems involve human judgment and learning, some involve equipment, materials and sites, and some involve dependencies among several organizations.

Pushing one link to an extremely high rate of production doesn’t automatically coordinate the others.

What’s more, cheaper development may make it easier to start changes whose cost to carry out hasn’t fallen at all.

Suppose software support for a new machine design took a single night. So the team decides to launch it as soon as possible.

But that decision may require suppliers to change production schedules, operations staff to change how they restock, clients to rearrange their sites, and after-sales to stock new spare parts.

From engineering’s point of view, it’s a cheap change. From the business’s point of view, it mobilizes a lot of resources that haven’t got any cheaper.

The code isn’t wrong, and the design may well be better, but changing direction often still eats into the time of manufacturing, sales, operations and clients.

So we need to distinguish cheap exploration from expensive commitment.

Having agents generate ten prototypes and compare them internally can be very valuable.

But once a prototype becomes a promise to a client and goes into purchasing, production, training and deployment, it’s no longer just a branch you can delete at any time.

exploring: ten prototypescommitmentpurchasingproductiontrainingdeployment

The cost of implementing a change and the cost of delivering on it may diverge more and more.

6. The biggest advances may never have come from rapid iteration

Still, concluding from all this that the future pace of software progress depends on feedback capacity or on how long a business takes to build seems too simple to me.

Looking back at consumer internet history, some important product changes weren’t reached by making small optimizations to an existing product in the first place.

In 2006, Facebook launched News Feed, pulling updates that had been scattered across users’ profile pages into a continuously updated stream.

The change was met with strong user backlash at first. Facebook then added privacy controls, but kept News Feed as the direction.

It’s an interesting case.

Judging only by the feedback at launch, users didn’t seem to like the change. But in hindsight, the feed became one of the most important interaction forms in social networking.

Similar changes include Stories and short-video feeds.

Changes to a product’s form like these involve users forming new habits, and are hard to derive from small improvements in existing metrics through local A/B tests alone.

They require someone to make a judgment about future user behavior while the evidence is still thin.

Of course, judgment can fail. Snapchat’s 2018 redesign shows that even with good reasons, a product team can’t be sure users will eventually accept a new way of interacting.

valuecurrent peakearly feedback gets worse

This makes me think of another possibility.

If most of a mature product’s growth in value comes from a few major changes, then what matters most may not be how many improvements get made each year, but whether we can recognize the opportunities worth changing the whole product’s direction for.

There’s an assumption here that hasn’t been properly tested: that new value in consumer products may be highly uneven, with a few improvements creating most of it.

Even if that holds, it doesn’t follow directly that judgment matters more than experimentation.

If those high-value ideas can be found through lots of cheap experiments, AI coding might instead make finding them dramatically more efficient.

Only when high-value opportunities are both hard to recognize in advance and impossible to confirm quickly through real behavior do the quality of judgment and calendar time become especially important constraints.

But whether it’s a major product innovation or a change to a real-world business model, it eventually has to be tested and carried out in the real world.

One right judgment can change direction, but it won’t build what comes after on its own.

7. A hundred times the capacity doesn’t all have to go into the same product

The discussion so far has rested on a default assumption: that once software productivity increases, we keep improving the same mature product.

But why must that be so?

If the cost of building software falls a hundredfold, the new capacity may flow somewhere entirely different.

For example, needs that used to have too few users to justify engineering resources suddenly become economically viable.

A niche workflow, a tool for one particular profession, even software someone tailors to their own daily habits could all get the chance to be built.

That way, we don’t have to keep adding features to existing products. We can create more products that are smaller and more targeted.

This could move software from standardized, mass distribution toward some degree of personalization.

But I’m not sure how large that space really is.

First, users’ time is still limited. Nobody suddenly needs a hundred times as many apps because software got a hundred times cheaper.

Second, the core value of a lot of software may already be concentrated in very few features.

For example, when Facebook acquired WhatsApp in 2014, it already had more than 450 million monthly active users. Its core experience then was still simple, reliable instant messaging.

That’s a reminder that software can create a great deal of value without constantly expanding its features.

And personalized software still has to solve data migration, interoperability, trust, and whether users really know what they need.

Users can more easily get the features they want, but the solutions users come up with won’t necessarily solve the real problem.

Still, falling development costs may give many niche needs that were never served before a chance.

That growth may not show up as the same product adding more and more features. It may show up as more independent software, more specialized services, and business models that couldn’t work before because the costs were too high.

And once software is cheap enough, some capabilities that used to require a large standardized product may gradually become tools users can combine, modify, or even generate themselves.

This may change how we understand software products today.

In the past, a company had to gather enough users’ needs together to pay for building a product. In the future, a feature may be worth building even if only a handful of people need it.

But cheaper software development doesn’t mean these new products can all easily build a complete business.

A small tool for individuals can be delivered directly over the internet; a product for a specific industry may still need sales, training, systems integration, customer support and long-term maintenance.

If physical goods or offline services are involved, it also needs supply chains, sites and a delivery network.

So growth in software productivity may make products and services more diverse, but how much of that new capacity a given business can absorb still depends on its own real-world conditions.

I think this is something easily overlooked when discussing AI coding productivity: we shouldn’t only ask how fast a mature product can keep iterating, but also, once software development is cheap enough, which products and businesses that couldn’t exist before finally get a chance to.

Both kinds of growth may happen at once. Some capacity goes into improving existing products, some into exploring new needs, and some into improving the links that have long held back what a business can deliver.

How much value they end up creating can’t be inferred from the speed of code production alone.

8. When code is no longer scarce

Back to the original question. If someone really can manage hundreds of coding agents at once and finish thousands of PRs a month, I don’t doubt that represents an enormous gain in productivity.

But we need to rethink what we should measure that gain by.

For mature software products, faster development lets us implement requirements, fix problems and try new product directions sooner. But users’ attention, habits and real feedback don’t grow in step with the volume of code.

For businesses involving hardware, supply chains and offline operations, the limits are different. Even if the product direction is entirely right and the software is done, production, delivery, customer relationships and organizational capability still take time to build.

These limits aren’t immovable. AI and automation can also improve how needs are judged, speed up experiments, optimize supply chains, even change how a whole business is organized. But each time one bottleneck is broken, we have to find the next link that’s really holding the business back.

developmentuser feedbackproductionclient trustbottleneckdark: what actually gets through the whole chain
speed up

That’s why I think productivity growth in the software industry shouldn’t be measured only by lines of code, number of PRs or how often features ship.

What’s really worth caring about is: how long does it take a product to go from an idea to being accepted by users? How long does it take a capability to go from finished design to actual delivery? How fast can a company discover needs, build capabilities, and keep creating value for users?

Once software development is no longer the most expensive, most time-consuming link, we may find that many problems once blamed on a lack of engineering resources were also limited by other conditions in the real world.

My judgment is that software development speed can grow by orders of magnitude, but the pace of progress of mature products and real businesses won’t automatically grow by the same amount.

That doesn’t mean the value of AI coding is overrated. Quite the opposite: it may let us see, more clearly than ever before, what besides software development decides how far a product can go.

Future competition may no longer be mainly about who can write features faster, but about who can judge more accurately what’s worth doing, get feedback from the real world faster, and actually turn the right ideas into products and services that can be delivered sustainably.

Code can be generated faster and faster, but user habits need to form, customer trust needs to build up, supply chains need to run, and real businesses need to be built step by step.

Software can come ever closer to being done instantly, but making it create value in the real world still takes time.