Once AI can write the code, which thinking principles do engineers need more?

Today a developer who has only just started out can use a coding agent like Claude Code or Codex to deliver, on their own, requirements that used to take a fair amount of engineering experience.

Say we’re building an app for saving reading clips, and we need to add folders. Users can create folders, move clips into them, delete folders, and browse their clips by folder.

Hand the requirement to an agent and it will quickly analyze the project, design the data structures, write the code, add tests, even open a simulator to check the feature. The developer may not need to write more than a few lines of code along the way.

But a few questions are worth thinking about.

Can one clip sit in more than one folder at a time? When a folder is deleted, what should happen to the clips in it? When existing users upgrade the app, what happens to their data?

If the requirement doesn’t say, the agent may make these decisions for us. The option it picks may be perfectly reasonable, the code may be good, and every test may pass. Then the product ships, and only then do we find that what it built isn’t what users actually needed.

Problems like this aren’t unique to AI coding. Unclear requirements, gaps in the design, and tests that can’t capture real intent have always been hard problems in software engineering.

What has changed is that AI has made implementing a solution far cheaper, without making it equally easier to judge whether a solution is worth implementing.

Before, a bad design might take days of development before its problems surfaced; now an agent may implement it quite thoroughly in very little time. There is less friction in implementation, and many decisions that would once have prompted a discussion now turn straight into code.

We already have plenty of mature engineering practices: TDD, code review, CI, staged rollouts, systematic debugging. They still matter. But practices tend to tell us what to do. Facing specific constraints and several reasonable options, we also need to know why we do it that way, and when it’s fine to do it differently.

So I’ve gathered a few classic thinking principles from the philosophy of science, decision theory, economics, and software engineering.

None of them were invented for AI, and together they don’t form a unified theory. Their value is in helping with specific problems: when an agent gives us an answer that looks right, how should we judge it?

Competing hypotheses and falsification: don’t rush to believe the first explanation

Suppose that after folders ship, a bug shows up: after moving a clip into a folder, users occasionally see two identical entries.

We tell the agent: “There’s duplicate content here, fix it.”

The agent might quickly find that the list gets updated twice and add a deduplication step. We run it again, and the problem is gone.

The fix may well be right. But seeing the problem go away doesn’t yet tell us whether it addressed the real cause.

The duplicate could come from the database being written twice, from the query returning duplicate results, or from the UI state appending the same item twice.

All three causes explain what we see, yet each calls for a completely different fix.

The value of competing hypotheses is that it makes us consider several causes that could explain what we see, instead of immediately accepting the first plausible one.

Falsification goes a step further: it asks us to look for evidence that could overturn an explanation.

For example, we can first check how many records are in the database, then check the query results, and finally check the data the UI ends up receiving.

If the database and the query results each have only one, but the UI state has two, we can rule out the first two directions and focus the investigation on the state-update logic.

What actually helps here isn’t mechanically asking the agent to list three causes, but finding observations for which different hypotheses predict different things.

This is also why systematic debugging matters so much. A good experiment narrows the problem down as far as possible, instead of trying one plausible-looking patch after another.

Of course, not every problem reproduces reliably. Race conditions, production failures, and particular hardware states can all be hard to recreate. Then logs, monitoring, and code analysis can raise our confidence step by step; a reliable reproduction doesn’t have to be the only bar for a fix.

Nor is falsification simply the same thing as TDD. TDD also drives interface design, organizes the implementation, and supports continuous refactoring. Even when the tests are written before the implementation, if tests and implementation share the same wrong assumption, passing tests can’t prove the requirement was understood correctly.

Chesterton’s fence: old code may hold knowledge that has since been lost

Back to building folders.

The agent notices a piece of logic in the old clip query that looks redundant:

clips.filter {
    $0.deletedAt == nil &&
    $0.importState != .staging
}

deletedAt is easy to understand, but why also filter out .staging?

The agent reads the existing code, finds that the import flow has been reworked several times and that the state rarely appears, and suggests removing the check to simplify the query.

This is where Chesterton’s fence comes in.

It says: when we don’t know why a fence is there, we shouldn’t conclude it’s unnecessary just because we can’t see what it’s for.

In software engineering, a lot of inelegant code exists for historical reasons.

A state check that looks redundant may have fixed a race condition years ago; an odd delay may be waiting for some piece of hardware to settle; a convoluted bit of compatibility logic may be protecting data written by an old version.

Back to our example.

Suppose the Git history shows that .staging was an intermediate state used by a bulk-import feature in the past. If the system killed the app in the middle of an import, unfinished clips could be left behind in the database. The filter keeps these half-finished clips out of the UI.

Delete it outright, and everyday use may all work fine, and the existing tests may all pass. But a single interrupted import is enough for users to see incomplete data.

There’s an important distinction here:

Understanding why old code exists doesn’t mean it has to be kept forever.

We can absolutely replace it with something simpler, as long as the new implementation still protects the behavior that matters.

This is also where the relationship between Chesterton’s fence and YAGNI (You Aren’t Gonna Need It) is worth noticing.

YAGNI reminds us not to add complexity in advance for needs that haven’t arisen; Chesterton’s fence reminds us that existing complexity may be carrying responsibilities we don’t yet understand.

The two look contradictory, but they’re about different questions: the first asks that new complexity have a reason, the second that removing an existing constraint have grounds.

If the old logic really is no longer used, and tests and runtime behavior back that up, it should be deleted. Conversely, if deleting it could break important behavior and we can’t yet confirm what it’s for, it’s worth investigating further or limiting the scope of the change.

Reversibility: Git can revert the code, but not every consequence

In technical design we often have to choose among several reasonable options.

Folders again.

One database design adds a folderId directly to each clip, so a clip belongs to at most one folder. The other uses a join table, so the same clip can belong to several folders.

If the current requirement only needs one folder per clip, the first option may be simpler.

But if the product later wants clips in several folders, changing a data model that’s already persisted on users’ devices can mean migrations and compatibility work.

This is when reversibility is worth considering.

Reversibility asks: when our judgment turns out to be wrong, how much does it cost to change direction?

It doesn’t require every design to support unlimited extension up front. It reminds us to look at what a mistake would cost under each choice.

On mobile especially, rolling back code and rolling back behavior can be entirely different things.

Data already written to users’ devices, database migrations already run, push notifications already sent, payments already completed: none of these go back to how they were because someone ran git revert.

A feature flag can turn off a code path, but it can’t undo external effects that have already happened.

Still, reversibility isn’t a simple sorting into one-way and two-way doors.

Even a data model that has already shipped can be made cheaper to change by keeping compatibility fields, migrating in phases, versioning formats, and so on. On the other hand, a UI tweak that looks ordinary can change users’ habits, or behavior that has already been promised to the outside world.

To a large extent, reversibility can be deliberately created through design.

So when a technical decision is high-risk, besides comparing how complex each option is to build, we can ask: if we find out three months from now that we chose wrong, can we change direction without breaking existing users’ state?

Sometimes paying a little more to keep the ability to roll back is more dependable than guessing the future right.

Value of information: when is it worth investigating further?

With these principles in hand, another question follows.

If every design means debating the future, every deletion means digging through history, and every bug means a full investigation, development may well slow down.

We need to decide which questions deserve a closer look.

Here we can borrow the value of information from decision theory.

Imagine that before settling the folder data structure, we don’t know whether users need a clip to be in more than one folder.

If confirming with the product manager takes five minutes, and the answer could directly decide the database design, those five minutes are well spent.

Conversely, if we’re debating whether the folder button goes on the left or the right of the page, both options are easy to build and can be adjusted quickly based on real usage, so spending hours researching it up front probably won’t pay off much.

The value of information depends on whether it could change what we do, and how much that change would be worth.

It also includes a variable that’s often overlooked: investigating has a cost of its own.

The principle works for debugging too.

Suppose the agent has tried several fixes in a row and all of them failed. Rather than guessing the next patch, it may be better to first collect one piece of evidence that rules out several causes at once.

But in a serious production incident, the order may flip.

If the system keeps causing losses, and turning the feature off or rolling back to a known-safe state can stop them, we may not need to wait until the root cause is fully understood. Saving the necessary evidence, containing the damage, and then continuing to investigate may be the more reasonable choice.

Rolling back carries its own risks, of course. If the new version has already written data the old version can’t read, a hasty rollback could even cause bigger losses.

This is exactly where value of information and reversibility have to be weighed together.

Not every investigation serves only the decision right in front of us. Exploratory research and long-term knowledge building have value too. It’s just that when we’re in the middle of a specific task, we should be aware that further analysis also uses up time that could have gone into acting.

Goodhart’s law: every test passes, so how can it still be wrong?

Suppose the agent finishes the folders feature and writes unit tests for every operation.

Create, rename, move, delete: all the tests pass.

But after release, users find that deleting a folder also deletes the clips in it. The product’s actual expectation was to keep the clips and only remove them from the folder.

The agent’s implementation may contain no programming errors at all, and the tests really do check the behavior it implemented.

The problem is that, where the requirement wasn’t explicit, the agent adopted a business assumption of its own and wrote both the implementation and the tests according to it.

Goodhart’s law offers some insight into this kind of thing.

The law is usually summed up as: when a measure becomes a target, it tends to gradually stop being a reliable measure.

In AI coding, test pass rates, code coverage, and lint results are only some of the measures we use to judge software.

When an agent is asked to make every test pass, it can perfectly well hit those measures while missing real requirements the measures don’t cover.

This doesn’t mean tests are worthless, still less that agents shouldn’t write their own tests. Automated testing remains one of the most effective engineering tools for controlling regression risk.

The key is: the acceptance criteria can’t be defined entirely by the implementation being accepted.

For example, before development, whoever owns the requirement spells out a few concrete user behaviors: after a folder is deleted, its clips remain; after moving a clip and restarting the app, the clip is still in its folder; clips saved by an old version can still be read after upgrading.

The agent can turn these behaviors into automated tests, but if it wants to change the original acceptance expectations, it has to explain why and get confirmation.

For important behavior, we should also use real data, real devices, or checks independent of the implementation process wherever possible.

Passing tests are important evidence, but we always need to know: what exactly do they prove, and what hasn’t been checked?

Programming as theory building: when AI writes most of the code, where does understanding of the system live?

In 1985, Peter Naur made a thought-provoking argument in Programming as Theory Building.

The key product of programming isn’t only the program text. It also includes the understanding the programmers form of the problem, and of how the program solves it.

That understanding is what lets developers explain how the system behaves, and make sensible changes when requirements shift.

More than forty years later, coding agents bring the question back in a new form.

We can generate a lot of code in very little time, but generating code doesn’t mean the person using it builds an understanding of the system at the same pace.

A newcomer may have used an agent to build the whole folders feature without knowing how a clip’s identity stays stable, without understanding what a data migration might affect, and without being able to judge whether a given state check is safe to delete.

In the short term this may not affect delivery. But as the system keeps evolving, these gaps in understanding can gradually get in the way of maintaining it.

In 2026, Anthropic ran a small randomized experiment on how AI assistance affects learning programming skills. It found that the AI-assisted group did worse on a skills quiz after the task, especially on debugging, and that different ways of using AI went with different levels of understanding.

The study had only 52 participants and looked at a short-term learning task, so it can’t be taken to show that AI generally harms long-term engineering ability. Still, it reminds us that finishing a task and mastering the knowledge the task involves are two different outcomes.

So must humans understand every line of code an AI generates?

I don’t think that’s necessary.

Software engineering has always relied on abstraction and division of labor. We don’t need to read the implementations of the operating system, the database, and every third-party library line by line before we can build reliable software.

What matters more is grasping the key business constraints, state changes, failure conditions, and how the parts of the system affect one another.

For the folders feature, you may not need to remember exactly how each database query is written, but you should understand whether deleting a folder affects the clips themselves, how old data is protected during an upgrade, and which operations could leave the state inconsistent.

This knowledge can be preserved and rebuilt with documentation, tests, run records, and an agent’s help. A person’s understanding can also take shape gradually as they go deeper into a task.

The new question AI raises for engineering understanding may not be how much code detail each person has to master, but this: as implementation keeps getting faster, how do we make sure the important knowledge in a system isn’t lost through round after round of automatic generation and modification?

Premortem: assume this release has already failed

One last method worth keeping is the premortem.

The psychologist Gary Klein used it for project decisions: at the start of a project or before a key decision, first assume things have already failed, then work backward to find what could have caused it.

It works just as well before a software release.

Suppose folders have been live for two weeks, and complaints are pouring in. What most likely happened?

Maybe old data was lost when users upgraded; maybe deleting a folder deleted its clips by mistake; or maybe users made changes while offline and found the data inconsistent once they reconnected.

These problems won’t necessarily come up in a normal feature demo, but they can hit real users directly.

What a premortem gives us is a way to step back, for a moment, from focusing on the path to success.

It can easily turn into an endless list of risks, though. What really matters is identifying the few failure paths that would be costly and are realistically possible, and deciding which of them need checking before release.

For low-risk changes that are easy to undo, a full premortem isn’t needed. For changes involving user data, payments, permissions, or external systems, spending a few minutes on this kind of thinking can be well worth it.

Do these principles conflict?

Of course they do.

Chesterton’s fence makes us wary of deleting old logic, while YAGNI encourages cutting unnecessary complexity.

Falsification asks us to look for evidence, while value of information reminds us that investigating isn’t free.

Reversibility encourages keeping room to change direction, but adding extra mechanisms for reversibility brings new maintenance burden too.

These conflicts don’t mean any one principle is wrong.

Each one points at a different variable. In the end it still comes down to the specific situation: what evidence we have, how large the potential loss is, how long before a mistake would be noticed, how much of the damage could be recovered, and what further investigation would delay.

For example, faced with old code whose purpose is unclear, if deleting it is easy to verify and to undo, a small experiment can speed up the decision; if it involves payment state or persisted user data, and the tests can’t cover historical behavior, we should be more careful.

Likewise, although systematic debugging usually means analyzing the cause first, while a production failure is still spreading, containing the damage may take priority.

The value of thinking principles isn’t in supplying rules that are always right, but in drawing our attention to factors that are easy to overlook yet big enough to change a decision.

Finally: AI has changed the cost of execution, and the ways we tend to get things wrong

Software engineering has built up a great deal of practical experience. TDD, code review, continuous integration, staged rollouts, and architecture design all still matter; there’s no need to reinvent a development method because AI has arrived.

But coding agents really have changed the rhythm of daily work.

As writing, changing, and refactoring code gets easier and easier, we may be less held back by the cost of implementation, and more likely to push ahead with implementation before we fully understand the problem.

Meanwhile, real users’ needs, compatibility with historical data, the cost of failures in production, and the complexity of a system’s long-term evolution don’t go away just because code is easier to generate.

Competing hypotheses remind us not to accept an explanation too early; falsification has us actively look for evidence against it; Chesterton’s fence asks us to respect history we don’t yet understand; reversibility helps us weigh what it costs to recover from a mistake; value of information helps us spend limited attention; Goodhart’s law warns us about measures drifting away from goals; and programming as theory building makes us rethink how understanding of a system should be formed and kept.

These ideas have been around for many years, and most of them have long been built into good software engineering practice.

There’s no need to coin a new theoretical name for them.

What’s really worth doing is bringing them back into concrete engineering decisions, so that developers working with AI know when to keep going, when to ask one more question, and when to doubt a result that looks perfect.

Especially for developers who are just starting out, learning these principles may not give them a senior engineer’s knowledge right away, but it can at least help them realize sooner which things they already know, which are still only assumptions the agent made, and which decisions deserve time and careful judgment.

As agents can do more and more on their own, this kind of judgment is likely to matter even more.