Broadsheet

Modified from a photo by RJD via Wikimedia Commons

The photo above is not from China; it’s from Japan. In the 1970s, Daiei was Japan’s top retailer. But after Japan’s asset bubble burst around 1990, it became Japan’s most famous “zombie” company — staggering along unprofitably, kept afloat by a constant stream of below-market-rate loans from UFJ Bank and other big Japanese banks. Eventually the company was acquired by Aeon, a more successful retailer, and its once-storied brand is slated to be retired for good in the next few years.

I tend to be very skeptical of comparisons between post-1990 Japan and post-2021 China, because there are just so many differences between the two economies (and between the global economic environments at the time). Their industrial policies are different, their trading relationships are different, their bubbles and busts happened for very different reasons, and so on. But in the case of “zombie” companies, there may be some important parallels.

What’s important about Daiei is not how it failed, but why it didn’t fail much sooner. Caballero, Hoshi, and Kashyap wrote a paper in 2008 arguing that “zombie” companies like Daiei held the Japanese economy back during the 1990s (and, in some cases, even beyond the 1990s).

The basic story is that after 1990, the Japanese economy slowed down, and lots of companies that used to be profitable — especially in the construction, retail, and trading sectors — were no longer profitable. These companies owed a lot of money to banks. If they stopped being able to pay back their loans, the banks would be forced to recognize bad debt on their books. This would get them in trouble with regulators (because of capital requirements), and it would also get them in trouble with the Japanese public.

So what the banks did was to lend even more money to the failing companies that already owed them a lot of money, at very cheap interest rates. The new loans were used to pay back the old loans, and the new loans would be classified on the bank’s books as “good” debt. This process — known as “evergreening” — kept banks from ever having to acknowledge their losses:

Peek and Rosengren (2005) document this empirically as well.

Evergreening kept a bunch of companies afloat — like Daiei — that had utterly broken business models. Theoretically, the companies could have eventually pivoted their business models and recovered, or Japan’s economy could have started booming again, etc. In practice, this never happened.

Caballero, Hoshi, and Kashyap argue that evergreening was very bad for the Japanese economy, because it hoovered up scarce resources that better companies could have used to grow. With all of those crappy loans clogging up their books, Japanese banks couldn’t lend to healthier companies. With big zombies like Daiei still able to employ large amounts of Japan’s best managers, young scrappy upstarts were deprived of talent. The authors argue that keeping all of this labor and capital locked up inside doomed companies contributed significantly to Japan’s long productivity stagnation.

Why did the Japanese government allow this to happen? Preserving employment at the zombie companies was probably a big part of it. Japan had a strong tradition of job security at that point in time, and to throw so many people out of work — even if they could have gotten new jobs eventually — would have been seen as cruel and unfair. Social unrest was a possibility. Bank bailouts may also have been deeply politically unpopular. In any case, whatever the reason, throughout the 1990s the government supported banks with various capital injections and regulatory forbearance, without forcing banks to cut off the zombies.

Anyway, that’s Japan. The question is whether something like this will happen in China.

China’s experience with its real estate bubble and bust doesn’t exactly parallel Japan’s, but there are some broad similarities. Since 2021, there has been a broad economic slowdown (probably more severe than the official numbers suggest), and a long-lasting chill in real-estate-related industries. This has predictably led to a rise in the number of loss-making companies:

You’ll notice on this chart that the share of non-performing loans has actually gone down since 2021, even as fewer companies are turning a profit. That suggests that lots of Chinese companies are being kept on life support by cheap bank loans. Here’s the Rhodium Group:

Some concrete data points suggest that China’s evergreening of debt is more widespread than is commonly the case in most market economies. The ratio of banks’ reported non-performing loans has decreased over the past years, while the share of loss-making enterprises increased…This would indicate Chinese banks have been sitting on large volumes of NPLs that have not yet been fully recognized. This is an open secret: The National Audit Office recently claimed in an annual audit report to the NPC that 16 of 43 audited banks last year had NPL levels that were double the officially reported figure…
Loan rollovers are a pervasive phenomenon in China…[T]he financial system…served as a shock absorber, channeling resources to enterprises facing losses to maintain output and prevent the defaults and bankruptcies that occurred in market economies.

Another Rhodium report finds that the proportion of loans made below benchmark rates has risen significantly since 2021, even though benchmark rates are lower than they were back then:

And the Dallas Fed has documented how more and more Chinese companies, especially in the real estate sector, aren’t making enough money to pay the interest on their loans:

All this — falling official NPLs, much more below-market lending, companies unable to pay their interest expenses, widespread suspicion that many of the companies whose loans are “performing” will never be able to repay those loans — matches the general pattern that Hoshi and Kashyap (2000) documented in post-bubble Japan. Banks have taken a bunch of losses, but have refused to recognize those losses, using a flood of cheap debt to keep their borrowers afloat.

A bunch of people have warned about this. Here’s Rhodium:

Because of the political incentives shaping China’s financial system, banks in China tend to extend or roll over debt to poorly performing or loss-making companies. This can have some of the same effects as a subsidy, by removing incentives for companies to stay profitable and isolating them from market forces that would otherwise lead to their restructuring or bankruptcy….Evergreening of credit, therefore, allows firms to…[reduce] domestic and global prices to unprofitable levels[.]

And here’s the Dallas Fed:

There is mounting evidence of “zombie lending” in China, banks rolling over bad loans to unprofitable firms and allowing the status quo to continue rather than recognize losses.

And here’s a Business Times story about how China’s government has allowed and even encouraged zombification, much as Japan did in the 1990s:

It’s impossible to quantify the true extent of the [bad debt] problem, though most economists say the ratio of bad loans is significantly higher than the 1.5 per cent official rate…One analyst at Absolute Strategy Research in London pegs it at about 10 per cent…Others say it could be double that amount…
While the [banks’] leniency [toward borrowers], largely condoned by regulators in Beijing, has helped maintain financial stability over the past few years, it also means the banking system is recycling capital into unproductive companies rather than spurring real growth in healthy firms…
[Government] officials have moved to bolster the nation’s six biggest banks with more than US$100 billion in fresh capital…[R]ather than cracking down on deadbeat borrowers, China’s banks are encouraged to cut them some slack. Regulators have for years urged the big banks to keep their reported bad loan ratio under 2 per cent, according to sources familiar with the guidance…As a result, banks routinely roll over maturing loans, extend repayment periods, or allow interest to be capitalised to avoid triggering NPL recognition.

Now you might be tempted to think — and I’ve seen a few people argue — that this only matters in a market economy. In a market economy, undercapitalized banks matter because banks have to succeed or fail on their own. In a state-directed economy like China’s, the theory goes, debt on the banks’ books might as well be on the government’s books.1 Banks can keep lending no matter how much bad debt they have, because the only entity that could punish them — the Chinese government — wants them to do so.

But while government control might avert a financial crisis, it doesn’t automatically solve the zombie problem, or make the comparison with Japan inappropriate.

First of all, it would be a mistake to see Japan’s government in the 1990s as operating at arm’s length from Japanese banks. It most certainly did not; in fact, it acted to support the banks that were supporting the zombies. The government bailed out the banks, deliberately turned a blind eye to the zombie problem, and encouraged banks to keep on lending to healthier companies despite the unrecognized bad loans on their books. That’s not too different from what China’s government seems to have done in response to the real estate bust, at least initially.

But simply having the government urge (or order) banks to keep lending didn’t solve the zombie problem in Japan, and it won’t solve it in China either. Even if the zombie companies don’t end up competing with healthier companies for capital, they compete with them for other resources. They compete for labor — workers who could be working at young, growing, healthy companies are instead being paid to continue to work for unproductive companies that are just spinning their wheels. They also compete for raw materials, for land, for energy, and so on.

These resources are not in infinite supply, even in China. As long as unproductive zombie companies are hiring workers, hoovering up metals and chemicals and watts of electricity, and taking up prime real estate, they’re holding back the rest of the economy. This doesn’t just manifest as higher costs for healthy companies — it also shows up as increased competition. In 1990s Japan, if a new retailer wanted to enter the scene, it had to compete with Daiei, the unproductive behemoth that was essentially being paid by banks to produce below cost. The same will be true in China.

In fact, this may be a reason for the “involution” that Chinese companies are experiencing. In the wake of the real estate bust, China’s government directed banks to lend to manufacturing companies instead of to real estate-related companies. They did this (though some of the loans ended up sneaking back into the real estate sector). In fact, a large percent of the “subsidies” that China dishes out to its manufacturing companies is through below-market-rate loans.

Some of these manufacturing companies will be successful and efficient — indeed, many already have been. But others are unproductive and inefficient. Instead of letting these die, China’s banks may keep them on as zombies as well, paying them to compete with China’s healthier companies. Here’s Alicia Garcia-Herrero from back in March:

In many sectors, including…electric vehicles, solar panels, batteries, and other green technologies…Chinese firms…keep selling at rock-bottom levels, sometimes below what it costs to produce, just to hold onto market share. A growing number of these companies cannot earn enough revenue to even service their debt…These “zombie” companies survive only because banks roll over loans and local governments provide subsidies to avoid job losses and keep tax revenues flowing…In newer, high-priority sectors like green tech, the share of zombie companies has hit 30 percent of total listed companies…
Without real productivity advances, [zombies] still join the price-slashing frenzy to stay in the game thanks to external support from banks or local governments. They cut prices aggressively…The outcome is predictable: collapsing profit margins across the board, even for the better companies, whose productivity is increasing.

When we Westerners think about the effect of Chinese zombification, we often think about the flood of cheap exports threatening to deindustrialize Europe and other regions. But while that export dominance might seem like a victory to China’s mercantilist leaders, it’s a double-edged sword, because zombification reduces productivity at home. In the long run, lower productivity hurts growth, despite the temporary bump from exports.

In other words, China’s fusion between the financial system and the state may have made zombification worse, not better. The Chinese state is not a ruthlessly efficient allocator of capital; it has sociopolitical goals just like any other state, and it fears the unrest that could result from widespread corporate failure and unemployment. Yes, it can tell banks to lend to manufacturers instead of property developers, but that just ends up adding more zombies to the horde.

And at some point, even state-owned and state-directed banks probably do care about profitability. Yes, the government can bail out any bank at will, but if you’re the bank executive or manager who dished out the bad loans and made a bailout necessary, your career might be over. This might be why corporate loans have started to fall slightly from the torrid pace of 2023-24:

Ultimately, when people write the story of China’s economy in the 2020s, zombification could end up being more fundamental to that story than exports. The parallels with Japan are not always real, but they’re real in this case — and so far, China’s government seems to be walking into a similar trap.

Update: In the comments, Jack Lowenstein asks a very important question: So what? Even if zombification proceeds in China, what are the downsides from the point of the Chinese government? He writes:

I think the critical difference between Japan’s “extend and pretend” policies and China’s is the geopolitical element.
Japan feared domestic social and political disruption - and was heavily influenced by “free market” vested interests. There was also a degree of denial by MOF and METI that the gogo years of the post war period up to the mid 1980s were really over.
The CCP and the PRC however are driven by the deliberate aim of de industrialization of critical parts of the OECD supply chain. Loans and other support to the companies that will deliver this outcome are not going to stop for economic reasons.
Sadly policy makers in most of the countries suffering these effects are ideologically unwilling to enact anti-dumping and other defenses to respond. So zombification will not stop in China. Yes the population of the PRC will pay a price. But since when did the CCP care about that?

This is a very important question, and I should have probably gone into that more in the post. Here was my response to Jack in the comments:

I think some of these are real differences, but perhaps not all of them.
“Japan feared domestic social and political disruption” <-- I actually don’t think this is a big difference. China is worried about social and political disruption as well -- just look at how fast Xi ended Zero Covid after some small scattered protests. The old social compact in China was “growth in exchange for political quiescence”. But with rapid growth now over, that social compact is gone, so the possibility for unrest is definitely there.
“There was also a degree of denial by MOF and METI that the gogo years of the post war period up to the mid 1980s were really over.” <-- I’m not sure this is different either. China has been overstating its growth since the bubble burst in 2021 (https://rhg.com/research/chinas-economy-rightsizing-2025-looking-ahead-to-2026/). This is often a tool the government uses to “smooth” growth between good and bad years (https://www.aeaweb.org/articles?id=10.1257/mac.20150074), suggesting that they think fast growth might come back.
“The CCP and the PRC however are driven by the deliberate aim of de industrialization of critical parts of the OECD supply chain. Loans and other support to the companies that will deliver this outcome are not going to stop for economic reasons.” <-- This is true, and I think this is an argument FOR zombification. Unproductive, unprofitable companies that fill supply chain gaps will continue to be supported with evergreened loans.
So the question becomes: What are the downsides of zombification from the regime’s perspective? That’s a topic I should have considered more. One answer is “social unrest” -- if slow growth makes the repressiveness of China’s regime less tolerable, then we could see popular anger at the industrial-policy regime. Remember that Japan was a very free society, where people could pivot from the pursuit of money to the pursuit of lifestyle and art and leisure. That’s not necessarily true in China.
Another possibility is that eventually China becomes more like the USSR. The USSR was famously unproductive, because it insisted on onshoring its entire supply chain. Right now, China looks hyper-competitive in a bunch of high-tech industries, but if zombies suck up more and more labor and other resources (including compute), that competitiveness could narrow over time.
Finally, there are fiscal dangers (https://rhg.com/research/chinas-financial-and-fiscal-decay/). When Europeans buy cheap Chinese EVs, part of the consumer surplus they receive comes out of the pockets of Chinese taxpayers and bondholders. Japan’s zombification caused it to run up an enormous amount of debt, which it was able to carry safely only thanks to A) persistently low demand and low natural interest rates, and B) the government’s ability to buy overseas assets that performed extremely well (https://www.ft.com/content/f7d3f20c-b303-4f6c-b4a0-8ee8906ae155). Now that the first of those has gone away, Japan’s government debt IS becoming a problem, with a plunging exchange rate and creeping inflation.
So while China’s government can get away with “damn the economics, full speed ahead” for a while, eventually I think something breaks...

Subscribe now

And since that debt is owed almost entirely domestically, the theory says that the debt doesn’t really matter in a macroeconomic sense; it’s just some Chinese people owing money to other Chinese people.

A few days back I wrote a piece called “AI enthusiasts are in a race against time, AI skeptics are in a race against entropy.”

I have notes on a whole pile of AI-related topics that I’d like to cover in depth: AI mandates, communication norms, code review, AI art, and more. Unfortunately, I got too many interesting responses to my last piece, and now I have to address those before I can move on to other topics. 😉

There were two types of interesting responses: the first on the technical merits, the second on ethical grounds. I will respond to each of these separately. Let’s take the technical side first, because it’s easier.

Somehow, a subset of readers came away believing I was telling everyone to ditch code review and push their shittiest code straight into production without reading it, right now, tout suite.1

That is not what I am doing. That is not what I think you should do. But I did not pick that example at random, and I will tell you why.

In 2025, the question was whether AI could ever generate “good” code

It’s easy to forget, but for most of 2025, the idea that AI-generated code was slop and might always be slop was not only a reasonable position to hold, it was the default, mainstream position.2

That question was answered decisively last November. Ever since Opus 4.5 came out, AI has been able to generate code that is approximately as good as that of the median software engineer, at least for common patterns, and much faster and more cheaply. I came out of a book hole and realized this in January, and over the first few months of 2026, it seemed like everyone around me was having a similar realization.

But many saw it coming much sooner.

The popular narrative holds that Opus 4.5 was what changed. But Opus 4.5 was more like the tipping point. Agentic harnesses (the code that wraps the LLM in a loop with tools) became a real thing in mid 2025, with precursors building back to late 2024. Tool use, function calling, MCPs…all of this wave was building over the course of 2025, and crested into real general purpose usability at the end of the year.

That’s what the enthusiasts were trying to tell us last year. Not only “this is coming”, but “this is coming faster than you think.”

As it turns out, they were right.

It was reasonable to be skeptical the first time

As you may know, I come from the reliability side of the house. The compliment I will pay to myself and my people is that we do not struggle to adapt to new realities. As soon as a problem is real and in front of us, we adjust smoothly, even eagerly, thanks to an unwholesome zest for lapping up disgusting technical messes (and the campfire tales we get to tell later).

The un-compliment I will pay myself and my people is that we sometimes struggle to accept that progress is real, that the continued existence of bugs and edge cases does not diminish the fact that huge swaths of problem space do get more-or-less solved over time, to the point they can be taken for granted by most people.3

The speed at which code went from total crap to “ah damn, that’s not bad” is what I have in the back of my mind, as enthusiasts are telling us that harness engineering and AI validation is real, it’s already here, and it’s getting better astonishingly fast.

Holding out for “I’ll believe it when I see it” was forgivable the first time, but much less so the second time. This is what it feels like to be on the inside of an exponential change curve, turns out.4

What happened in 2025, exactly?

I want to pause here and be very clear about what I think is happening. Then I’m going to tell you what specifically I am excited about, and why.

You are under no obligation to join me there. But there are way too many sweeping statements out there right now about “it was never X” — “it was always Y” — “the future belongs to xyzzy” 🤮 — and I want to be crystal clear how conditional and specific and contextual my claims are.

What happened in 2025 was this: the economics of code production were turned upside down. Instead of being very hard, time-consuming, and expensive to generate code, it became effectively free and instant. Lines of code went from being treasured, reused, cared for and carefully curated, to being disposable and regenerable, practically overnight.

For most of computing history, the primary way people have learned to understand software is by writing the code. Once you've achieved some mastery, reading and discussing code gets you most of the way there. (I might argue that software engineers have always relied far too heavily on the code instead of sensemaking the system through observability.)

“The real product of a software team is shared understanding”

Many great software engineers hold that true product of every (good) software engineering team has always been a shared understanding of the software we own. That it gets stored as cache state in our fragile little meat brains, frequently flushed to disk, deployed to production, committed to github, but our minds are where meaning has always lived.

Is it any wonder that software has always been such a fiercely collectivist endeavor, exquisitely sensitive to relationship dynamics and manners and questions of fairness and emotional valence? It’s exactly what you’d expect when part of your brain lives in other people’s brains, and your collective interdependence is sky high.

It’s something that I love about this industry. But there’s no denying that minds have been a poor container for certain aspects of the software development model. We are forgetful, distractible, impatient. We are bad at spotting small details, we grow habituated to repetition. Worst of all, the model in our heads diverges massively and perpetually from the world our users interact with.

Anyway, SREs have never quite bought that explanation. To us, it’s clear that the true product of every (good) software engineering team is production.

Only prod is prod. Test in prod, or live a lie.

(This is all backstory. I am getting to the point, I promise.)

Turns out, this is an engineering problem after all

We issued our AI mandate last August.5 I had seen enough to know that this was happening, and it was time to do the responsible thing. Honeycomb is a devtools company, and people come to us to help with hard problems on the forefront of technology. I was all in on AI, but I can’t say I was super excited about it, in my heart of hearts.6

Then I found Chad Fowler’s writings on Phoenix Architectures.

If you don’t know what I’m talking about, you should honestly stop reading my shit right now and go read his. Chad is the guy who coined the term “immutable infrastructure” in 2013. His best-known essay is “Relocating Rigor”, because Martin Fowler7 mentioned it recapping a Thoughtworks meetup on the future of software. I replied with “Production Is Where the Rigor Goes”, complaining that they didn’t talk about production enough.

When I wrote that, I think “Relocating Rigor” was the only piece I had read. But soon I found the rest of it, and after reading two or three essays, it just clicked. I knew exactly what he was talking about. I could predict the rest of what he was going to say. And then, reader…then I got excited.

This has all happened before, and this will all happen again

I am going to give you a small sample of Chad quotes, just enough to get the gist. Here’s one from “The Death and Rebirth of Programming”.

Immutable infrastructure. Stateless services. Containers. Blue-green deployments. Infrastructure as code.
These ideas all share a common premise: never fix a running thing. Replace it.
AI pushes this premise beyond infrastructure and into application code itself. When rewriting is cheap, editing in place becomes risky. Mutation accumulates entropy. Replacement resets it.

Another favorite: “The Deletion Test”.

Here’s a simple test you can apply to any software system you work on:
Imagine deleting the entire implementation.
Most engineers experience deletion as existential. Code feels like the thing. It’s what we write, review, version, deploy, and debug. Losing it feels like losing the system itself.
When people say, “We can’t just throw the code away,” what they usually mean is something more precise:
We don’t know exactly what behavior is required.
We don’t know which failures are unacceptable.
We don’t know what invariants must always hold.
We don’t know how to tell if a new version is correct.
We don’t know which bugs are intentional fixes for forgotten edge cases.
Those are not code problems. They are evaluation problems.
Code becomes precious when it is the only place knowledge lives.

and,

For most of software history, treating code as durable was reasonable.
We treated code as permanent because the labor to produce it was the bottleneck. Rewriting was expensive. Re-validation was risky. Implementations accumulated meaning over time. Structure, tests, comments, bug fixes, and tribal knowledge fused into something you learned not to disturb.
That made sense when production was the constraint.
When regeneration is easy, code stops being an asset and starts acting as a cache: a materialized view of understanding that is useful while current, disposable when stale.

“A materialized view of understanding that is useful while current, disposable when stale.” I think that might have been the exact line that made it click in my head.

Do you remember the sysadmins?

I am just barely old enough that my first job title was “System Administrator”. I was a teenager, working at the university, with root on every machine in the days before they learned they should definitely not do that.8

I lived through the shift from handcrafted server pets to immutable infrastructure cattle. I didn’t really understand what was happening at the time, but I’ve contemplated it a lot in recent years. I wrote this in the final chapter of “Observability Engineering”, 2nd edition (available for download as of Wednesday, June 17th!):

The shift from handcrafted servers to immutable infrastructure taught us that mutability is the sworn enemy of understanding. Any artifact that is edited in place creates drift. Drift is what makes systems impossible to maintain.
Our ability to kill and regenerate infrastructure components is the reason we trust it. At Honeycomb, we kill the oldest Kafka node off via cron every Tuesday. That’s why we are confident in our bootstrapping and balancing processes: everything is repeatable, the data can be regenerated, the commitments live elsewhere.
The fact that we cannot regenerate our code in the same way is a sign that we do not understand it. We do not know which commitments we have made, we do not know which dependencies will break. We find them by breaking them, mostly.

Think of all the years of your working life you have wasted on painful migrations and rewrites. Think of replacing load-bearing legacy code. Think of all the strangler figs.

Lines of code have been doing too much. The code has been the bundled up repository of developer intent, user expectations, implicit and explicit behaviors, the only fossilized composite record we have of bugs gone by. It’s too much!

Lines of code are not the ideal artifact to review

And look at all the domains that have been neglected due to the towering, all-consuming expense of maintaining and mutating lines of code. Where are the artifacts I can review and discuss to understand how our architecture is evolving? Where are our architecture artifacts, period? What if we could discuss and converge on an architecture diagram, and the code could be regenerated from changes to the architecture, instead of the architecture being kinda-sorta inferred from the code?

I am not asserting that all code will eventually be AI-generated to spec, bypassing human understanding. The feasibility of this whole endeavor hangs on the question of what a spec is, or what a spec could be. Anyone who has ever done a painful database migration should have learned some goddamn humility about our ability to extract and formalize users’ expectations in a replayable, automate-able way.

But I think that every step we can take in that direction will be good for us.

The tools to do this don’t exist yet, but many of the ideas do exist. Most come from operations and QA, two domains that software engineering has historically been rather snobbish about.

Those tests and techniques are not about testing for correctness or what ought to be happening, they are about observing and encoding what is happening. Behavioral tests, characterization tests, capture/replay, traffic splitters. Observability (the good kind).

Our brains were not built for validation

Having nondeterministic code in production is finally forcing us to do the things we should have done all along. Instrumenting with traces. Tests and evals in production. Production is not what happens after development is over, production is a stage of development.

Human brains are not good at validation. The nitpickiness, the repetition. This is the worst thing to be clinging to, y’all. There are so many better things for us to want to preserve and assert for ourselves in the production and maintenance of software. We are never going to beat the machine when it comes to validation — we are literally the weakest link!

My money’s on humans for a good long time when it comes to creativity, inspiration, leaps of logic, and a lot of other things, but PLEASE do not rest your killer argument for humans in software on us being the best quality gate. OMG. 🙈

Alright. I’m almost done here. Just one more thing.

Nondeterministic systems will require more engineering discipline, not less

I think what many engineers have found so alienating and terrifying about the last two years of AI discourse has been the way so many prominent AI voices appear to be gleefully declaring that software is no longer an engineering problem. “SaaS is dead!” “Making AI great at coding was the strategy that unlocks everything else”, and so on. Even Adam Jacob, one of my dearest friends and someone who is rarely wrong about technology, seems to anticipate a bloodbath of software jobs.9

If 2025 was the year of vibe coding, where AI got as good at generating lines of code as the median software engineer, and the range of possible futures often felt destabilizingly, impossibly wide open, I feel like 2026 is shaping up to be a return to discipline.

The knowledge in our heads is unavailable to AI until we encode it into the system, after all. The returns on those investments will be massive and nonlinear. We might argue that they always would have paid for themselves in the long run. But now every CEO in existence is chomping at the bit to get some of those AI cookies, so let’s give it to them. Discipline first, cookies second.

This is our chance to bring our engineering values to the mainstream

The share of software engineering teams that work in short, fast feedback loops (the cardinal sign of discipline in my book) is, and always has been, appallingly small. Five percent, maybe? Definitely less than 10%. AI tooling brings this more within reach than ever before. Or it can. It could. The discontinuous returns on investment in engineering discipline are real enough that it just might happen.

I am not worried, at least in the near term, about AI creating massive, discontinuous returns on investment in the absence of engineering discipline. (Many will try, and it will be entertaining to watch.)

But value is backed by durability, not disposability, and I don’t see that changing. Bits are cheap and fast and governed by the rules of logic and language, but anything with value must ultimately resolve with physical systems: persistence on the one side, user experience on the other.

People do not want to wake up every day and log in to Slack and find the buttons and menus all subtly moved around. People do not want financial transactions that complete most of the time. Determinism is not going anywhere, my friends.

AI is not magic. This is still engineering. As Adam says, “it’s still technology, and technology needs technologists.” And I for one am looking forward to learning new and interesting engineering problems, reviewing different kinds of artifacts.

And never doing another sticky, picky, two year long API rewrite or strangler fig migration, ever, ever again.

~charity

P.S. Thanks to everyone who read a draft and gave me feedback: Dave Williams, Chad Fowler, Adam Jacob, Mark Ferlatte, Austin Parker, Erwin van der Koogh, Ankur Bhatt.

I was not trying to be neutral or even-handed in my last piece, only to give a baseline of courtesy to everyone. But I think it’s revealing how many times I was accused of being “so overly hard on skeptics”, by skeptics, and “so overly hard on enthusiasts”, by enthusiasts, and sometimes simply “It’s sad how some people can’t accept reality” with no indication which side they meant. Lord.

Fred Hebert and I gave the closing keynote at SRECon in March of 2025 where we told SREs they should get to know AI, maybe even try vibe coding (pause for laughs), because otherwise their critiques wouldn’t land as well.

Seriously, that was our big pitch. Learn AI so that you can complain more effectively.

Infrastructure, for example. I think this is true of a lot of engineers, btw. I just think it’s really really true of the type of engineer that signs up to be an SRE. Technological pessimism and ADHD, our two most defining traits.

There is a segment of AI enthusiasts who believe we are entering an era of eternal exponential growth, in which the machines begin to build better and better machines, in ways we cannot understand.

I think those people are bad at math. The only thing we know for certain about exponential growth is that it will end. It always does. either in an S curve or a crash. (For a good time, google Heinz van Foerster and “our great-great grandchildren will be squeezed to death.”)

I definitely think we will use machines to build the machines — duh, we already are — but that’s about recursion and specialization. I think the exponential curve we are on the inside of now was created by sloshy free money chasing high returns, plus the properties of software as a function of language and logic, plus the biggest discoveries always happen in the early days of a technology boom, because low hanging fruit gets picked first.

My personal sense — and keep in mind that I am no kind of expert on AI — is that the exponential advancement in AI models leveled out a while ago, and gains are becoming harder to earn and more incremental in nature. I may turn out to be very wrong, of course. But even if there were no more AI innovations moving forwards, the past year has unleashed enough pent-up force to radically reshape the software industry as we know it. Like a pig in a python, we will be dealing with the consequences for a long time to come.

More on this coming EXTREMELY soon. Watch the Honeycomb blog!

The tech is cool, but as a thinking, feeling, breathing human who cares about other people, it can be hard to get excited about anything that so many people are this upset about. It’s also hard to get excited about something when so many of the loudest voices are out there talking gleefully about putting everyone permanently out of work, and so many artists and writers and people from developing nations are talking openly about the impact on them.

Hold your desire to jump in and berate me here, I beg you. Like I said, I will deal with the ethics and morality of using AI in my very next post. Be honest, your attention span is no more up for reading a 10,000-word essay than mine is up for writing one. (Can we blame AI for that too?)

“The Other Fowler.” I gather they’ve been making this joke for like.. fifty years.

I share a longer version of this story in the second edition of “Observability Engineering, chapter 32, downloadable later this week!!"

Adam is rarely wrong about technology, and I am 100% sure he is living and working in _a_ future of software engineering. I am less sure it is the future we will all be living in. If the hardest part of software has never been writing code — as is my belief — it logically follows that even if the economics of code production drop to zero, the hard parts will still be hard.

Off into the sunset

Astronaut Jessica Meir Assists With Hardware Updates for NASA’s Cold Atom Lab

Downloads

Astronaut Jessica Meir Assists With Hardware Updates for NASA’s Cold Atom Lab

Description

Discover More Topics From Photojournal

What the CFTC Actually Governs, and Why It Matters Here

The States Tried to Step In. Trump Sued Them.

What Crash Gambling Actually Looks Like for Users

Warren’s Letter and What Comes Next

FAQ

The Regulatory Vacuum Isn’t an Accident

Related Stories

Comments

Comments

Related Stories

The first years

The collapse

Comments

Related Stories

Comments

Related Stories

Downloads

August 19, 2025

References & Resources

You may also be interested in:

Discover More from NASA Earth Science

Subscribe to Earth Observatory Newsletters

Earth Observatory Image of the Day

Earth Science Data

Comments

Related Stories

In 2025, the question was whether AI could ever generate “good” code

It was reasonable to be skeptical the first time

What happened in 2025, exactly?

“The real product of a software team is shared understanding”

Turns out, this is an engineering problem after all

This has all happened before, and this will all happen again

Do you remember the sysadmins?

Lines of code are not the ideal artifact to review

Our brains were not built for validation

Nondeterministic systems will require more engineering discipline, not less

This is our chance to bring our engineering values to the mainstream

Trying the Cloudflare MCP

Using the API instead

It’s a shame

Actual alternatives

Voting Reform Became a Trust Fight

AI Regulation Hit the Brakes

Mobile Betting Shows Why Regulation Must Track Behavior

Assisted Dying Split Parliament and Families

Bangladesh’s Reform Vote Raised a Hard Question

Cyber Laws Returned Under a New Name

Betting Platforms Live or Die by Clear Rules

Gold Medal Game Changing the Conversation

4 Nations Face-Off Blueprint

Why International Hockey Hits Different

Olympic Impact on the NHL

Should the All-Star Game Go?

Comments

Recurring 1:1s with Inner Circle

Extended Staff Meeting

All Hands

Here’s The Rub

A Perfect Storm of Clutter

More Than Tidy Shelves

Who’s Hiring Organizers Now?

Beyond Decluttering: The Specialized Services Driving Demand

A More Thoughtful, Sustainable Approach

Organizing As A Lasting Foundation

FREQUENTLY ASKED QUESTIONS ABOUT PROFESSIONAL ORGANIZER SERVICES

What does a professional organizer do?

Why are more people hiring professional organizers?

Is professional organizing only for people with clutter problems?

What is move management?

Can professional organizers help with downsizing?

Revised rules

What have we done in practice

Comments

Tuesday, February 6, 2024