Animations for System Builders: Metastability and Lost Writes

Animations for System Builders: Metastability and Lost Writes

Pretty pictures help us learn, maybe?

One of the things I love using LLMs for is helping me understand systems better. In the Strands Decider post I wrote about how I often ask ask me questions to check I understand this. The other technique I’m enjoying right now, and am delighted to see folks around me using too, is building animations.

Animations can help us build an intuition for how and why systems work, or how and why they fail.

Here’s one example: did you know that a Raft cluster can lose writes if even one of the cluster members has a storage layer that lies about fsync()?

When I built Stability Sim, I wanted to help engineers understand metastable failures better. That worked pretty well on an abstract level (e.g. retries and queues make metastability really easy to trigger), but I still hear doubts that these behaviors happen in real systems. After all, those real systems are production-proven, and wouldn’t we have shaken out the behaviors by now if they existed?

Let’s start with Postgres.

In this example, captured from traces on real Postgres 16 running in a real MicroVM. Before a load spike, the database is easily able to handle its 20 queries-per-second load, thanks to the whole working set sitting in the page cache. But after the load spike, we’ve filled up a lot of the memory running queries (each takes 12MB), which now run slower because they’re served off the disk rather than the page cache. I like this example, because it shows that even simple, non-distributed systems have these behaviors.

What about etcd?

Again, we have load spike triggering the metastable state. Writes, in this case. The overloaded peers look like they’re unavailable, which triggers a storm of application-level resends. I like this example, because etcd is widely-used quality software. Operational experience doesn’t seem to effectively shake out these kinds of issues.

When I wrote Garbage Collection and Metastability, a widely-known systems academic told me that it was a fantasy. I’ve seen it happen in the wild in ways I couldn’t share, and I can forgive people for not believing this can happen based on the hand waving in the blog.

So it’s nice to have a real world example, picking on etcd again.

At the start, we’re ticking along nicely at 500 reads/second, before spiking load to 1000 reads/s. The read spike increases the time spent in GC, increasing average tenure, and making less CPU available for the read workload. This is a self-reinforcing loop, and the system doesn’t recover without intercession.

I believe the same thing could happen on a runtime without GC, due to increasing work in the allocator, but haven’t been able to reproduce that yet.

The animations here are all generated with Kiro and KiroCrew, mostly with Opus 5.5. The actual results came from a real test environment, which captures the traces that back the animations.

Rocket Report: SpaceX says no thanks to military launches; Spanish rocket nearly ready

Welcome to Edition 9.16 of the Rocket Report! Lots of SpaceX-related news this week including a couple of Thursday shockers. In one story, the company told the US Space Force thanks, but no thanks after being offered three military launch contracts. Then, on the civil space side of things, SpaceX's Starship may required up to 22 refuelings to support a lunar landing, a NASA safety panel said.

As always, we welcome reader submissions, and if you don't want to miss an issue, please subscribe using the box below (the form will not appear on AMP-enabled versions of the site). Each report will include information on small-, medium-, and heavy-lift rockets as well as a quick look ahead at the next three launches on the calendar.

Miura 5 getting closer to launch. PLD Space says its Miura 5 rocket, due to undertake its inaugural launch next year, is nearing final integration, Aviation Week reports. The rocket is one of three Miura 5s the company has already manufactured, with the other two designed to qualify key system elements, CEO Raúl Torres said in an October 19 program update. PLD Space now targets a first flight of the small-lift vehicle before mid-2027.

Read full article

Comments

Latest estimate for Starship launches required to support a Moon landing? 22.

NASA does not have, nor want, nuclear weapons. But when it comes to a particular metric that is part of the Artemis Program to land people on the Moon, NASA guards a particular number like it is trying to prevent outsiders from cracking nuclear codes.

I am speaking of the number of Starship launches required to fuel a Starship-based lunar lander in low-Earth orbit such that it can fly out to the Moon, land there, and return to lunar orbit.

This number has been a moving target over the years. The general response from NASA has typically been, 'We don't know for sure, so we're not going to say.' In 2023, for example, the then-director of NASA's Human Landing System, Lisa Watson-Morgan, told Ars, "We have a general idea, but I’m reluctant to say exactly what that is."

Read full article

Comments

US official: Russia's stalker satellite sure looks like an orbital weapon

Four years ago, a senior official in Russia's foreign ministry painted a bullseye on commercial satellite networks that assist Ukraine in its fight against invading Russian military forces.

On Wednesday, several days after a Russian military spacecraft moved within striking distance of one of those commercial satellites, the same Russian diplomat assured the world there's no reason to worry.

There's a sharp incongruity in those statements from Konstantin Vorontsov, deputy head of Russia's delegation to the United Nations. Vorontsov's comments Wednesday in the UN General Assembly's First Committee came moments after a US official called attention to recent maneuvers by Kosmos 2612, a Russian military craft that approached a US-registered commercial satellite some 340 miles (550 kilometers) above Earth.

Read full article

Comments

Friday assorted links

1. Weird diets (NYT).

2. Is this what the Denisovans looked like?

3. Measuring bottlenecks for AI.

4. How is violence diffferent when women lead drug cartels?

5. “Israel’s economy is growing so fast, it will leapfrog the Netherlands and Australia within the next 18 months, in time for it’s 80th birthday—not by 2030, as predicted—to rank second only to the USA in GDP per capita of all nations with populations over 10 million.”

6. Latin America seems to be climbing the high-tech value chain.

7. ZMP by any other name (amazing how much people used to hate on that idea).

8. Chatting with Milton to run your central bank?

9. Warhound.

10. “Somehow, @srajagopalan and @lachygroom have managed to fund a hard-tech revolution in India. There’s something rather massive going on, and everyone else has missed it.”  Link here.

The post Friday assorted links appeared first on Marginal REVOLUTION.

       

Comments

 

Don’t Monetize Me, Bro!

Can you fill in the blank?

They don’t love me for who I am. They just love me for my….

Time’s up—everybody submit your answers. [Long pause while I read the responses.] Whoa, what’s wrong with you folks? Are you’re living inside a David Lynch movie?


Please support my work—by taking out a premium subscription for just $6 per month.

Subscribe now


The correct answer to that question today is much tamer than your responses. In the year 2026, they only love you for your data.

Those tech bros smile at you. They pay compliments. They serve up flirtatious innuendoes. But don’t get fooled. They just want to canoodle with your bits and bytes.

That is what intimacy amounts to in the present day.

But even data is starting to resemble a David Lynch film. Amazon is now compiling highly inappropriate profiles on users—that’s how Sarah Esterman learned that she has a bad father.

Another customer was told that she has “flat buttocks.” She doesn’t deny it—but do we really want Amazon monetizing these intimate details?

With this in mind, let me update my previous statement: “Tech bros only love you for (data about) your flat buttocks.”

That’s a kink too far for me, my friends.

This controversy made me curious about my own Amazon footprint. I’m trying to shop there less often nowadays, but they still know way too much about the Honest Broker. So, with some trepidation, I tracked down my profile. Here are the most revealing details.

Read more

Hegseth’s Minions Break Eisenhower’s Globe

U.S. Defense Secretary Pete Hegseth wanted a 30-inch floor globe used by Dwight Eisenhower during World War II moved to his office. Rather than wait for specialist movers, his staffers tried to move it themselves,… More

A Google Maps Roundup

Whatever you think about generative AI—and you know what I think—you have to concede that it isn’t very good at making maps out of the box: there are so many examples of it going horribly… More

Rethinking networks in space with Aalyria’s Brian Barritt

In this episode, SpaceNews’ Mike Gruss talks with the CEO of Aalyria Brian Barritt about how networks are evolving, what scaling the company means and the role of software interoperability […]

The post Rethinking networks in space with Aalyria’s Brian Barritt appeared first on SpaceNews.

An Alright Week for D.C.’s Crime Stats

As of 9am Friday, there was one homicide in D.C. this week. To date this year, D.C. has reported a total of 79* homicides this year (there was one more murder reported from May of this year). Last year, during the same time period, we had 113 homicides, and in the surge year of 2023, there were 224 homicides during that same time period.

Both violent and property crimes trended upwards this week.

While we’re still off pace for another 33% annual reduction in homicides, the first time since the last year of the Biden Administration, this week had a decline in homicides from four to one.

Hoping for a zero homicide week next week.

*Three of the 82 murders reported this year actually occurred in other years (e.g., a missing persons case from 2023 turned into a homicide case this year with new evidence).

Prediction Markets Are Accurate

No surprise to me but a new paper looks at the forecast accuracy of over one million prediction questions from Polymarket, Kalshi, Manifold and the forecasting platform Metaculus. A virtue of the new analysis is that it covers many different areas from business to elections to geo-politics. An event predicted to happen p% of the time happens about p% of the time. Can’t beat that.

Using approximately one million resolved contracts and questions across four collective forecasting platforms, twelve domains, and time horizons ranging from hours to years, we evaluate the calibration and accuracy of collective forecasts in real-world settings. We found that forecast probabilities were overall highly calibrated: events assigned a probability of p% occurred approximately p% of the time. This broad pattern was evident across platforms and many domains, although fine-grained analyses revealed systematic tendencies toward over- or underestimation in highly consequential areas. While this does not mean that forecasting platforms provide certainty about individual events, our findings suggest that large-scale aggregation of judgments contains meaningful information about real-world uncertainty; when examined in aggregate, the predictions generated by the platforms provided informative estimates of event likelihood.

The post Prediction Markets Are Accurate appeared first on Marginal REVOLUTION.

       

Comments

Related Stories

 

Protecting Americans from Medicine

Max Doppelt: In January 2025 my mom’s lung cancer spread to her brain. The drug with the best chance of helping her was approved in China but not here.

A doctor told me, almost as an aside, that if she were Chinese she could buy it at a pharmacy.

So I flew to Shanghai and came home with 26 boxes in my luggage. That was 19 months ago. My mom is still here, still working, and in her words has had an amazing three years.

The drug was approved in China in 2023. The FDA approved it in July 2025. It still is not for sale here. I know of at least 30 Americans who have made the same trip.

I have been scared to tell this story because I worried it could cost my mom access to the medicine keeping her alive. But the problem matters more than my silence. These drugs need to be developed and available here, and that means FDA reform.

A piece at KFF Health cover’s Max’s story and adds this insanity:

Rep. John Moolenaar (R-Mich.), who chairs the House Select Committee on the Chinese Communist Party, in June co-sponsored legislation with Rep. Debbie Dingell (D-Mich.) requiring all U.S. pharmaceutical deals with Chinese companies to be cleared by the Treasury Department.

Got that? A son flew to Shanghai to buy pharmaceuticals to extend his mother’s life and the House Select Committee on the Chinese Communist Party wants a Treasury lawyer monitoring the baggage claim. Yes, I am angry.

Addendum: See also my earlier post on Operation TrialBlazer and Montana’s SB535. My 2009 TED Talk is the ur-source for my argument that a rich China and India will create many benefits for the United States, including many more cancer drugs.

The post Protecting Americans from Medicine appeared first on Marginal REVOLUTION.

       

Comments

Related Stories

 

Traction!

Video of a match striking against a textured surface creating a small flame with a dark background.

Tribologists study friction: what happens when things rub together. It’s indispensable, from the wheel to the paths of stars

- by Jennifer R Vail

Read on Aeon

Podcast with Rabbi David Wolpe

Here is their episode summary:

Economist Tyler Cowen joins David Wolpe for a conversation about AI, religion, and what makes us human. Tyler makes the case that AI’s biggest impact may be on status: who gains it, who loses it, and what happens to experts like rabbis when anyone can ask ChatGPT all their questions. He also suggests that losing our monopoly on intelligence may make us feel more unique as humans, rather than less.

You can listen here.

The post Podcast with Rabbi David Wolpe appeared first on Marginal REVOLUTION.

       

Comments

Related Stories

 

"I'm more worried about democracy than artificial intelligence": Italian press on Moral Economics in advance of the International Festival of Economics

In the run-up to the  International Festival of Economics in Turin starting today, Vanity Fair in Italy publishes an interview with me under this headline:

Alvin Roth: «Sono più preoccupato per la democrazia che per l’intelligenza artificiale»

 "I'm more worried about democracy than artificial intelligence" 

 

The Festival itself publishes this article:

ROTH E LA SFIDA DEI MERCATI RIPUGNANTI  ROTH AND THE CHALLENGE OF REPULSIVE MARKETS 

 

And here's an essay I sent in English, translated into Italian, which you can translate back pretty well:)

Eco­no­mia in cerca di morale
Esi­stono mer­cati con­tro­versi (alcol, gioco d’azzardo, utero in affitto...) su cui i vari Paesi hanno posi­zioni diverse, riflette il Nobel Alvin E. Roth: non sem­pre si arri­verà al con­senso col­let­tivo, ma è lecito esplo­rare com­pro­messi La Lettura
Di ALVIN E. ROTH (tra­du­zione di Rita Bal­das­sarre)

Economy in search of morale
There are controversial markets (alcohol, gambling, surrogacy...) on which the various countries have different positions, reflects the Nobel Prize winner Alvin E. Roth: collective consensus will not always be reached, but it is legitimate to explore compromises 

Will bystander economies be able to finance their governments?

From Jesús Fernández-Villaverde:

Will AI poison the public finances of many modern states? There is a real risk that it will.

Yesterday, I circulated a new paper, “Saving for the Machines,” which makes a simple point. Building the AI transformation will require enormous investment. The only way to induce agents to save for that investment is through a large increase in the real interest rate. For instance, in my benchmark scenario, the ten-year real yield jumps from 5.1% to 13.6% on impact.

But that means government borrowing will become much more expensive. Here is a back-of-the-envelope calculation. Take a government that owes 100% of GDP in 2025 (a typical case in Western Europe) and runs a balanced primary budget from then on (not so typical but never mind; I want to build an optimistic scenario). That means the debt ratio changes only with the gap between the real interest rate and growth.

I input the interest rate and the growth rate for my benchmark transition to full automation, and I roll over the debt at the short rate. I consider two economies. The first economy is an AI adopter, for example, the U.S., which experiences an AI boom at home. The government pays the new interest rate but also enjoys faster growth.

The second economy is a bystander, for example, Spain. This economy lives in an integrated financial market but does not adopt AI (this is an extreme case of zero adoption, just to make the exercise transparent): it pays the same world interest rate and continues growing at its old 1.6% trend.

Let’s first consider the adopter. With an intertemporal elasticity of substitution of one-half, the Euler equation implies that consumption growth is about half the real interest rate. Hence, the gap between the interest rate and growth does not close during the AI-driven growth boom. Instead, it widens. For example, with a unit elasticity of substitution among tasks, the real rate rises from 5.1% to 42% by 2040, while growth increases only from 1.6% to 22%. Debt doubles by 2038, reaches 296% of GDP in 2040, and 907% in 2045. To hold the ratio at 100%, the government needs a primary surplus of 10% of GDP in 2035 and 24% from 2045 onward. With a low elasticity of substitution of 0.2, the numbers are milder, but the pattern is the same: 204% in 2040, 397% in 2045, and a stabilizing surplus of 13% of GDP.

But the adopter has a way out. The capital share goes to one, so net capital income is about 88% of GDP, and a flat tax on it can cover the surplus. A 36% tax on net capital income holds the debt at 100% of GDP in the limit with unit elasticity, and a 20% tax does so with strong complementarity across tasks. The tax is above the gap because it slows growth, from 21.9% to 13.6% with unit elasticity and from 11.7% to 9.1% with an elasticity of 0.2, while it does nothing to the rate the Treasury pays, which technology pins at the machines’ net return. Painful, but the boom survives and the adopter can pay its debt.

The grim news is that the bystander economy has no way out. Its capital income is still a third of GDP, which grows at 1.6%, and the surplus it needs is 11% of GDP in 2030, 26% in 2035, and 44% forever with unit elasticity (18% and 24% with an elasticity of 0.2). On a balanced primary budget, its debt doubles by 2033 and triples by 2035. No tax system raises that.

Two more bad pieces of news for the bystander. First, long yields jump on impact, so a Treasury that issues ten-year debt pays more, and sooner, than rolling over at the short rate assumes: it is the same bill but front-loaded. Second, the bystander’s own capital flows to the machines abroad, so its GDP and tax base fall below trend. One relief is that households can save for other people’s machines and get rich on the returns, which the bystander’s government can tax to some extent. This back-of-the-envelope calculation is a benchmark, not a forecast of what is to come. But the arithmetic is simple, and it runs counter to the hope that an AI-fueled growth boom will take care of public debt, because growth comes with an interest rate that is twice as high. Not a pretty future for most of Western Europe. Due to AI, the choice is between default, inflation, or, most likely, financial repression: capital controls that keep domestic savers from investing in machines abroad.

I am somewhat more optimistic than this.  The assumption of zero Spanish adoption already is noted, but also ideas are public goods, and Spain can benefit from American (and Chinese) AI, and the resulting scientific advances.  The long-run supply of capital also might be quite elastic, we will see.

The post Will bystander economies be able to finance their governments? appeared first on Marginal REVOLUTION.

       

Comments

Related Stories

 

Apple Highlights 50+ iPhone Apps That Have Already Been Optimized for the Duo

Apple Newsroom:

New editorial collections for iPhone Duo apps and games will go live on the App Store this Friday, with additional app updates rolling out throughout the year. Apps optimized for iPhone Duo are easy to identify with a Made for iPhone Duo badge on product pages across the storefront.

This story highlights dozens of apps that have been optimized. This sort of transition happens every time Apple introduces a major screen change. It happened with the iPhone X in 2017, it happened when the iPhone 6 added a Plus size, it happened when the iPhone 5 changed the aspect ratio from 3:2 to 16:9, and, most memorably of all, it happened when the iPhone 4 went retina. Every time this happens it sucks to some varying degree of suckiness to use apps that haven’t been optimized. And soon enough, all apps still under active development get updated.

Developers know that iPhone users care how apps look and work. iPhone users will switch to competing apps for something like this. I stopped checking Mastodon on my phone for the last week and half because Ivory isn’t optimized. I stopped reading my messages on Signal — I just wait until I’m back at my desk. If I bought a Duo, I’d literally stop using Signal if they don’t update their app soon. Life’s too short to use shitty apps. iPhone developers get that.

Out of all those transitions I listed above, my gut feeling is that there are more third-party apps ready to go on day one for the Duo than for any of those previous transitions. No, it’s not all apps. Some of the apps that aren’t yet updated are quite popular, and others are ones that users can’t easily just switch away from (like, say, your bank’s app). But developer support is not a problem for the Duo. If anything the launch of the Duo has exemplified better than any event in iPhone history just how different the iPhone and Android developer communities are. “Flagship” Android foldables have been out for over 7 years and developer support for them today remains nothing close to developer support for the Duo — and the Duo doesn’t even launch until tomorrow.

 ★ 

New Developments in the Expanding “Wedgewood” Polling Scandal!

I’m not sure where to start with this. The latest news is some mix of Webistics, Jukt Micronics and Animal House. But here goes. Yesterday we noted that a FiftyPlusOne investigation had led to the outing of “Wedgewood Polls” as a publisher of entirely fake polls. It’s still unclear the precise motives, but some political narrative setting aim and gaming betting markets seem like good bets. The details are wild. (But see the earlier post or FiftyPlusOne’s write-up for more.)

Early this morning I woke to a short item in The Washington Post noting another pollster, SoCal Strategies, had straight up disappeared. By disappeared it meant that overnight its website, Substack and Twitter account had all vanished. In the earlier write-up of the investigation of Wedgewood, FiftyPlusOne had noted that one of the ways it had initially vetted Wedgewood was speaking to people at another polling firm where one or more of the Wedgewood guys had worked. I can’t remember if I’d already heard SoCal’s name come up in this story. But when I saw the Post item I immediately knew it had to be them. Yes, fake polls are a problem. But stuff like we’re seeing here is super rare. They had to be connected.

It turns out that this too was almost certainly prompted by the FiftyPlusOne investigation of Wedgewood. (Here’s their followup piece on SoCal.) They’d apparently reached out to SoCal in the process of investigating Wedgewood. But as the Wedgewood thing blew out they went radio silent. And now they’ve disappeared entirely.

If any of you remember the Stephen Glass saga and Jukt Micronics, a lot of the latest round of this story feels like that. It’s not clear that a number of the named people involved even exist. And in the cases of people who FiftyPlusOne seems to have verified do exist, it seems like people may have been impersonating them. So like a latter day Stephen Glass, it’s quite possible that both of these companies and all the people involved may be one dude in his basement with a half dozen burner phones.

A slight wrinkle to the story is that, reading between the lines, it’s possible that SoCal may have started out as a legitimate operation and then been taken over by fraudsters. But that’s still unclear.

The surest bet for this whole thing still seems to be Jack Sims, a Republican operative from Florida who is a prolific operator on right-wing spaces. But we probably need to be open to the possibility that he may not actually exist.

2027, Checks and Balances and Court Reform

Here is one of those things we can see on the horizon and it will be important to be ready for. It will become especially important if Democrats take both the House and the Senate.

It’s a given that even with only the House under Democratic control, we’re in for a period of pretty intense inter-branch conflict. Just how intense, whether Democrats will fight as much as their partisans demand … those things we can’t know. But this basic feature of the next Congress is pretty clear. What I’d like to discuss here is a wrinkle in that fabric that hasn’t gotten enough attention.

We know that the corrupt majority on the Supreme Court has been busily dismantling the constitutional order and empowering the second branch of government (the executive) over the first (congress). But that fact has been significantly obscured by the fact that the president’s party controls Congress. And it’s more than just that Republicans control both chambers. The current Republican party is subservient to Donald Trump to a degree unknown in modern American history and arguably at any time in American history. So to a great degree the first branch is being run from the second branch. You don’t clearly see the impact of the Court handing the power of the purse to the president because the Congress already gave it to him. Or, more specifically, you don’t see a lot of fighting over it. You don’t see the topic getting litigating in the court of public opinion and political action, or Congress trying to reclaim those powers. So we don’t yet how much additional law the Corrupt Court would make to prevent Congress from wrestling some of that power back.

In a way, maybe this just comes under the heading of “everything sucks.” But we can be more proactive and forward thinking than that. No form of Court reform can take place before 2029 at the earliest, and then only with a Democratic trifecta. So no matter what happens on November 3rd, 2027 and 2028 are together the window of time for making the case for Court reform, building public support for it, most especially making a public argument that the Court’s corruption isn’t just about technicalities of independent agencies or something called impoundment.

The issue of abortion rights and the Court’s role in it is heavily politicized, the issue of money in politics and Citizens United is, but to a much lesser extent. Callais and the protection of Black voting rights and political power is heavily politicized in the Black community but much, much less outside of it. The point is that while the public’s approval of the Court has nosedived in recent years, Court reform is little known or understood outside a fairly small slice of the electorate. To make court reform electorally viable you have successfully argue that it affects the lives, pocket books, safety and freedom of average voters.

That’s the challenge that has to be taken up in the next two years, no matter what the outcome of the election is. But if Democrats capture both houses of Congress, a lot of that public argument should take place, will inevitably take place, as part of the fight over the near-dictatorial power the Corrupt Court gifted to Donald Trump.

And yes, it’s Donald Trump. Because while other Republicans would likely get some of the benefits, all these decisions were made under the majority’s “when it’s Republicans” doctrine.

One final point: the point of Court Reform is to reform the Court. To put it more specifically, it is to break the power of the corrupt majority which is making war on the federal Constitution and restore the constitutional order. But that’s not the only aim. Inclusive of that effort is the goal of shaping the Court’s behavior, providing an incentive to bring itself back within the dictates of the Constitution to forestall Court Reform. This doesn’t mean it’s a bluff. Speaking just for myself but I suspect others as well we should put all our energies into reforming the Court and do so no matter whether or how its behavior changes. But we can’t know the future. We can’t know how quickly robust Democratic majorities can be assembled in its favor. We can’t know whether there will be a Democratic trifecta in 2029. So we are doing both. Because the core driver of the Court’s behavior is the majority’s belief in its own impunity, in the unreviewability of its own actions.

New Serious PHP Vulnerabilities Released and Patched

PHP maintainers have released fixes for two critical vulnerabilities:

PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35 fix two critical vulnerabilities that can allow unauthenticated remote code execution. All PHP users are strongly encouraged to update immediately.

Who is at risk: all PHP installations serving web requests, including PHP-FPM, Apache with mod_php, php-cgi, FrankenPHP and LiteSpeed. We consider the risk highest for mod_php, with a demonstrated exploit where an unauthenticated request can trigger the attack before any application code runs.

The vulnerabilities:

  • RCE via OOM corruption, CVE-2026-109618 (GHSA-w8hw-w9fx-j79w): out-of-memory handling can interrupt PHP while internal request data is only partially initialized. Request cleanup may then treat the invalid data as valid.
  • RCE via stale global state, CVE-2026-109619 (GHSA-6wmf-4rv5-p3rp): request-owned internal state can survive after its backing memory has been freed. A later request handled by the same worker may use that stale state before normal initialization resets it.

Updating PHP is the only way to regain security. Do it now.

More news.

Good explanations.

Brazil’s weird economy needs serious reform

Neither Lula nor Flávio Bolsonaro have the stomach for it

AI compute is the new oil—and needs a new market

Financial firms race to create instruments to trade processing power

Sam Bankman-Fried says he has paid his debts

The crypto tycoon speaks with The Economist from prison in California

New research tries to defuse the “China shock”

An almighty fight over an influential finding

Links 10/22/26

Links for you. Science:

Controversial US-backed vaccine trial in African newborns halted
How Bad Is Another Case of Covid?
NSF closes fiscal 2026 with over $1 billion left unspent
Marine life dramatically declined in Gulf of Mexico in years after BP oil spill, study finds
Start-Up Claims to Use A.I.-Created Viruses to Treat a Patient (the corporate press release doesn’t actually describe if the treatment worked though)
A simple touch can bring dying corals back to life in just 7 weeks
‘Back at Square One’: Christina Applegate Says Her M.S. Was Misdiagnosed

Other:

Why I wish you didn’t want to see Nathan Fielder’s Elizabeth Holmes documentary. Holmes’ compelling lies, and why we should all, perhaps, just look away
‘Reverse-Engineer Anything’ Is the New Wild West of AI
Musk and Thiel are Evil Demagogues. What Sustains Them?
He’s Adapted to a Wheelchair. He Wishes D.C. Would, Too.
U.S. officials privately advise states fearing Trump election meddling
Progressives Are Applying The Wrong Litmus Test. The proximate obstacle is corrupt Republicans, and disempowering them requires democracy reforms, not a sufficiently bold agenda.
Dem Candidate In Arizona: My Rape Was Not A ‘Hoax’ Concocted for Votes
JD Vance Will Now Hear Your Petition to Remain
Could Mississippi Elect Its First Black Senator Since Reconstruction?
Musk’s Fake Flying Car And ‘CEO Said A Thing!’ Journalism
Online forum controlled by Nevada GOP House candidate contains slurs and Nazi talk, AP review finds
Republicans Drop Holy Sh*t Ad Featuring Amy Acton’s Estranged Pedophile-Marrying Mother
Recipe Scammers Are Using an AI-Generated Emily Blunt as a Slop Puppet to Promote Their Garbage
What Makes Men in Groups Behave So Badly?
How Corruption Got This Bad. Over the course of decades, the Supreme Court has legalized behavior that is poisonous to a functioning democracy.
There Is a Zionism That Needs the Support of Progressives
Inside the Conservative Takeover of the University of Texas
Putin’s American friends: ‘Realist’ critics of Nato echo Moscow’s talking points on Ukraine
Computer Chips Are the Most Magnificent Things Humanity Has Ever Made
Prediction markets are now ‘bespoke insurance’ for businesses, and a Los Angeles restaurant group’s $60,000 hedge on Kalshi is the latest example
94 Year-Old Electronics Billionaire Accused of Hiring Goons to Infest Chicago With Huge Rat Swarm
Conservatives are turning on Mar-a-Lago face
The State Department Is Pushing Out Senior Foreign Service Officers
The 75-year-old screenagers
Trump Pardoned Venezuelan Billionaire After Family Offered Alina Habba $50 Million Deal
We Got This
D.C. tenants say Buwa Binitie runs an empire of neglect. Now, they’re fighting back.
Former D.C. Attorney General Crashes Protest to Defend Landlord
‘I literally just leave my house’: In this Utah neighborhood, homeowners say a noisy data center campus is disrupting their lives
Dinner under $15? It’s possible at these 7 D.C. spots

How Long Do Wealth Shocks Persist? Less Than Three Generations in England, 1700–2026

What happens across generations to wealth shocks? Do they endure and even magnify, or do they dissipate? By implication, how much modern wealth is attributable to events before 1900? This paper uses random shocks to family size in England before 1880 that created wealth shocks for the children to measure the persistence of wealth shocks. Fertility in England before 1880 was a biological lottery. For richer families, family size strongly influenced child wealth. This paper finds that such biology-induced wealth shocks had no impact on descendant wealth by three generations later. Since wealth itself persisted across more than five generations this implies that, in the long run, wealth mainly derives from sources other than inherited wealth. The observed link between nineteenth century and modern wealth does not lie in wealth transfers alone. Wealth persisted because of the familial inheritance of behaviors associated with wealth accumulation and retention.

That is from a recent paper by Gregory Clark and Neil Cummins.  Via the excellent Kevin Lewis.

The post How Long Do Wealth Shocks Persist? Less Than Three Generations in England, 1700–2026 appeared first on Marginal REVOLUTION.

       

Comments

Related Stories

 

Thursday assorted links

1. The cherries trade.

2. New YouTube channel for Indian architecture.

3. Where do Math Olympiad gold medal winners end up?  “By total gold medal count, China is the clear leader, followed by the pack of USA, Russia, and South Korea. Over 20 years China has 105 out of a possible 120 gold medals which means nearly all their students achieved gold. This is so successful that it’s common knowledge that it is harder to make the China IMO team than to get a gold medal in the IMO itself.”

4. Is El Salvador cost-competitive?

5. Rebecca Lowe on Bach.

6. Do the Democrats still care about the Ivy League?

The post Thursday assorted links appeared first on Marginal REVOLUTION.

       

Comments

 

Anti-Vaccination Could Hurt Republicans at the Polls

At least two doctors who are former Republican congressmen think so (boldface mine):

If current polling trends hold, Republicans could lose their majorities in both the House and Senate this November. Most pundits attribute this potential blue wave to voters’ frustration with gas prices and the broader cost of living. But another issue is quietly hurting GOP candidates: our party’s often mixed messaging on vaccines…

Voters have certainly noticed — and they’re blaming Republicans. According to a recent KFF poll, more than 60% of voters now disapprove of the Trump administration’s handling of vaccine policy. Only 28% of independents approve. Democrats now hold a 16-point advantage on the issue: Forty-one percent of voters trust them more to handle vaccine policy, compared with 25% who trust Republicans…

President Trump has been a strong voice for vaccines, affirming that childhood immunizations work and encouraging parents to vaccinate their children. But he has promoted vaccine misinformation in the past, and in August, he signed an executive order to slim down the childhood vaccine schedule, and at the signing, suggested a link between vaccines and autism that decades of research have ruled out.

Meanwhile, the Department of Health and Human Services has thrown a key vaccine advisory panel into turmoil, threatened government vaccination programs, and downplayed the danger of measles — all at a moment when Americans are looking to their government for clarity…

Our party’s own data warned us that backsliding on vaccines would hurt our prospects. In 2025, the Republican polling firm Fabrizio Ward published a memo finding broad bipartisan support for routine childhood vaccines and warning that undermining them would cost votes.

The numbers are stark. Seventy-four percent of Americans say ensuring broad availability of Food and Drug Administration-approved vaccines should be a priority. Nearly 70% say they would be more likely to vote for a congressional candidate who vows to preserve vaccine access over one who seeks to rescind it. And crucially, more than 50% of voters say they’d be less likely to support a candidate who doesn’t support vaccines, even if they align with that candidate on other issues.

In other words, silence on vaccines is becoming a dealbreaker for voters who would otherwise vote Republican.

For a very long time, Republicans have been able to spout stupid rhetoric in many areas and thereby build out a Coalition of the Stupid, but now that they’re actually turning that rhetoric into action, they are finally starting to suffer for it politically.

Couldn’t have happened to a nicer bunch of assholes.

Modern Principles of Economics!

If you are teaching economics Modern Principles is the book to use! Great and original writing, over one hundred videos, iClicker, Achieve and lots of additional support. Contact Mike Krotine to get an examination copy.

The post Modern Principles of Economics! appeared first on Marginal REVOLUTION.

       

Comments

Related Stories

 

Bhutan: the last place

Photo of a boy in a sports jacket, looking thoughtful with his reflection in a dimly lit mirror behind him.

What happened when people in Bhutan, the last country on Earth without television, were allowed to watch it in 1999?

- by Aeon Video

Watch on Aeon

Other people

A group hug in a shed doorway, with people embracing closely. Blue walls, metal roof and a door are visible.

Disconnection is at the heart of today’s crises. So my husband and I chose a radical and revealing remedy: living communally

- by Elizabeth Oldfield

Read on Aeon

Market design and water conservation in the the Great Salt Lake

Silvia Console Battilana, who co-founded Auctionomics along with Paul Milgrom, writes in Forbes about the prospects for market design efforts to help conserve Utah's Great Salt Lake. 

How Market Design Could Help Save Utah’s Great Salt Lake by 
By Silvia Console Battilana

 "the Great Salt Lake is drying up.

"To save the lake while protecting water users, business leaders can adopt a little-known but powerful economic tool: market design. When water levels drop, the consequences immediately hit commercial supply chains and local commerce. Market-based incentives give businesses a way to reward upstream water conservation, bringing more water to the lake without hurting regional growth. The invisible hand of the market could identify the least expensive conservation, compensate participating water users and restore the Great Salt Lake.

...

"The shrinking of the Great Salt Lake has many causes, and there is no silver-bullet fix. But any solution to refill the lake will likely involve conservation within Utah. Seventy-one percent of the water diverted from the lake’s basin is allocated to agriculture. If those commercial users could make more efficient use of water, more would reach the lake.

The catch is that many conservation practices, like water-friendly infrastructure and precision irrigation, are expensive. Under Utah’s traditional “use-it-or-lose-it” water rights system, users have little financial reason to conserve. Without modern economic incentives, asking businesses and farms to voluntarily cut back simply isn’t realistic.

Even if water users could be forced to cut back, it’s not clear who should cut back, or by how much. The costs of water conservation aren’t the same across users, and they can be very hard to estimate.

Market design can address both of these challenges. Crucially, markets don’t force particular actions or behaviors. Instead, well-designed markets create an incentive for farms and communities to cut back on water and be rewarded for their efforts. Anyone who can cost-effectively reduce consumption will do so, while others aren’t compelled to conserve if it would be too expensive.

Market Design Works
Even when the stakes are high and a lot stands in the way, intentionally designed markets can successfully align self-interest with the public good.

Further, market design can be used to align incentives even when money isn’t exchanged: it has been used for organ donation exchanges and food banks. Harnessing the invisible hand of the market to align incentives in pursuit of a common goal is an effective tool for difficult problems."



 

 

A simple point about the intertemporal regulation of AI

At some point AI will be more regulated than it is today.  Furthermore, I believe that most parts of the American economy, once they are regulated, end up overregulated.

Thus it is likely that the AI sector will, at some point, end up overregulated.

Given that reality, perhaps the AI sector should be underregulated today.  After all, overregulation will have decades, maybe centuries, to catch up and even the ledger.

Many regulatory issues can be considered within this framework.  You can think that AI is “underregulated today”, in the static sense of that term, and still not favor additional regulation right now.  You might prefer to keep it underregulated for some while to come.

The post A simple point about the intertemporal regulation of AI appeared first on Marginal REVOLUTION.

       

Comments

Related Stories

 

Do mass shootings in universities lead to permanent effects?

Using a stacked difference-in-differences design covering shootings between 1990 and 2025, I compare cohorts at affected universities before and after the shooting with cohorts at matched control universities. I find no detectable effects on time to degree, but undergraduates enrolled at the time of the shooting are 4.2% less likely to pursue further studies. Those who do continue shift toward health: the share of exposed cohorts who later enroll in a health degree rises by 1.3 percentage points (8.8%), and this effect persists among cohorts who enroll after the shooting. Effects on students present during the shooting are consistent with a response to trauma; for later cohorts, I find evidence of sorting, as master’s students with prior healthcare experience become more likely to enroll in affected universities. These results suggest that mass shootings leave institutional and cultural legacies that outlast the students who witnessed them.

That is from the job market paper of María Medellín-Esguerra, who is currently on the job market from Brown University.

The post Do mass shootings in universities lead to permanent effects? appeared first on Marginal REVOLUTION.

       

Comments

Related Stories

 

Updates to My iPhone Duo Review

Nothing major, but since publishing at 7:00pm last night, I’ve made a few additions and changes to my iPhone Duo review. I’m calling them out here, so that if you’ve already read it, you can see the changes. If I make more, I’ll add them to the top.

Regarding the camera, added and/or clarified the following (in bold):

As for the camera itself, it’s terrific. It’s clearly not of the same caliber as the iPhone 18 Pro, but it’s good. The technical specs of the Duo’s main 1×/2× camera are nearly identical to those of the year-ago iPhone Air, and I love the iPhone Air’s camera. If anything, the Duo’s camera should be better, because it gets the year-better A20 Pro imaging pipeline. Which, notably, brings support for the same new Photographic Styles, including textures like film grain, that were introduced with the iPhone 18 Pro. However, the Duo is not “Pro” — it does not get camera features like “Pro controls” (variable lens aperture, shutter speed, white balance, histogram), Apple Reference Image, spatial video, ProRES (video or RAW), or Apple Log. Apple’s “shot with iPhone” keynotes will continue to use iPhones Pro.

Also, notably different from the Air, the Duo has an ultrawide 0.5× second camera lens. This brings both the ultrawide perspective and macro photography. Like the Air, the Duo’s main 1× camera has a minimum focus distance of 15cm. That’s closer than the 17 and 18 Pro cameras (20cm), but there are a lot of times you want to snap photos of receipts or documents closer than 15cm. With the Air, you can’t focus closer than 15cm, because it only has that one camera. With the Duo, you can focus on subjects down to just 2cm away, using the 0.5× lens or the 1× lens in “macro” mode (which switches to the ultrawide lens but sensor-crops to the 1× field of view).

Regarding the security of “Unlock with Apple Watch”, I added this note:

(It’s worth noting that the wrist tap on your watch for this gives you a button to tap to lock your iPhone immediately — it’s not just a notification that it happened, it presents you with an action to undo it.)

Added this note (in bold) about it occasionally rotating (uncomfortably) in pocket, because it happened to me again twice today:

It does not go into or out of my jeans pocket smoothly. Sometimes, in my pocket, it rotates 90 degrees, which is uncomfortable. Folded up it never stops feeling heavy. It is heavy, compared to other iPhones.

Added this updated regarding the janky crossword keyboard in the NYT Games app on the Duo’s inner screen:

[Update, Thursday: One day later and there’s an updated version of the NYT Games app, and they’ve fixed the orthodontic mess of their crossword keyboard, in both orientations. Screenshot. This goes to show just how fast the Duo-optimized updates are streaming in from developers. My review was up-to-date when I published at 7:00pm ET Wednesday, and outdated by the time I woke up Thursday morning.]

 ★ 

Heavy Rainfall This Weekend in the Southern Plains and in South Florida; Bering Sea Storm in Alaska

How to install an app on your Commodore 64

It’s time to install an app! But here in 1986, we still call them “programs,� not apps. So, you’ll want to set aside some time to get started installing a program.

Since you have a Commodore 64, there are really only two options for getting new programs this month: COMPUTE!’s Gazette and RUN, and you have to hope that one of those magazines isn’t sold out at the book store. So let’s get going.

Go To The App Book Store

You will need to ask your mom if you can ride your bike to the mall. The dirt mall is closer, but the magazine stand at the dirt mall doesn’t carry any computer magazines, so you’ll need to take a longer ride to the good mall, where the book store actually has computer magazines. It’s a couple of miles to the mall, so usually you would have a friend come with you, but you called their house and nobody picked up, so today it’s a solo trip.

Only one book store carries computer magazines at this mall; the other one only carries magazines about “serious� topics, but they can order back issues of a magazine for you if you want, you just have to wait a few weeks and they’ll call you when it comes in.

The guy at the book store doesn’t know the difference between the different computer magazines, so you can’t ask him if they have the one you want in stock, and he’s probably not going to answer a kid anyway. So you just have to go through all the titles one by one, and hope nobody put a copy of People or Omni in front of it by accident.

There are no ratings, and no reviews of the programs themselves. The closest thing you get is the magazine’s own reviews of hardware — modems, mostly — and a cover that promises you the issue has the thing you want. This month’s cover has a “Buyer’s Guide To Modems� banner across the top.

The cover of COMPUTE!’s Gazette, March 1986 (issue 33), with Survivor as the lead feature and a “Buyer’s Guide To Modems� banner across the top

Once you’ve got a copy of COMPUTE!’s Gazette and you’re checking out at the store, you can see that it’s three dollars for the magazine (about nine bucks in 2026 money). If you’re lucky enough to have a five dollar bill, you’ll still have two dollars left over after buying the magazine. That means you’ll have enough money to swing by the record store later and grab a 45 of one of the new singles, like Sheila E.’s “A Love Bizarre,� written by Prince, or The Bangles’ “Manic Monday,� written by Prince, or Prince and the Revolution’s “Kiss,� written by Prince.

Pick Your Program

Now it’s time to race home with the new magazine in hand. You’ll have to wait for everybody to get off of the TV that’s by the computer so you can switch it over to the channel the computer is on and get down to work.

Turn on the switch, and you’re at the blinking blue cursor where it says READY. while the screen is still warming up a bit.

When it comes to installing apps programs, it’s all about options, and this month, we’re spoiled: there are two! They’re both games: Survivor, an arena game where, “if you’re quick of mind, eye, and hand, you might outfox your opponent,� and Shifter. Truly, an embarrassment of riches. Since Survivor is the one on the cover, we’ll go with that one.

(Two or three programs a month is pretty typical. That’s… the app store.)

Type It In

Getting this program running is standard stuff — you just read the article on page 36 that describes the game, and then flip to pages 106 and 107 where there’s the code for a short BASIC loader, and then continue on to pages 107 and 108, where there are 345 rows of densely-packed listings consisting of 9 hexadecimal character pairs, all of which have to be typed in perfectly with zero errors or else your entire game will refuse to run.

A quick sample of the early part of your task looks like this:

The start of the machine language listing for Survivor, from COMPUTE!’s Gazette, March 1986, page 107

After you’re done typing that page in, you’re about a quarter of the way done, assuming you didn’t make any mistakes.

Wait, what’s that? Where are you supposed to be typing these in? Simply type them into the MLX program where you… wait, what’s that? You don’t have the MLX utility that’s used to type in machine language code for games and programs listed in the magazine?

Oh, don’t worry, you’ll just need to type in the code for that program. There’s an article explaining what MLX is, which you can read really quickly on page 93, and then you’ll find the code for it on page 94. That one’s only 100 lines of code, so it should be easy enough to type in. And of course, you’ll know if you made any mistakes when typing, thanks to the Automatic Proofreader.

Wait, you don’t have the Automatic Proofreader program?

Well, no worries. It’s only another 31 lines of BASIC code, which you can find on page 97. (This one you have to type in perfectly on your own; there’s no proofreader for the proofreader.) Once you do that, you’re all set.

The Automatic Proofreader article and BASIC listing, from COMPUTE!’s Gazette, March 1986, page 97

Now, you can just use the program that lets you enter the program to make your program. It should only take about half an hour to type in the code for the Automatic Proofreader, and maybe another hour or two for MLX. Let’s call it two hours total between them.

After that, Survivor is ~30 lines of BASIC plus ~350 lines of hex code, so… maybe six hours? But at least MLX will catch any typos as you go.

Once all of this is complete, be sure to save your work to a floppy disk so you don’t have to type it all in again! If all you have is a cassette deck and you’re saving to tape, it’s going to take probably 10–20 minutes of staring at a blank screen when you save, so set aside some time for that.

In total, it should only take you about 8 hours to have Survivor ready to play, plus however long it takes you to get to the mall, buy a magazine, and come back.

Ready To Play

(Click or tap to get started.)

[This is Survivor by Kevin Mykytyn and Mark Tuttle, from COMPUTE!’s Gazette, March 1986, running in the VICE emulator. The game loads and runs on its own, which takes a few seconds.]

Controls: the yellow player uses W/A/S/D to move and F to fire; the cyan player uses the arrow keys to move and Enter to fire.

Next Steps

The Download

Someday, rather than typing all of this in, you might be able to use your 300 baud modem to download already-created versions of these games and tools. You could dial a local BBS, if there’s one nearby. If you’re unlucky, the only one you can reach is a long-distance call, and you’ll be paying by the minute for as long as you’re connected. Or you could look for a local dial-up number for the Quantum Link online service (which will eventually evolve into America Online!).

Transfer times can range from tens of minutes to hours, and of course during those calls, you are tying up what’s usually the sole voice phone line for the household. Anyone picking up a phone handset in the house threatens to knock your connection offline, forcing you to start the whole process all over again.

Customization

Now that you’ve enjoyed your game of Survivor, there are some fun ways to customize your little computer and give it a personal touch.

Just for context, here’s a to-scale comparison of the screen resolution of your Commodore 64 to the dimensions of an iPhone Duo’s unfolded screen.

Commodore 64 screen compared to the iPhone Duo inner display, at the same pixel scale A 2670 by 1878 pixel rectangle represents the iPhone Duo inner display. In its top-left corner, drawn at true size, is the Commodore 64 screen: 384 by 272 pixels including border, with a 320 by 200 pixel text area. The C64 fits about 48 times into the Duo display. On hover or focus, the C64 shows its READY. prompt and the Duo shows the 180 by 180 pixel App Store icon at the same scale. READY. Commodore 64 384 × 272 pixels (320 × 200 content area) Icon for scale 180 × 180 pixels iPhone Duo inner display 2670 × 1878 pixels
Hover over (or tap) the comparison to see a C64 prompt and an App Store icon at the same scale.

You can’t change the wallpaper on your Commodore 64, but you can easily change the background colors for both the center area of the screen and the border. All you have to do is manually assign the proper numeric color code to the correct memory register corresponding to the values of each of those settings.

Memory address 53281 will set the background color for the primary content area, and a color code of 1 sets a nice readable white background, so it only takes a simple POKE 53281,1 command at the prompt, and you’re all set. Similarly, POKE 53280,4 will make the border of the screen a nice bright purple color, and of course there are a full complement of 16 colors to choose from.

Naturally, none of this customization is stored anywhere permanent, and it will all be lost if you turn off your computer or run a program.

READY.

I hope you’ve enjoyed this brief overview of how to get one game running on your Commodore 64, and perhaps gained a little appreciation for how things have gotten a bit simpler over the years.

Also, I will be entertaining a motion to return to calling our software “programs� instead of apps once more. Perhaps we could use the term to distinguish the software that’s created by humans actually typing things in, as opposed to the kind that’s merely generated by machines?

Central Pacific Tropical Weather Outlook


Central North Pacific 2-Day Graphical Outlook Image
Central North Pacific 7-Day Graphical Outlook Image


000
ACPN50 PHFO 232300
TWOCP

Tropical Weather Outlook
NWS Central Pacific Hurricane Center Honolulu HI
Issued by NWS National Hurricane Center Miami FL
200 PM HST Fri Oct 23 2026

For the central North Pacific...between 140W and 180W:

Active Systems:
The National Hurricane Center is issuing advisories on Hurricane
Vance, located several hundred miles south-southwest of Acapulco,
Mexico.

Western Central Pacific:
An area of low pressure is forecast to form near the International
Date Line, several hundred miles west of Johnston Atoll late this
weekend or early next week. Environmental conditions could support
some slow development thereafter while the system moves generally
eastward to east-northeastward.
* Formation chance through 48 hours...low...near 0 percent.
* Formation chance through 7 days...low...20 percent.

$$
Forecaster Gibbs
NNNN


Atlantic Tropical Weather Outlook


Atlantic 2-Day Graphical Outlook Image
Atlantic 7-Day Graphical Outlook Image


000
ABNT20 KNHC 232301
TWOAT

Tropical Weather Outlook
NWS National Hurricane Center Miami FL
800 PM EDT Fri Oct 23 2026

For the North Atlantic...Caribbean Sea and the Gulf of America:

Tropical cyclone formation is not expected during the next 7 days.

$$
Forecaster Gibbs


Eastern Pacific Tropical Weather Outlook


Eastern North Pacific 2-Day Graphical Outlook Image
Eastern North Pacific 7-Day Graphical Outlook Image


000
ABPZ20 KNHC 232300
TWOEP

Tropical Weather Outlook
NWS National Hurricane Center Miami FL
500 PM PDT Fri Oct 23 2026

For the eastern and central North Pacific east of 180 longitude:

Active Systems:
The National Hurricane Center is issuing advisories on Hurricane
Vance, located several hundred miles south-southwest of Acapulco,
Mexico.

Western Central Pacific:
An area of low pressure is forecast to form near the International
Date Line, several hundred miles west of Johnston Atoll late this
weekend or early next week. Environmental conditions could support
some slow development thereafter while the system moves generally
eastward to east-northeastward.
* Formation chance through 48 hours...low...near 0 percent.
* Formation chance through 7 days...low...20 percent.

$$
Forecaster Gibbs