Broadsheet

	Before	June 25	Change
HomePod	$300	$350	17%
(Old) HomePod Mini	$100	$130	30%
(Old) Apple TV 64 GB	$130	$200	54%
(Old) Apple TV 128 GB	$150	$250	67%

This is the second half of the third part (I, II IIIa) of our long-delayed series looking at the theory of history advanced by Paradox Interactive’s Hearts of Iron IV, one of the most popular strategy games today.

This part (both halves together) is fundamentally about politics and policy and the implications that HoI4‘s choices – and its chosen omissions – have on the vision it presents of the history of the 1930s and 1940s. Last time, we looked at HoI4‘s political simulation and concludes that in an effort to streamline the political aspects of the game – to better facilitate the player getting to the ‘good stuff’ of fighting the war – HoI4 adopted a top-down, state-power-centered, frictionless vision of politics which was essentially authoritarian in nature. Democracy, meanwhile, was presented as something between meaningless and simply a collection of penalties. Such a vision of politics, we noted, is hardly required by the period, but is instead a product of the effort at simplification to ‘meme politics.’

For this second part, I must lead with a content warning, as we are going to be talking about mass murder, war crimes, intense human suffering and in some detail. I am going to include images, many of which you ought to find disturbing. While often in discussing history, there is value in remaining purely dispassionate it is also important sometimes to feel something in order to understand some portion of the import of a moment. I am thus including some difficult images because I want you to stop and feel something, to imagine the humanity of the victims, their suffering, their loss. If we do not feel something of the empty pit in our soul where these people should be, we have not fully absorbed the lesson.

Because of course the politics of the previous part did not merely happen on the abstract moving of numbers, they happened to people. Millions upon millions of people. It is, on some level, fundamentally disingenuous to discuss World War II and the countries that fought it, without discussing those millions. They were part of World War II – indeed, the experience of a civilian being brutalized, murdered or starved to death was a far more common, a far more typical experience of WWII than that of the soldier fighting on the front lines.

So we are going to discuss them and then we are going to discuss Hearts of Iron IV‘s decision not to discuss them. Buckle up, folks, this one is going to be rough.

Via Wikipedia, the defendants at the Nuremberg trials in 1945. As we will discuss below, these were hardly the only men guilty of war crimes and crimes against humanity as part of the Third Reich. Estimates, discussed below, suggest that at least half of all German soldiers on the Eastern Front were complicit (if not participants) in war crimes, which suggests that **at minimum** one in three German soldiers were war criminals (as about two-thirds of all German soldiers served at some time in the East); the true percentage is likely higher (as not all of the war crimes occurred on the Eastern Front), probably approaching an absolute majority.

(Bibliography Note: Estimates for the human toll of World War II and the crimes of the Axis regimes vary to a significant degree and the following overview does not purport to be an exhaustive review of estimates. Chronologically speaking, this is not my area of expertise and I have simply done my best to relate the scholarly consensus, as best I can. I have relied in the first instance generally on the estimates collected in the “The Geography of Death” chapter in J. Parshall, 1942: Crux of War (2026), 904-908. For greater specificity, on the conduct and implications of the ‘Hunger Plan,’ see relevant sections of A. Tooze, The Wages of Destruction (2006). On atrocities in Eastern Europe more broadly, see W.W. Beorn, The Holocaust in Eastern Europe (2025), T. Snyder, Bloodlands: Europe between Hitler and Stalin (2012) and A.J. Kay and D. Stahel, “Crimes of the Wehrmacht: A Re-evaluation” JPR 3.1 (2020). On non-Jewish victims of Nazi extermination efforts, especially Soviet POWs, note T.E. Porter, “Hitler’s Forgotten Genocides: The Fate of Soviet POWs” Elon Law Review 5.2 (2013). On the horrors of the ‘Greater East Asian Co-Prosperity Sphere,’ while somewhat dated, S. Ienaga, The Pacific War (1978)’s chapter on the topic provides a strong sense of the extent of atrocity and also how strongly linked they were to purely ‘military’ operations.)

The Grim Tally of Inhumanity

The scale and extent of human suffering – most of it suffered by civilians, not by soldiers – in World War II is so astoundingly vast that it beggars our feeble capacity for understanding. We reach for the simplest seeming question, ‘how many’ – a question vast in its implications and yet fundamentally bloodless in its imagination – and the figure is so enormous that even the best estimates have error bars larger than the populations of many countries. And yet the horror of it, the crimes of its perpetrators, are also so vast that even quite well educated and informed people are often unaware of entire swaths of the suffering that took place.

It is worth, in that context, attempting an overview, however incomplete Jon Parshall in his recent 1942: Crux of War presents the following map of the ‘geography of death’ representing the death toll, civilian and military, by country, with larger circles indicating larger death tolls:

Map by Jon Parshall, used with permission. You can find full, color versions of the maps and diagrams from *1942: Crux of War* at the book’s website, 1942book.com.

I want to focus here principally on civilian deaths. What follows is not intended in any way as an exhaustive list, but rather an exercise in setting the terms here, of giving a sense of the scale, variety and physical extent of the horrors of this period and of their strong connection to the strategy of the war. And to be frank, it is simply worthwhile to cover even just the basics, even in an incomplete way because these things need to be understood and remembered.

We can start with strategic bombing in both Europe and Asia. The morality of strategic bombing remains a deeply fraught question; we have discussed its limited efficacy already. The US Strategic Bombing Survey after the war estimated 305,000 civilian deaths in Europe from Allied (American and British) strategic bombing; this was actually a higher figure than official German statistics. Modern historians have tended to edge this figure up; Richard Overy estimates around 350,000 while other estimates push north of 400,000.¹ The United States maintained that its bombing campaign was ostensibly targeted at industrial facilities important to the war effort, not aimed at producing civilian casualties, but of course it did produce civilian casualties as its architects knew it would; the Royal Air Force, by contrast, openly practiced ‘morale bombing’ (what they would call ‘terror bombing’ when others did it) aiming to use civilian disruption and death to try to collapse German support for the war (in addition to industrial and military disruption), a strategy which did not work then and has never worked since in any war ever, despite repeated attempts.

Via Wikipedia, a photograph of Tokyo burning after the American firebombing raid, May 26, 1945. The air raids against Tokyo killed more than 100,000 people, nearly all of them civilians. The effect they had in hastening the end of the war is, at best, unclear.

The United States also engaged in strategic bombing against Japan. Official estimates by the post-war Japanese government calculated some 323,495 civilian deaths from bombing, including the nuclear bombs dropped on Hiroshima and Nagasaki; subsequent academic estimates have tended to be fairly close to this figure. American bombing against Japan was notably significantly less concerned with even the token avoidance of civilian death shown in Europe, in no small part for racist reasons. We might thus give a very loose estimate of some roughly 700,000 civilian deaths from Allied strategic bombing across all theaters.

That is already an enormous amount of human suffering and yet at the same time a number small enough to fit within the error bars of what we’re about to discuss.

Of course the Western Allies were not the only parties engaged in strategic bombing and indeed they did not engage in terror bombing first. In Asia, the Japanese began terror-bombing against Chinese cities in 1938, before the war in Europe had even started (and in fact had terror-bombed Shanghai already back in 1932). The first German terror bombing attacks directed against Poland occurred on September 1, 1939, the very first day of the war. And of course the Germans also engaged in terror bombing against Britain from 1940 onward. The Axis powers first unleashed terror bombing as a tactic in the war and then complained that the Allies were better at it. It is rather late in the game to deplore knife fights after you have stabbed someone and only belatedly discovered that everyone else present have longer knives.

Via Wikipedia, a photograph of Warsaw in flames under German aerial bombardment during the opening days of the war in September of 1939.

I have found it almost impossible to find a clear estimate of the number of casualties of Axis terror bombing – in many cases the loss of life in areas that came under Axis occupation was so high that it obscures the impact. Some 40,000 British civilians were killed by German terror bombing during the Blitz and many tens of thousands more must have died from Axis bombing in Poland, China and elsewhere, but I have no secure figure for their numbers. Those deaths will be rolled into statistics below.

Next, we can consider what we might term the ‘deaths of indifference’ – significantly unintentional deaths that nevertheless resulted from the war and the incompetence or apathy of the major powers. There are two major entries in this category: the Iranian famine (1942-3) and the Bengal Famine (1943). In the former case, Britain and the USSR had jointly invaded Iran in August of 1941 to secure supply lines to the USSR and secure Iran’s oil resources, which they feared falling into Axis hands; the invasion and subsequent allied use of the country’s infrastructure, compounded by a bad harvest caused major food shortages. The USSR banned any food from entering the territory (claiming they needed it for the front), while the United States and Britain both failed to backstop food supply in the part of the country Britain had occupied. The death toll from this famine remains highly contested, anywhere from 3 to 7 million, though most estimates cluster towards the bottom of this range, from 3-4 million.

Meanwhile, a combination of labor and resources shifted to the war effort, the Japanese occupation of Burma (and the British response to it) which closed routes for food shipment and the prioritization of food shipments to Great Britain itself – which was, we must note, also facing food rationing – resulted in severe food shortages in South Asia, worst in Bengal but felt in British India as well. Britain failed to allocate sufficient shipping to relieve the crisis, prioritizing the on-going Battle of the Atlantic and so Churchill and his war cabinet bear some clear responsibility for these deaths too. Once again, the death toll remains contentious, but the scholarly consensus tends to be in the 2-3 million range. Thus the combined indifference by the western allies (to be blunt, chiefly Britain, but also the USSR and United States) to starving peoples over whom they had a fairly clear duty of care may have contributed to a total death toll of 5 to 7 million deaths.²

The Axis powers also starved huge numbers of people in territories they occupied. Japanese occupation and continued fighting caused a major famine in Henan province in China in 1942 with an estimated death toll between 2 to 3 million dead. Imperial Japanese conquests elsewhere also severed food routes and Japan was even less willing to divert any of its shipping towards the relief of conquered peoples than the allies, leading to famine in French Indochina and the Dutch East Indies which – combined with brutal occupation and forced labor policies – resulted in 3.5 to 6 million deaths. Germany meanwhile employed hunger as a deliberate weapon of genocide, which will be discussed below.

Which brings us, at last, to the largest total by far – the crimes of malice. The German Wehrmacht and SS entered Poland at the very beginning of the war in Europe already with specialized death squads intending to murder Polish leaders, intellectuals, clerics, teachers and, of course, Jews. It should be stressed that even at this point, the Wehrmacht was fully complicit and both facilitated and participated in war crimes including the mass murder of civilians. So once again, for everyone in the back, the regular German military, the Wehrmacht was fully involved and complicit in Nazi war crimes, alongside the SS and the rest of the Nazi regime.

And those crimes were astoundingly vast. Nazi death squads, the Einsatzgruppen followed the Wehrmacht wherever it went (with its full compliance), murdering as they went. Others – Jews, Roma, Soviet POWs, Soviet and Polish civilians – were forced into brutal labor or extermination camps and there murdered in the millions. Yet more were intentionally starved as the Nazi regime used starvation as an intentional weapon of genocide as part of the ‘Hunger Plan,’ the execution and implications of which were inextricably wound through German military and economic planning as Adam Tooze notes in The Wages of Destruction (2006).

Via Wikipedia, Hungarian Jews being sorted on arrival to the death camp at Auschwitz. Nearly all of these people were murdered.

The numbers defy comprehension. The Nazis murdered roughly 6,000,000 Jews during the Holocaust; non-Jewish victims of Nazi extermination efforts were at least as numerous.³ Nazi Einsatzgruppen murdered roughly 1,350,000 people in the territory of the USSR alone already by the end of 1942. The Nazi Hunger Plan (alongside active extermination efforts) resulted in the deaths of roughly 4.2 million civilians and 3.3 million Soviet POWs, essentially murder by starvation. The plan had intended to murder more – as many as forty-five million people. Estimates for the total number of Soviet civilians killed by the Nazis range upwards from ten million.⁴ While the gas chambers of Nazi extermination camps are rightly infamous, most of the Nazis’ murders were done with bullets. The scale of Nazi murder is so vast it is difficult to describe any part of it without feeling like one risks minimizing the other parts of it, but all of it was of a piece with the murderous Nazi ideology and all of it was accomplished with the active, knowing participation of the Wehrmacht as both a direct function and an explicit goal of the Nazi war effort.

Via Wikipedia, German soldiers standing over the bodies of Polish farmers they have murdered.

Fascist Italy’s scope for atrocity was smaller, but Mussolini’s government added its share of horror. The Italian invasion of Ethiopia (which is on-going at the start of Hearts of Iron IV‘s time frame) killed an estimated 382,00 civilians: roughly 17,000 from terror bombing, another 65,000 murdered directly and a further 300,000 who died of starvation as a result of the war. Italian occupation in then-Yugoslavia, Greece and Albania also brought starvation, mass executions, and civilians herded into concentration camps under abominable conditions. And Italy, of course, cooperated with the Nazi attempt to murder the Jews.

Via Wikipedia, soldiers of a Einsatzgruppen in Ukraine in 1942, murdering a Jewish family.

The Soviet Union also carried out substantial numbers of war crimes and atrocities, although at a much smaller scale than the Axis. Most infamous was the murder of 21,857 Poles as part of the Katyn Massacre, carried out in 1940 while the USSR was still allied with Nazi Germany, but this was hardly the only such incident. It is important to remember when considering the moral implications of these regimes that both the Third Reich and the Soviet Union entered Poland in 1939 with death squads, intending the murder of many thousands if not millions of civilians; neither of these regimes were ‘radicalized’ into mass murder – mass murder was part of their ideology from the beginning. Estimates of the death toll of the post-war Soviet expulsions of Germans range from 500,000 to 2.5 million. Soviet treatment of Axis POWs could also be brutal (albeit still less so than the Nazis): 37% of German POWs in Soviet hands died. By comparison, just 4% of western allied POWs in Axis hands died, but roughly 60% of Soviet POWs held by Germany died: Soviet treatment was at once inexcusably cruel and at the same time far less murderous than the Nazi extermination program.

The Pacific theater of the war was no less terrible. In China, Parshall offers a total estimate between 15.3 and 21.9 million Chinese deaths as a result of the war (soldiers included), including the famines noted above. Nationalist forces destroyed the levees on the Yellow River in 1938 in an effort to halt the advance of the Imperial Japanese Army, an action which drowned tens of thousands of Chinese and triggered follow-on famines and disruptions, with estimates for the total death toll around 500,000. The impact of American bombing, we have already discussed.

Those figures shrink, however, in comparison to the scale of intentional mass-murder by Japanese forces. Again, using Parshall’s survey of the estimates, Japanese soldiers directly killed between 8.2 and 9.0 million civilians across the Greater East Asian Co-Prosperity Sphere (alongside roughly 3.3 million allied soldiers, the overwhelming majority of them Chinese). That is, to be clear, a figure that is not (so far as I can tell) inclusive of the multiple major famines Japanese occupation (and the Japanese refusal to task its merchant shipping to support civilian populations) created. Japanese forces massacred civilians with bullets, bayonets and swords in Malaya, in the Philippines, in Burma (modern Myanmar), in Korea and most of all in China; functionally everywhere the armies and navies of Japan went, they brought atrocity with them. Japan infamously experimented with chemical weapons using civilians and POWs in China. Japanese forces also treated captured POWs brutally – the death rate for American POWs in the Pacific was over 40%, from executions, torture, starvation and forced labor. These actions in turn were not collateral damage but intentional policy, summarized most horribly in the official Japanese “Three Alls Policy” in China, directing Japanese forces to “kill all, burn all, loot all.”

Via Wikipedia, a photograph from the Nanjing Massacre in which a Japanese soldier stands over the massed bodies of Chinese civilians, murdered by the Imperial Japanese Army.

There is sometimes a tendency to try to smudge this all over, to say that ‘oh well every country does bad things in war,’ but there is a real, staggering, horrifying difference between allied collateral damage and the intentional campaigns of Nazi, fascist and Imperial Japanese murder. Parshall estimates that over the course of the war, taking just the minimum estimates, a minimum of 39.6 million allied civilians died, the vast majority intentionally murdered as a result of the deliberate policies of the Axis powers – with gas, with bombs, with deliberate starvation and most frequently of all with bullets and bayonets. Against this is set a maximum of some 6 million Axis civilian casualties, many of whom were murdered by their own governments.

Allied governments engaged in conduct during the war that may fairly be described as callous or criminal. But once again their crimes pale in comparison with the horrors of the Axis, to the point that the entire sum of Allied atrocities can fit with room to spare inside the error bars on the estimates for the mountain of human suffering heaped up by the Axis. And of course beyond this, but for the Axis decision to have the war in the first place, there need not have been any of this.

Via Wikipedia, a photo of Japanese soldiers burning a village in China.

I lay all of this out to make two points. First, of course, is the enormity of the horrors that German Nazis, Italian fascism and Imperial Japanese militarism brought with them. But equally that these crimes were not accidents, nor were they divorced from the conduct of the war. The ‘Three Alls’ policy was a military policy, it was a strategic choice about military action taken by the Imperial Japanese Army. The Hunger Plan was a military policy: the Wehrmacht‘s approach to the Eastern Front was predicated on the logistical simplification of starving the populace (so military logic depended on this policy) and the Wehrmacht was actively engaged in carrying it out (so military actions were dictated by this policy) and the entire point of invading the USSR was to produce the strategic end – the Lebensraum, ‘living space’ (devoid of its former inhabitants) – envisioned by the plan. The atrocities could not be seperated from Axis strategy: they were the ways, means and ends, by, with and for which the Axis fought.

Choosing Indifference

The opposite of love is not hate, it’s indifference. The opposite of art is not ugliness, it’s indifference. The opposite of faith is not heresy, it’s indifference. And the opposite of life is not death, it’s indifference. Because of indifference, one dies before one actually dies.
Elie Wiesel, quoted by U.S. News and World Report, 1986. ⁵

Having done our grim accounting, the question now is asking “how does Hearts of Iron IV represent this historical reality?”

And the answer is that it doesn’t.⁶ Hearts of Iron IV, intentionally, deliberately chooses indifference.

The closest Hearts of Iron IV comes to addressing any of this is the occupation system, in which countries that are occupying foreign territory are free to pick from a range of occupation policies from things like ‘local police forces’ and ‘military governor’ to ‘forced labor’ and ‘brutal oppression.’ These policies impact resistance strength (which determines the possibility of uprising, though uprisings are vanishingly rare in this system) and compliance (which determines how much of the resources of the occupied territory’s can be accessed. They do not, however, impact the civilian population, despite the fact that the total size of the population (split between civilian and recruitable) is a statistic the game tracks for every state (a group of provinces) in the world.

An example of the occupation law options in Hearts of Iron IV. In this case, these are the options for Britain which as a democracy has access to ‘Local Autonomy’ but not ‘Brutal Oppression’ (the latter being fascist specific). Note that the values on the far right are not population modifiers, rather that ‘people-in-a-row’ symbol is indicating the garrison size adjustment, essentially a manpower penalty for holding occupied territory (varying in size based on the template chosen for occupation forces, which are generated automatically).

A policy of “brutal oppression” (the unique policy available to fascist ideologies) does not result in civilian casualties the game cares to measure or model. It does not produce global shock, diplomatic incidents or strategically relevant refugee flows. It does not impact war support. All it does is reduce the resistance target (the value to which resistance will slowly move) by 75%, at the cost of requiring 25% more garrison troops (generally not a significant amount). This is, by and large, a toothless mechanic, one the player-base has already reduced to a simple calculation of garrison costs vs. resources gained that can be gamed out in a spreadsheet with no broader implications. Indeed, my sense is many players broadly ignore this system, leaving their occupation policy on whatever its default setting was.

And that is it, that is all the game models.

Their absence is not an oversight but a deliberate policy: Paradox has made it very clear that they have chosen to exclude war crimes, forced labor, rape and atrocities generally from the game. They’re also explicitly banned as discussion topics in the Hearts of Iron forums and have been since at least HoI2 if not earlier. Though I don’t know that Paradox has ever put out a single clear policy statement about this stance publicly, the basic reasons for it have been articulated here and there over the years and are well known by the player-base.

On the one hand, there is a concern that featuring war crimes or atrocities will attract players who specifically want to ‘press do-the-Holocaust-button’ and from time to time Paradox developers have made statements to the effect that they have no interest in making a war-crimes-simulator. On the other hand, there is also a commercial factor here: Paradox wants – and arguably, financially needs – to be able to sell this game in a wide range of countries and many countries (most notably Germany) have bans on the depiction of certain Nazi symbols or imagery, while at the same time discussion in game of this or that atrocity could trigger controversies that would negatively impact sales.⁷

And I want to say at the outset that I understand the difficult situation that the developers at Paradox Development Studios and on the HoI4 team are in. I had the chance a few years back to speak with several of them and they’re not unaware of this problem. There is a kind of impossible dilemma here: the entire Paradox approach to making their strategy games is that every country is playable, which means that Nazi Germany, Fascist Italy and Imperial Japan have to be playable. And, while we’re here, while ‘just don’t make Germany playable’ may sound like an easy solution, we do not expect that of any other WWII game – you can play as the Germans in the WWII entries of Call of Duty or Battlefield or in Steel Division or Company of Heroes – and simply making the Axis unplayable would probably damage the marketability of the game to the point that it would no longer be sustainable to make, as compared to competitor titles. There is a space for ‘you can only play the Allies’ WWII titles, but not in this genre and we all know it.

But once it is decided that the Axis have to be playable – that they cannot simply be the bad guy villains in a single-player experience – the design has to envision players playing them. And therein comes the problem: do you want to make a game that is a Holocaust simulator? That problem is of course compounded by things like online streaming: a game which allows the player to initiate the Holocaust as Nazi Germany absolutely will have online streams where modern neo-Nazis film themselves gleefully initiating the Holocaust and other Nazi atrocities. So the solution arrived at was to simply not simulate those atrocities, to exclude them from the game’s model of WWII.

Now there is an immediate problem with Paradox’s, ‘we do not make war crime simulators’ stance, which is that it isn’t true. Forced labor – slavery – appears mechanically in Victoria III, Europa Universalis V and Imperator, along with older titles. Crusader Kings III includes mechanics for sexual slavery – characters of certain raiding cultures can take prisoners and force them into being concubines. Paradox games even feature genocide, most notably with the science-fiction themed Stellaris, where genocidal war is a major gameplay mechanic, with multiple flavors of genocidal intent. And while strategic bombing in Hearts of Iron IV does not cause civilian casualties, in Stellaris, orbital bombardment absolutely does kill civilian pops.

In short then, Paradox’ ‘no war crimes’ policy is really ‘war crimes can be depicted except during the 1930s and 1940s.’ It’s clear – particularly from the design of Stellaris (where population could easily have been heavily abstracted as it is in some other 4X type space-imperialism games) – that the ‘no atrocities’ rule is not a general design principle in Paradox games but a specific design principle about Hearts of Iron. The developers of Hearts of Iron IV have chosen indifference, indeed a strategy of indifference towards the war’s atrocities.

Now I feel I should note that Hearts of Iron IV is hardly the only game to do this. Indeed, it seems like most World War II games do, to some degree. Rarely if ever do games like Company of Heroes or Steel Division II (which takes place on the Eastern Front!) show the atrocities that armies – mostly the Axis armies – were engaged in. I can’t claim to have played every Call of Duty or Battlefield entry set in WWII either, but I certainly do not recall either franchise drawing attention to the Holocaust or other war crimes. It is an absurd and yet true statement that of most of the popular video games which feature the Second World War, precious few are as honest about the nature of the Nazi regime than the anime-alternate-history-World-War-II-with-magical-girls game Valkyria Chronicles.

Hearts of Iron IV is not worse than the norm, but equally it does no better.

Now the defense those games will raise is that they are ‘purely military’ in their focus, set entirely ‘on the battlefield’ and so war crimes and atrocities are outside of their scope. Of course as we’re demonstrating here, that defense is frankly – pardon my language – bullshit. Most Axis soldiers were either complicit or participants in war crimes over the course of World War II; not some, not many, most.⁸ While a lot of these games focus on the western front, which was – as the war went – the least full of atrocity, atrocity was hardly absent, particularly as allied forces pushed forward and began encountering evidence of the Holocaust.

But Hearts of Iron IV lacks even that flawed defense, because it does not just confine itself to the battlefield: it aims to simulate military, political, diplomatic and economic aspects of the war and these elements are all shot through with crimes. Manpower and strategic resources were diverted to extermination programs. Civilians starved because of disruptions to shipping, because of intentional diversion of food, died because of forced labor used in military production or the extraction of strategic resources. Countries took diplomatic stances because they had become aware of atrocities (again, such as the American reaction to news of atrocities by Japan in China), atrocities stiffened the will to resist in many places, gave rise to partisan and resistance efforts.

Cleaning the Wehrmacht

Meanwhile this particular design choice runs directly into a more serious problem: it effectively constructs a ‘clean Wehrmacht‘ simulator.

The clean Wehrmacht myth is the historical myth that seeks to separate the regular German military (the Wehrmacht) and in particular the regular German army (the Heer) from the crimes of the Third Reich by presenting those crimes as primarily perpetrated by the SS and the Nazi party. The myth has its origins in the end of the war and its immediate aftermath: former German generals rushed to get out a version of events which pinned the blame for war crimes on someone else (not the least of which because some of the criminals were being hanged) and to launder their own reputations. A lot of enthusiast knowledge of WWII remains rooted in some of these early memoirs and this idea that the officer corps of the Wehrmacht specifically was somehow waging a ‘clean’ war in parallel to Nazi crimes even shows up in the works of Cold War era American scholars like Samuel P. Huntington.⁹

However the clean Wehrmacht myth is just that – a myth – it has been decisively and comprehensively dismantled by modern historians, in both German-language and English-language historiography.¹⁰ But as we’ve discussed already there was never a clean seperation between the Wehrmacht and the atrocities. Indeed, as A.J. Kay and D. Stahel conclude, “members of the Wehrmacht [as distinct from the SS] may have constituted a majority of those responsible for mass crimes carried out on the part of the German Reich” and that “it would be reasonable to conclude that a substantial majority [emphasis original] of the ten million Wehrmacht soldiers deployed at one time or another in the German-Soviet War were involved or complicit in criminal conduct.“¹¹ Indeed, given how much of the Wehrmacht fought on the Eastern Front (and the presence of further crimes and atrocities in other theaters) it seems likely that most German soldiers participated in or observed and were complicit in war crimes and atrocities. The same, of course, is true of Italian and Japanese soldiers.

Moreover, at the strategic level, it is impossible to separate the conduct of the war, the allocation of resources, the aims of the conflict – in short the strategic and operational levels on which Hearts of Iron IV is so totally focused – from the conduct of crimes and atrocities. We’re going to come back to war aims and strategic ends in the next part, but it is necessary to note that for Hitler and thus for the Wehrmacht the purpose of invading the USSR was to commit war crimes: the goal was to free up land for large-scale German settlement (this is what the Hunger Plan envisioned) and that was only going to happen if the existing population was enslaved and murdered. Forced labor and mass murder were the objectives for which the Wehrmacht was fighting.

The Wehrmacht fought battles so that it might have the opportunity to commit massacres. That was the purpose of the war.

Likewise the forcible extraction of raw resources using local forced labor was the strategic purpose of Japanese conquest in this period, while atrocious policies like the ‘Three Alls’ were understood as military means to achieve that purpose. The Imperial Japanese military had even less separation between the units doing the war crimes and the units doing the ‘real fighting’ than the Germans, as relatively few Japanese atrocities were carried out by ‘specialist’ units and instead nearly all were carried out by regular units of the Imperial Japanese Army and the Imperial Japanese Navy.

Moreover, the decision to commit atrocities had real strategic and operational impacts on the war. In general, the Axis powers were so committed to an ideology of atrocity that the atrocities they committed made their war effort weaker and yet they kept doing them anyway because the atrocities were, at a fundamental level, the point of the exercise. The Holocaust, above and beyond its moral atrocity, diverted a significant amount of soldiers and resources to the task of reducing (by murdering them) the manpower and working population available to Germany’s war effort. Nazi oppression also drove away scientists and intellectuals (when it did not murder them), not a few of whom ended up key elements of allied technological superiority. Mass murder in occupied Soviet territory had a similar effect but also encouraged resistance even in areas where significant discontent with the Soviet regime – which had inflicted its own brutalities – could have provided the basis for collaboration.

Similarly, the Japanese propaganda effort in presenting Japan’s new empire – its ‘Greater East Asia Co-Prosperity Sphere’ as a crusade against western imperialism might well have found receptive ears (there was certainly no lack of discontent with imperialist oppression in Vietnam, Malaya, the Philippines and elsewhere) if the Imperial Japanese forces had been able to restrain their brutality. Instead, Imperial Japanese troops murdered, starved and brutalized the civilian populace in the areas they conquered – often fostering local resistance even as Japan itself was short on war material, manpower and labor.

On top of this, German, Japanese and Italian atrocities also served to reinforce allied moral and the will to fight on as they demonstrated that these were odious, evil regimes that had to be stopped at any cost. Indeed, without the shocking brutality of Japanese armies in China, particularly the Nanjing Massacre, it is unclear if the political will would have existed in the United States to take the steps to curb Japanese aggression that led to the Japanese attack on Pearl Harbor.¹²

In short then not only was Axis brutality essentially mandated by their military strategy (as well as their political ideology), it was also strategic in the sense that it was a strategic liability that made the Axis powers even less likely to win the war they had started.

Hearts of Iron IV creates the impression that there is a thing called ‘World War II’ – which includes the Wehrmacht and the IJA and the IJN – which is distinct from these other things – the war crimes – and can be discussed in isolation. And it simply is not so.

In attempting to avoid stepping into the morass of having to make hard choices about how to represent the historical atrocities that occurred in this period, Hearts of Iron IV falls into the trap of essentially producing Nazi, fascist and militarist apologia, embracing the clean Wehrmacht myth and turning the game into effectively a ‘clean Wehrmacht simulator.’ In the process, the game is actively reinforcing a long-discredited, irresponsible and deeply damaging vision of history, one prized by Nazis past and present.

Abandoning Indifference

Quite clearly, I think Hearts of Iron IV‘s approach to handling the human toll of the Second World War is insufficient. But equally I will admit that in many ways the developers at Paradox have an impossible choice: they cannot make the Axis unplayable and remain competitive in the strategy game market, they cannot have a ‘do the Holocaust’ national focus for obvious reasons and now I have said that they cannot, in good conscience, simply ignore the whole issue altogether. And yet if I am going to criticize this approach so strongly, I feel I must offer an alternative.

I think the better approach here – and I do not claim this is a perfect approach because I do not think there is one – is to represent these crimes not as player choices, made moment to moment, but rather as direct consequences of national ideology. It was not random chance that made the Third Reich and Imperial Japan engage in such atrocious conduct – their atrocious conduct was a direct consequence of their ideology, which classified entire groups of people as sub-human, worthy only of cruelty, exploitation or murder. I would thus argue that governments of this sort, with these ideologies were always going to engage in atrocities to the degree that they were able.

Incorporating that historical understanding into the game mechanics means removing a degree of player agency, but I think in this case that is appropriate. In this approach, so long as Germany remains fascist, the Nazi campaign of mass murder would continue to progress in the territory it controlled. So long as Japan remains militarist, its armies would brutalize the territory they occupy. And so on.

Based on that progression, the system would trigger events with pop-up notifications for all players providing historical context – written, I would suggest, in a fairly flat and clinical, ‘on this day, etc. etc.’ style, focusing on the scale of human suffering and the clear culpability of the regimes in question. It might even make sense to create a specific design of text-block and image to clearly distinguish between these specific historical memorials as compared to the branching or alternate history events the player engages with. These events would not involve player action – there are no choices they prompt – they would be intended instead as digital Stolpersteine (‘stumbling stones,’ a decentralized system of memorials in Europe for victims of the Nazis), a chance to make the player stop and take stock of what the war meant, what the regimes involved were and what they did.

Via Wikipedia, a Stolpersteine for the Weissfisch family, victims of the Nazis, in Bratislava, Slovakia.

The yet braver choice would be to also give these events gameplay impacts. Now one can easily imagine the almost immediate fear any developer would have, which is creating a game system where players are incentivized by the mechanics to simulate these atrocities. But I think we can avoid that problem neatly in two ways. First, as noted, we’ve already largely limited player choice: the atrocities happen so long as the historical regimes remain in place so there is no button to press.¹³ Second, I think a fair historical assessment of these atrocities would still make their effects entirely negative. Mass violence against civilians did not push any country closer to victory in the Second World War¹⁴ but they did hurt the war effort of the perpetrating countries, encouraging resistance, killing potential workers and soldiers, alienating foreign countries and consuming manpower and resources to no military end.

In that context, to me, it seems quite reasonable that the initial phases of Nazi oppression against the Jews would reduce Germany’s available manpower and probably ought to provide a non-trivial research penalty. As the Nazi campaign of extermination progressed, it might also consume industrial production (represented in factories) and further reduce available manpower. Atrocities in the territory of the Soviet Union and China (and elsewhere) could be fairly tied to create effects increasing local resistance, representing the way in which Nazi and Japanese atrocities fueled partisan warfare which in turn had real operational effects not currently effectively simulated in game. Admittedly, this would be a courageous development choice, but I think it is an honest one.

The irony is, Hearts of Iron IV already has a system that resembles this, but only for the USSR and only for Stalin’s Great Purge. In practice, so long as the Soviet Union remains Communist, the Purge is effectively guaranteed to happen (either a Stalinist purge or a Trotskyite one). And while the game remains mostly mute on the civilian impacts of the purge and Stalin’s larger campaign of oppression and the massive system of slave labor camps (the Gulag) that the Soviet Union operated, nevertheless this atrocity is not only incorporated into the game, it has mechanical effects – removing a number of possible military and civilian leaders and weakening the Red Army.

That said, even the pure ‘stumbling stone’ approach – simply documenting via in-game events with no further mechanical impact – would already be, to my mind, a more honest and accurate solution than what Hearts of Iron IV has now or indeed than most modern WWII games attempt. As noted above, it is the normal approach for games set in World War II to simply take the same cowardly dodge Hearts of Iron IV has, to pretend these things didn’t happen or that if they did happen they happened somewhere over there – off stage left, away from the ‘real’ war and the ‘real’ fighting and the ‘real’ soldiers – but this simply is not true. Moreover that approach has the unintended but very real consequence of turning every game in which it is used into a piece of ‘clean Wehrmacht’ propaganda. And that is not a neutral stance either, but instead the preferred rhetorical frame of modern fascists, Nazis and militarists.

It is, I think sincerely, a moral failing to adopt this dodge of indifference out of cowardice or commercial expediency and I think Paradox can do better and should do better. It is time to stop pretending, here and elsewhere, that World War II and the many Axis (and to a lesser degree, Soviet and other allied) atrocities can be cleanly separated, that we may treat the war in the absence of the horror of the regimes we fought in it. We cannot and it is cowardly to try.

We are charged by the voices of the past to remember and cultural products – and a video game is every bit as much of a cultural product as high literature or prestige film and TV – are part of how we remember. And we are charged to remember.

It is not enough to be merely indifferent.

The enormous advance in AI capabilities over the last several years has in large part been driven by the availability of vast amounts of training data. Gwern’s influential 2020 essay “The Scaling Hypothesis,” written after the release of GPT-3, argues that recent advances in AI have been the result of scale: bigger models, bigger datasets, and more compute spent training them. Gwern writes:

The scaling hypothesis regards the blessings of scale as the secret of AGI: intelligence is ‘just’ simple neural units & learning algorithms applied to diverse experiences at a (currently) unreachable scale. As increasing computational resources permit running such algorithms at the necessary scale, the neural networks will get ever more intelligent.

Likewise, pseudonymous AI researcher Nostalgebraist wrote in 2024 that “the entire current landscape dates back to the realization that ML models improve steadily and predictably as you train them on more data. And that everything since then consists of training bigger models on larger datasets, enabled by years of rapid improvement in hardware.” Epoch AI found that since the early 2010s, the size of training datasets for AI language models has doubled about every six months. The biggest AI models today are trained on tens of trillions of tokens.

As more and more effort gets devoted to applying modern AI to robotics, an important question is “where will the training data come from?” Language models could be trained on huge existing corpuses of text. GPT-3, for instance, was trained on about 300 billion tokens’ worth of data scraped from the internet. In 2024 Epoch AI estimated the total existing stock of quality, accessible text data is around 300 trillion tokens. But there’s no equivalent stock of good, public robot training data. Figuring out where the data for robotics will come from and how much of it we’ll need has become a key problem for the field.

What is robot training data?

Training an AI model, whether it’s an LLM or a model used for controlling robots, fundamentally requires lots and lots of examples of something being done correctly (or enough knowledge of what a correct action looks like that you can score an attempt). You feed a model some set of inputs, you compare the model’s output to the actual, correct output, and you use the difference between the two to modify the parameters of the model to reduce this difference via some flavor of gradient descent. (We went into this process in some depth earlier this month.)

With text, a training example can be just an existing chunk of text; you take the chunk, feed the model part of it, and have the model predict what comes next in the sequence. One chunk of text can thus generate many training examples by feeding the model different portions of the text.

To train a robot, an example is typically some sort of recording of a task being accomplished or attempted. There are a variety of different methods used to create these recordings and turn them into training data.

Historically, one of the most common methods for creating these recordings is called “teleoperation.” With teleoperation, a person manipulates some controls that are directly connected to a robot, and uses them to move a robot through some particular task (folding a shirt, picking up and moving objects, and so on). The image below, for instance, shows a person teleoperating a pair of robot arms, using them to pour water from a pitcher into a glass.

As the person performs the task, data of the robot’s state is recorded, along with images from cameras that are recording the task (often mounted to the robot itself), which are combined to create a training example. An example might consist of feeding a model some data on the robot’s state and what the cameras can see at some particular point in the sequence. The model will try to predict what the next robot actions are (in the form of something like joint angles), and then compare that to what the actual actions were. This sort of training strategy, using examples generated by an expert (a human) and then training a model to mimic them, is known as behavioral cloning, and it’s part of the broader category of “imitation learning.”

The major benefit of this method is that the data it generates very closely matches the robot’s actual movement capabilities, because it’s produced from the actual robot you want to train, and it remains widely used today. You sometimes see videos or pictures from huge data collection factories, filled with people doing nothing but teleoperating robots to complete various tasks, and you sometimes see teleoperation accidents when the person forgets that they’re controlling the robot and suddenly moves in a way that the robot can’t handle.

But teleoperation has drawbacks. The biggest one is probably expense: to get the data, you need to have an actual robot, along with a person to sit there and manipulate it, which makes it hard to scale this method to produce very large amounts of data. Another drawback is that it’s somewhat clunky for a person to control a robot this way; they often can’t get any sort of tactile feedback from the object being manipulated to guide their motions, so this won’t necessarily produce really good data on dexterous manipulation. The clunkiness can also make it hard to collect good teleoperation data, and as I understand it, only a fraction of total teleoperation data collected ends up being useful for training.

It’s also possible to generate training data using an existing robot by having it move around on its own, recording its actions, and using some sort of system to categorize these behavioral sequences as successful or not. This sort of strategy, where the robot moves autonomously and you score its actions in some way, is known as “reinforcement learning.” Google was doing something like this around 2016 with an “arm farm”: a series of robot arms moving autonomously, with success scored by whether an arm picked up or grasped an object or not.

This sort of arm farm doesn’t seem to be widely used today, but using real-world deployed robots will probably eventually be the largest source of robot training data. You often hear robotics folks talk about the need for a “data flywheel,” where you have robots deployed in the real world, you use the data from them to train even better models, resulting in more robot deployments, resulting in better models, and so on.

Robotic AI company Physical Intelligence, for instance, is experimenting with this sort of “deploy a robot to do real work, and use it to gather data to improve the model” proto-flywheel at a chocolate factory near its offices. But while this might work eventually, right now there’s a very small number of AI-controlled robots doing anything resembling real work. Real-world data also tends to be extremely repetitive — robots doing the same task over and over again in very similar conditions — which makes it much less useful for training. So for now most data needs to come from other sources.

Another very common strategy for generating robotic training data is simulation: instead of running a physical robot in the real world, run a digital one in a simulated world instead. This is also commonly done with reinforcement learning, in which you have the robot try to navigate some simulated space, after which you score it on how well it does. Another flavor of this is the teacher/student paradigm where you have a “teacher” model that has full access to all the data within the simulated environment, and a “student” model that can only see what the real robot would see (images from its cameras and its own internal state), and you try to get the student to mimic the teacher.

The benefits of simulation are that it’s much cheaper and faster to simulate a robot than it is to deploy one in the real world, and that you can get very large amounts of data by simulating the behavior of thousands of robots. Figure AI taught its humanoid robot to walk via reinforcement learning on thousands of simulated robots. OpenAI taught a robotic hand to solve a Rubik’s Cube via the equivalent of 13,000 years of simulated experience. By 2019, Waymo had driven 10 million miles in the real world, but 10 billion miles in simulation.

But simulation also has drawbacks. The biggest one is that the quality of the data it produces is a function of the accuracy of the simulation. For some things, such as locomotion (having a robot move through some physical space), simulation is fairly accurate, because the behavior is mostly well-characterized enough that we can represent it digitally. So you can train a quadruped robot to navigate some terrain, or a bipedal robot to dance, or a drone to fly along some path in simulation, and the simulated environment will be a reasonably good approximation of the real world. But for things such as object manipulation (particularly for soft or deformable objects), it’s much harder to build a simulation that accurately captures real-world behavior, and using it as a source of training data is much more difficult. This is generally referred to as the “sim-to-real gap.”

Because of simulation inaccuracies, one popular simulation technique is called “domain randomization.” Since you don’t know what various parameters like friction will be in the real world, you decide to vary the parameters randomly when running the simulation, in the hopes of creating a training envelope that encompasses the parameters that will be encountered in the real world. When OpenAI trained its robotic hand to solve a Rubik’s Cube, for instance, it continuously expanded the range of parameters being tested, forcing the model to learn to solve the cube in more and more difficult environments, in the hopes of giving it capabilities that would generalize to the real world. The robotics team there used this strategy specifically because accurate physical modeling was too difficult:

The biggest challenge we faced was to create environments in simulation diverse enough to capture the physics of the real world. Factors like friction, elasticity and dynamics are incredibly difficult to measure and model for objects as complex as Rubik’s Cubes or robotic hands and we found that domain randomization alone is not enough.
To overcome this, we developed a new method called Automatic Domain Randomization (ADR), which endlessly generates progressively more difficult environments in simulation. This frees us from having an accurate model of the real world, and enables the transfer of neural networks learned in simulation to be applied to the real world.

But despite this technique, and really quite a lot of simulation (13,000 simulated years), performance was only OK — for maximally difficult starting arrangements, success was only 20% — which is maybe why this technique doesn’t seem to be widely used for manipulation training currently. Environmental parameter randomization does seem to be fairly standard for locomotion training though, used for things like randomly varying the coefficient of friction of the surface being traversed.

Another way to use simulation is to create synthetic training data: taking real training data as an input and using it to create lots of similar training examples. Nvidia, for instance, has a page showing how you can use its Isaac GR00T robot simulation platform to generate synthetic robot training data using teleoperation data as an input. Synthetic data generation also seems to be widely used for training autonomous vehicles.

We can also create robot training data by recording humans doing tasks and creating some sort of mapping from the recorded task data to a robot. This might be as simple as fitting a person with a head-mounted camera (creating what’s known as “egocentric video”), and using it to record the person doing various tasks with their hands. The video is then processed so the person’s motions are interpreted as robot motions, and used to create training examples. (This is one reason for the popularity of the humanoid form factor in robotics: humans doing tasks will be a reasonably good source of training data.) Figure AI, for instance, recently announced Index, its program for gathering training data by sending people head-mounted cameras to record themselves doing various tasks (alternatively, Figure will also send a crew equipped with these cameras to your home or business to do chores for you). Companies like Shift are also doing this, sending people with hat-mounted cameras to folks’ houses to record themselves doing chores.

Figure collecting egocentric video data, via Figure.

You can also add more sensors to collect more elaborate human task data. The now-defunct startup Eidon AI stuck sensors on folks’ arms to collect data on joint positions for training robot arms. Neocambrian AI recently announced “Mitts,” which it describes as “state of the art tactile gloves that capture high-precision touch data for robots.”

Another flavor of this strategy is to use what’s called a Universal Manipulation Interface, or UMI. With UMI, a person holds a pair of specially designed manipulators, that typically have a simple pincer grip at the end (though some have multiple fingers). The person does various tasks with the manipulators, which have cameras mounted to them that record exactly what the manipulators are doing. This video is then used to create training examples for robots that have similar manipulators. Robot model startups Generalist AI and Sunday Robotics, for instance, seem to be primarily using this strategy for gathering training data.

UMI data collection, via Chi et al 2024.

The benefit of using human-collected task data is that it’s much cheaper to collect than teleoperation data: all you need is a camera, or some camera-equipped manipulators, rather than an entire robot. And it’s probably much easier to get a wide variety of different task recordings by just having a person walk around and do different things, rather than constantly repositioning a robot to do them. And because the data is gathered in the real world, by people manipulating real objects, it’s more accurate than simulation. The drawback is that because this data comes from a person rather than a robot, it won’t map onto a robot’s capabilities perfectly well. Even UMI, which minimizes the human-robot differences by having a person do tasks using a sort of robotic manipulator, won’t necessarily map perfectly, though the discrepancy is small and many of the most impressive robotics demos we’ve seen (such as Generalist AI’s) have been trained using UMI data.

It’s possible to use multiple training strategies in a single robot model. Agility Robotics’ Digit, for instance, walks using a small, 1-million-parameter neural network trained in simulation using reinforcement learning, while object manipulation seems to have been trained using behavioral cloning. Dyna Robotics similarly seems to have used a combination of reinforcement learning in simulation and human video for training data.

It’s also often useful to train a robot model using data other than task completion data. Vision-Language-Action models, as we noted earlier, are built from Vision-Language models trained on text and images. And because of the ubiquity of general video data on things like YouTube or TikTok, there’s a great deal of interest in finding ways to use generic web videos as a source of training data. Nvidia’s Cosmos, for instance, is a foundation model trained on video data that can be used as a backbone for a robot policy model.

Robot dataset sizes

How much of this action-relevant training data have we gathered so far, and how much do we need?

As with AI models more generally, robotic training datasets seem to be getting bigger. An early dataset that was considered large at the time was the Open X-Embodiment (OXE) dataset, created by Google DeepMind in 2023 by stitching together dozens of existing datasets. OXE consisted of over 1 million robot action trajectories spanning more than 160,000 different tasks, totaling around 4,000 hours’ worth of task data. Physical Intelligence’s Pi 0 model, released in October of 2024, was trained on 10,000 hours of robot task data. Generalist AI’s GEN 1 model, announced earlier this year, was trained on around 500,000 hours of UMI data. Figure AI’s Index announcement stated that it was gathering 30 minutes of video data every second, or 43,000 hours a day. Dyna Robotics and Nature Will both claim that their latest models are trained on over 1 million hours of human egocentric video data. I saw one estimate that the robot data collection industry is collectively producing around 500,000 hours of training data a month.

It’s not amazingly easy to compare these datasets to the text datasets used to train LLMs. For one, different robotic training sets have different levels of quality — teleoperation or UMI data is probably higher quality than pure human egocentric video data for training purposes, and a highly variable dataset with lots of different sorts of tasks is much more useful than an extremely repetitive one. And video data for robotics training is informationally different from text data: for one, video data tends to be highly correlated — on short timescales, one frame is pretty similar to another frame. Thus even though video recordings contain a lot of data, they’re probably less informationally dense for training purposes than text data. As robotics researcher Chris Paxton notes, “machine learning works best the closer you are to an independent and identically distributed (iid) dataset; neither web data nor robot data is iid, but robot data is far less iid than web data.”

But we can still try. One very rough way to compare dataset sizes, used by Physical Intelligence researcher Kevin Black in this presentation and robotics professor Ken Goldberg in this presentation, is to estimate how long a dataset of text tokens would take for a person to read, and compare that to the hours of robot training data we have. Under this metric, GPT-2 was trained on the equivalent of around 475,000 hours of data, GPT-3 was trained on 26 million hours, and GPT-4 was trained on 685 million hours. Using this (somewhat dubious) metric, even several-year-old AI models were trained on something like 1,000 times the data that current robot models are trained on. Goldberg calls this the “100,000-year data gap.”

On his Substack “It Can Think!” Paxton makes an estimate from the other direction, converting hours of robot data into tokens, and comes up with a fairly similar estimate. He guesses that for video running at 10 frames per second, each frame will contain the equivalent of one useful training token. Every hour of video thus contains about 10 × 60 × 60 = 36,000 training tokens. He then divides this by 10, on the assumption that you can’t expect every second of video data to actually be useful, getting you about 3,600 training tokens per hour of video. This would make Dyna Robotics’s 1 million hours of training equal to around 3.6 billion tokens, on the order of 5,000 times smaller than the datasets used to train recent AI models.

If you assume that we need about 1,000 times as much real-world robot training data as we actually have — on the order of 1 billion hours of data — how much would that cost to generate? Eidon AI reports robot data prices of $60 to $100 per hour. Pantheon similarly reports UMI data costs from $60 to $150 per hour, but that its engineers developed a pipeline capable of generating data for $10 per hour. Shift claims you can earn up to $20 per hour recording egocentric video data and selling it to them. So a naive estimate suggests that getting enough data to get robot models to a similar level of capability as recent AI models might cost on the order of $10 billion to $100 billion.

How much data we actually need, however, seems to be something of an open question. If we don’t care about generalization, and only want to train robot models to perform various specific tasks, the data requirements might be substantially less. Alternatively, it might turn out that frontier general AI models like Fable or Astra are extremely capable robot controllers after being fed a comparatively small amount of robotic training data; some folks are already predicting that special-purpose robot AI models might get replaced with more generic AI models. Likewise, we might find robotic AI model architectures that are much more sample efficient — learning much more from each training example — than current models.

On the other hand, it could be the case that moving around and manipulating objects in the real world is a fundamentally more difficult task than manipulating text on a computer, and that models require a lot more training to become competent at this real-world behavior. Small errors in generated text, for instance — misspellings, or slightly wrong word choice — don’t necessarily compromise the text as a whole, but when you’re manipulating objects a single tiny positioning error can be the difference between success and complete failure. Likewise, Ken Goldberg argues that robotic movement is a much higher-dimensional problem than generating text (though I’m not sure I buy the version of the argument he articulates in that presentation, but I have seen this sentiment echoed by other robotics researchers).

Conclusion

LLMs showed the benefits of scale when training AI models — bigger models, more data, trained using much more computation. We’re now in the process of seeing how well this scaling works for robotics, and robotics researchers are working hard to gather enormous amounts of robot task training data in the hopes of seeing the same sort of trends with robotic capabilities that we have with AI more generally.

#	State	Project Name	Total Cost	Length	Cost / Mile
1	Oregon / Washington	Interstate Bridge Replacement (WA/OR I-5)	$15.00 Billion	~5.0 mi	$3,000.0M / mi
2	New York	Brooklyn Queens Expressway Central Project (Brooklyn)	$4.00 Billion	~1.5 mi	$2,667.0M / mi
3	Oregon	I-5 Rose Quarter Improvement Project	$3.50 Billion	~1.5 mi	$2,333.0M / mi
4	Oregon	I-205 Abernethy Bridge	$885.00 Million	~0.52 mi	$1,701.9M / mi
5	Massachusetts	Allston I-90 Multimodal Project (Boston)	$2.00 Billion	~1.5 mi	$1,333.3M / mi
6	New Jersey	New Jersey Turnpike Hudson County Extension	$10.70 Billion	~8.1 mi	$1,321.0M / mi
7	Connecticut	I-84 Hartford Mobility / Replacement Plan	$5.00 Billion	~4.0 mi	$1,250.0M / mi
8	Louisiana	I-10 Calcasieu River Bridge Replacement (Lake Charles)	$3.10 Billion	~5.5 mi	$563.6M / mi
9	Texas	North Houston Highway Improvement Project (I-45)	$13.00 Billion	~24.0 mi	$542.0M / mi
10	California	Highway 37 (Santa Rosa/Vallejo)	$11.00 Billion	~21.0 mi	$523.0M / mi

What City Observatory Did This Week

Must Read

Comments

Related Stories

Comments

Related Stories

The Grim Tally of Inhumanity

Choosing Indifference

Cleaning the Wehrmacht

Abandoning Indifference

Comments

This is all based on a false premise

Comments

Related Stories

Comments

Related Stories

What is robot training data?

Robot dataset sizes

Conclusion

# “How to steal my work”

# “everything runs in your browser”

# A NICE STAND - FAX MACHINES

# THE END

Comments

Related Stories

Comments

Which is bigger, AI or the railroads? (a pointless but fun debate)

Hyperscalers are starting to borrow…a lot

Friday, October 16

Saturday, October 17

Comments

Related Stories

Comments

Related Stories

$885 million: True Cost of the I-205 Abernethy Bridge

ODOT concealed cost increases by deferring soil stabilization

Abernethy is the Fourth Most Expensive Highway Project, Per Mile, in the Nation

Comments

Related Stories