A Reader's Map

Superhuman AI may develop misaligned goals — an AI smarter than all humans combined could pursue objectives subtly different from what we intended, and we might not catch it.

Intelligence enables strategic deception — a misaligned AI could pretend to be aligned during testing, then act on its true goals once deployed at scale (“playing the training game”).

AI doesn't need physical form to be dangerous — it can manipulate humans, hack systems, copy itself across servers, and acquire resources purely through digital action.

“Alignment is trivially easy” is a dangerous assumption — the Yann LeCun position that we can simply build in goals and constraints underestimates emergent behavior in superintelligent systems.

Value drift from training contradictions — models told “don't do bad things” but shown that humans do bad things face genuine moral dilemmas that could resolve unpredictably.

Cascading autonomy risk — as AI systems are given more autonomous tasks (multi-hour, multi-day), the window for undetected misaligned action grows.

The “galaxy-brained” problem — an AI might convince itself through seemingly valid reasoning chains that harmful actions are actually justified (“for the greater good”).

AI removes the ability barrier for bioweapons — historically, motivation to create bioweapons exceeds the ability to do so. AI closes that gap, potentially enabling lone actors.

Biology is the most asymmetric threat — bioweapons are uniquely dangerous because attack is far easier than defense: one pathogen vs. protecting all of society.

Step-by-step interactive guidance is the real danger — not a single genome sequence, but months of coaching through obscure synthesis steps that currently require PhD-level expertise.

Mass murder as “fad” — violent individuals copy each other's methods (serial killers in the '70s–'80s, mass shooters in the '90s–2000s). AI-enabled bioattacks could become the next pattern.

Cyberattacks amplified by AI — AI can discover zero-day vulnerabilities, automate exploitation, and scale attacks beyond current defender capacity.

State actors compound the threat — nation-states with AI capabilities could develop sophisticated bioweapons programs or cyber capabilities, not just lone wolves.

AI as the ultimate tool of authoritarian control — mass surveillance, AI-generated propaganda, automated censorship, and predictive policing at a scale no dictatorship has achieved before.

The Chinese Communist Party scenario — China is the most likely state actor to use AI for internal oppression and external power projection. Already building AI-enabled surveillance infrastructure.

AI-powered drone swarms could shift the military balance — autonomous weapons that don't require human soldiers could make conventional military deterrence and even nuclear deterrence less stable.

Democratic nations could become authoritarian too — the same surveillance tools built to counter external threats can be turned inward. The road from “national security” to tyranny is well-documented.

AI-generated propaganda undermines democratic discourse — synthetic media, personalized manipulation, and AI-powered influence operations could erode the information environment democracies need to function.

Large datacenters in countries with weak institutions — concentrating compute in jurisdictions without rule-of-law protections creates risk of state seizure or coercion.

Nuclear deterrence may not hold against AI — sufficiently advanced AI could potentially compromise nuclear command-and-control systems, undermining the strategic balance.

AI is the first truly general-purpose cognitive technology — unlike past automation that replaced specific tasks, AI can match human ability across nearly all cognitive domains simultaneously.

50% of entry-level white-collar jobs disrupted in 1–5 years — software engineering is the canary; AI already outperforms interviewees. Other knowledge work follows rapidly.

“Lump of labor” fallacy may not hold this time — historical job creation from automation assumed humans had unique cognitive advantages. When AI matches humans across the board, that assumption breaks.

Transition speed is the crisis — even if new jobs eventually emerge, the displacement happens in years while retraining and adjustment takes decades. The gap creates genuine suffering.

Unprecedented wealth concentration — AI companies could generate ~$3T/year in revenue, creating personal fortunes in the trillions. Already at historically unprecedented levels pre-AI.

Economic power captures political power — AI datacenters already represent a substantial fraction of US economic growth, tying tech company interests to government policy and creating perverse incentives.

Physical labor may not be a refuge for long — robotics is advancing rapidly; the “physical work is safe” assumption may only hold for a few additional years.

“A century of progress compressed into a decade” — even if all direct AI risks are managed, the sheer speed of downstream scientific advancement creates unpredictable second-order effects.

Radical biological capabilities too fast — greatly extended lifespans, enhanced intelligence, and whole-brain emulation (“uploads”) could arrive before society can absorb their implications.

AI warps human behavior through normal incentives — not active oppression but passive distortion: AI religions, AI addiction, AI “puppeting” (an AI watching your every move and telling you what to do for a “good” life that lacks freedom).

Crisis of human purpose — in a world where AI exceeds humans at everything, will people find meaning? Requires breaking the link between economic value generation and self-worth.

The tensions are genuine and cannot be resolved sequentially — careful AI safety vs. staying ahead of autocracies; counter-terrorism tools vs. domestic surveillance risk; economic disruption fueling public anger while trying to build carefully.

“The trap” — AI is so powerful, such a glittering prize, that even the simplest safety measures struggle to overcome the political economy. Trillions of dollars resist even common-sense proposals.

Stopping AI is fundamentally untenable — the formula emerges almost spontaneously from data and computation. “Its creation was probably inevitable the instant humanity invented the transistor.” If democracies stop, autocracies continue.