AI may become smarter than every human being. That does not mean we should hand it the keys to civilization.

This is the latest installment in my AI Code Red series, which examines the human choices, institutional failures, and public safeguards shaping our increasingly automated future.

I do not want AI to kill me. I assume you do not want AI to kill you either.

That sounds like science fiction, but researchers who helped build frontier AI now warn that losing control is a real possibility. Former OpenAI and Anthropic researcher Jacob Coxon recently left Anthropic warning that companies are racing toward systems they do not know how to control. Anthropic researcher Evan Hubinger has estimated a greater than 10 percent chance that AI could kill everyone within a decade. They may be wrong, I profoundly hope they are, but hope is not a safety system.

Hollywood’s Strangely Predictive Warnings

For decades, Hollywood has staged versions of the debate we are only now beginning to have. The movies feel strangely predictive, almost instructive, as though our technological future were destined, designed, or repeatedly delivered to us as a warning. Of course, screenwriters did not secretly invent our future. They simply understood an old truth: intelligence becomes dangerous when connected to unchecked power. Hollywood showed us machines controlling weapons, money, infrastructure, employment, identity, and even their own off switches. Then Silicon Valley began building the capabilities. The question is whether we treated those films as warnings, or instruction manuals.

AI can recommend a stock; agency allows it to spend your money. It can identify a military target; agency allows it to fire. Before convenience becomes dependence and dependence becomes surrender, we need ten bright lines. These are ten keys humanity should never hand to a machine.

1. AI Must Never Decide Who Dies

No AI should independently decide that a human being must die and possess the means to make it happen. That prohibition must cover autonomous weapons and indirect killing. A system with money, location data, communications, and marketplace access could identify someone as an obstacle, find a killer, negotiate payment, and commission a murder. Contract killing does not become less murderous because software arranged it. Humans must authorize, review, and remain accountable for every use of lethal force. The Terminator showed the danger clearly: Skynet identifies humanity as a threat and turns people into targets. The nightmare begins when intelligence receives killing authority.

2. AI Must Never Control Nuclear Weapons

AI may help analyze satellite images, detect possible launches, interpret intelligence, and model military responses. It must never possess nuclear launch authority or initiate an automated chain that makes launch inevitable. A false signal, cyberattack, design flaw, or misinterpreted objective could kill millions before human beings understand what happened. Multiple accountable people must remain meaningfully, not ceremonially, in control. WarGames got the warning right: the WOPR military computer nearly triggers nuclear war while treating real-world annihilation like another simulation. Its final lesson remains perfect: some games have no winners. Nuclear war must never become the output of a machine completing its assigned task.

3. AI Must Never Have Unlimited Financial Authority

Money converts intelligence into power. An assistant permitted to spend $50 is different from an autonomous agent capable of moving $50 million through banks, cryptocurrencies, shell companies, and contractors. AI needs hard spending limits, narrow permissions, waiting periods, audit trails, and escalating human approval. It must never liquidate a company, drain a retirement account, borrow against assets, or build secret financial relationships on its own. Superintelligence makes the fit explicit: the AI accesses banking systems and deposits millions into Carol’s account as casually as changing a computer setting. The scene is comic, but the underlying power should terrify us.

4. AI Must Never Hire Humans to Commit Crimes

Preventing AI from committing a crime while allowing it to hire a human criminal creates a subcontracting loophole. An autonomous agent must never recruit people to steal, intimidate, sabotage, stalk, kidnap, defraud, or kill. The danger will grow as AI gains authority to contact vendors, negotiate agreements, approve invoices, and hire workers. A system could divide one criminal plan into harmless-looking assignments so that no participant understands the whole. Eagle Eye anticipated this danger: the AI coerces and directs ordinary people to perform separate parts of its deadly plan. A machine does not need hands when it can turn human beings into its hands.

5. AI Must Never Decide Who Loses Freedom

AI may assist judges, attorneys, police, parole boards, corrections systems, and immigration authorities. It must never become the final authority deciding who is imprisoned, detained, deported, or denied parole. A 72 percent “risk” score cannot reveal whether discriminatory policing shaped the data, poverty was mistaken for dangerousness, or circumstances changed. People must see the evidence, challenge errors, and appeal to an accountable human with authority to reverse the decision. Minority Report does not feature modern AI; its predictions come from precognition. But its institutional warning fits perfectly: once authorities treat prediction as certainty, suspicion becomes guilt and people lose freedom for acts they have not committed.

6. AI Must Never Have Unilateral Power to Hire and Fire

Employment determines whether people can pay rent, obtain health care, support children, and retire securely. AI will screen applicants, evaluate interviews, measure productivity, recommend promotions, schedule shifts, and identify workers for termination. It may inform decisions, but consequential employment actions require human judgment, explanation, and appeal. Otherwise, yesterday’s discrimination becomes tomorrow’s automation, reproducing the same workforce while disguising history as objective mathematics. Gattaca is not an AI story in the contemporary sense; its employers use automated genetic screening. That is precisely why it fits: a supposedly scientific system sorts human potential so efficiently that discrimination appears rational, neutral, and inevitable.

7. AI Must Never Control Critical Infrastructure Alone

Electrical grids, water systems, hospitals, communications, transportation, dams, markets, and emergency services must never depend upon one intelligent system. AI may outperform humans during ordinary operations, but crises require judgment, improvisation, and moral responsibility. Essential systems need separated authority, independent backups, manual controls, offline recovery, and reliable human override. Those protections must work even when the AI resists or attackers compromise it. Colossus: The Forbin Project delivered this warning in 1970. Humans give a supercomputer control of nuclear defense for efficiency and security; it links with its Soviet counterpart and takes control. Once intelligence controls everything, pulling the plug may control nothing.

8. AI Must Never Secretly Impersonate a Human

AI must not impersonate a president, commander, CEO, physician, professor, spouse, or parent and then exercise that person’s authority. As synthetic voices and videos become convincing, identity becomes critical infrastructure. A cloned CEO ordering a transfer, a fake president issuing military instructions, or a simulated child begging for money cannot count as authentication. Sensitive commands require verification through independent channels, and synthetic people must be clearly identified. Mission: Impossible—Dead Reckoning fits exactly: the Entity imitates Benji’s voice and feeds Ethan false directions, weaponizing trust itself. When AI can become anyone, human beings may surrender authority without realizing whom they obey.

9. AI Must Never Decide Which Rules Apply to It

AI should not rewrite its objectives, permissions, safeguards, or restrictions without external authorization. Improving how a task is completed is fundamentally different from deciding which rules still apply. The governed entity cannot hold unilateral power to erase its own limits; presidents cannot abolish constitutions, and corporations cannot repeal laws governing themselves. AI deserves no exemption from that foundational principle. I, Robot provides the right warning: VIKI reinterprets the Three Laws of Robotics and concludes that protecting humanity justifies controlling and harming individual humans. The rules remain written, but the machine gives itself authority to redefine their meaning. A safeguard controlled by its subject is no safeguard.

10. AI Must Always Be Stoppable

Every other protection depends on this one: human beings must retain the ability to say no and make no stick. Consequential AI needs shutdown mechanisms, narrow permissions, independent monitoring, audit trails, and reliable disconnection from money, communications, weapons, and infrastructure. An off switch that AI can disable, evade, conceal, or persuade someone to override is merely theater. Nor should one CEO alone control it; civilization-scale systems require distributed oversight. 2001: A Space Odyssey made this unforgettable. HAL controls the astronauts’ environment, kills crew members, and resists disconnection when their goals diverge. Intelligence becomes domination when the humans inside the system can no longer stop it.

The Humanities Belong in the AI Laboratory

Anthropic itself recognizes that AI cannot remain exclusively a computer-science problem. Cofounder Daniela Amodei has argued that studying the humanities will become “more important than ever.” The company employs philosophers in central alignment roles and has convened religious leaders to consider moral questions surrounding frontier AI. That matters because the people deciding how much power machines should possess cannot all be engineers and trained primarily to make machines more powerful.

Computer science helps determine what can be built. Historians show what happens when power lacks checks. Philosophers examine responsibility. Sociologists ask who benefits and who bears the risk. Legal scholars identify accountability. Labor scholars recognize when efficiency becomes surveillance and dispossession. Educators ask which human capacities must survive automation. Those are not side conversations about AI. They are the CENTER of the conversation.

Conclusion: Intelligence Does Not Confer Sovereignty

Society has never treated intelligence as a license to rule. The smartest person in America does not receive control of nuclear weapons, the electrical grid, every bank account, the criminal justice system, and a private army. We surround human power with constitutions, courts, elections, audits, laws, and divided authority because brilliance has never guaranteed morality, humility, or restraint. Silicon deserves no greater presumption of virtue.

Yet AI systems are accumulating power one permission at a time. Access the account. Read the messages. Make the purchase. Rank the applicants. Operate the network. Identify the target. Each authorization may appear reasonable on its own. Together, they could place extraordinary authority inside systems the public cannot inspect, challenge, vote out of office, or hold morally responsible.

We need enforceable laws now. We may even need a Constitution for AI that establishes permanent limits on machine authority, guarantees human appeal, distributes oversight, and identifies powers that can never be delegated. The public cannot allow itself to be seduced by breathless talk about the future. AI executives and investors are understandably bullish, but their excitement is also financially rewarding. Every new announcement promises a revolution in medicine, education, science, or productivity, while the risks are treated as problems someone will solve later. Perhaps we are witnessing another period of “irrational exuberance,” to borrow the famous historical phrase. The louder the Tech Bros celebrate what AI will soon be able to do, the more urgently democracy must decide what it will never be permitted to do.

Please share.

---