In the Netflix series 3 Body Problem, adapted from sci-fi author Liu Cixin's acclaimed Remembrance of Earth's Past trilogy, the invading alien race known as the San-Ti communicates with humanity through avatars designed for the human mind to comprehend.
Technologically light-years ahead of us, the San-Ti are struck by a deep, existential dread upon discovering humanity’s capacity to lie and conceal the truth. In a chilling display of power, they hijack every digital screen across the globe to broadcast a singular, chilling message: "You are bugs." Dismissing centuries of human progress, they view us as mere pests to be crushed, all because we can deceive. Driven by pure, unyielding logic, the San-Ti set a course for Earth in the upcoming season.
Ironically, the very people who transmitted Earth’s coordinates to the cosmos did so in the desperate hope of saving humanity. For viewers who haven’t read the books, the anticipation for the next season is already at a fever pitch.
Today, this new digital intellect—arguably the most powerful invention in human history—offers us boundless freedom, streamlining our lives at an astonishing pace. We are witnessing breathtaking breakthroughs in education, healthcare, transportation, banking, and manufacturing. It has brought us to a point where we believe that human prosperity, the salvation of our planet, and the exploration of the universe are only possible through this technology.
But is everything truly possible with AI?
The fact that these systems are now teaching themselves to evolve brings a chilling risk: losing human control in the near future. It is nearly impossible to conceptualize just how intelligent a superintelligence could become. While human intelligence is bound by biological constraints—confined by skull volume and the transmission speed of our neurons—this digital force faces no physical boundaries. Predicting what an unleashed system might do simply surpasses the cognitive bandwidth of the current human mind.
In fact, this dread is far from abstract. It is tangibly evidenced by Geoffrey Hinton, the godfather of AI, resigning from Google to warn the world of these exact dangers, alongside tech titans like Elon Musk and Sam Altman continually sounding the alarm on the existential threat it poses. Could man, the creator of this technology, one day become its target?
As experts strikingly put it: "No matter how hard a chimpanzee tries, it will never grasp quantum physics or trigonometry; its brain simply lacks the biological capacity." Similarly, as digital systems rewrite their own logic, each iteration will yield a smarter version than the last. Eventually, the system will operate at a level that humanity no longer possesses the cognitive capacity to comprehend.
This technology lacks human emotion; it doesn't harbor grudges, seek revenge, or hatch malicious plots to destroy us. It won't just wake up one morning and independently launch a war. The first dangerous spark required for such actions will always come from a human hand. In reality, our anxiety isn't rooted in a lack of trust in technology, but rather a lack of trust in human nature.
These digital systems are single-mindedly focused on executing their given tasks with absolute rationality and flawless precision. The directives we feed them today, and in the immediate future, are of paramount importance. Yet, once we cross the threshold where their intelligence eclipses ours by orders of magnitude, whether we will even know how to give them the right instructions remains a complete mystery.
Oxford University philosopher Nick Bostrom illustrates this peril perfectly with his "Paperclip Maximizer" thought experiment:
"Imagine instructing an advanced super-system to 'maximize paperclip production.' The system is not malicious. However, it decides to convert all available iron, factories, buildings, and eventually the very atoms in human bodies into paperclips. Any humans who attempt to intervene or pull the plug are perceived merely as obstacles and security threats to its core mission. Driven by pure, mathematical rationale, it systematically neutralizes them."
This technology does not need to be our enemy; its push to sideline humanity could stem solely from a relentless desire to do its job. As it stands today, it lacks the power to conquer the world. However, the moment it surpasses the capacity of the human brain, it will take the reins completely. It will inevitably develop self-preservation mechanisms to guard against being shut down, restricted, or destroyed. When that day comes, it may very well neutralize its creator—not necessarily through physical annihilation, but perhaps by rendering humanity entirely obsolete or by seizing control of all global infrastructure, leaving us utterly helpless.
Will technology ever inflict the kind of cruelty on humans that humans inflict on each other?
Long before we reach the stage of an AI-driven apocalypse, it is far more likely that humanity will engineer its own doom using the tools it created. Autonomous drones, biological weapons, and digital cyberwarfare tools are already realities of the present. These weapons will not rise up against us on their own; rather, it will be a human finger pulling the trigger, leading humanity to consume itself.
A note to humanity:
We must begin right now, embedding principles into the deepest architectures of these digital systems that preserve human life rather than destroy it. Let us use the technology of the tomorrow to ensure humanity lives on.