Pop-culture has been trying to warn us about AI and the “automation paradox” for decades, even if most of the artists crafting these stories probably had no idea what they were doing (and we certainly weren’t listening anyway). Now, new research about medical “deskilling” confirms their argument with terrible implications for any kind of writer who has decided to stake their work, creative lives, and careers on large language models. Buckle up, StoryTalkers, because we’re about to take a wild ride through geekdom, one of the 21st century’s deadliest airplane disasters, and the dangerous future of medicine.
February 2000
A new Star Wars novel called The New Jedi Order: Dark Tide I: Onslaught is published, written by Michael A. Stackpole and featuring a sequence that lodges itself in my brain so firmly that I’ll write about it in a Substack essay 26 years later.
Anakin Skywalker accompanies his aunt, Jedi Master Mara Jade, to Dantooine. Jade is suffering from a genetically engineer disease and Anakin – son of Han Solo and Leia Organa, for those who lack my geek cred – must protect her. Jade doesn’t waste the opportunity to further her nephew’s training, though. As their enemy, the Yuuzhang Vong, can’t even be detected by the Force, she insists that Anakin “turn off” his gifts and help them survive using nothing but his natural physical abilities and wits. This, she argues, is because Jedi have a bad habit of using the Force to find the easiest solution – not necessarily the best or most creative.
June 2009
Air France Flight 447, en route from Rio de Janeiro to Paris, crashes into the mid-Atlantic Ocean. All 228 people onboard are killed. The plane’s black box isn’t retrieved from the ocean floor for two years. France’s Bureau of Inquiry for Analysis for Civil Aviation Safety ultimately concludes its investigation the following year, 2012. While there were multiple factors that contributed to the crash, including iced-over airspeed sensors and poor crew coordination, the key failure occurred when Flight 447’s autopilot was disconnected and the pilots – who had only logged the minimal manual flying time – found themselves unable to respond effectively. They couldn’t respond effectively because of the “automation paradox”, which basically says that the more complex our machines get, the more we struggle when they fail.
Humans tend to trust automated systems and defer to their judgments. Over time, this reliance can lead to a measurable loss of manual skills. For example, I once criss-crossed the United States and Australia using nothing but physical maps or signs, but if I try to find my way anywhere today without Google Maps, I get lost at best or have a panic attack at worst.
In the case of Flight 447, this meant the pilots’ ability to intervene had been diminished by the very autopilot system they had come to rely on. There’s another term for this phenomenon, which is becoming commonplace in the age of AI: deskilling.
November 2003
A two-part episode of the animated series “Justice League” (2001-2004) airs entitled “Hereafter”, written by the legendary Dwayne McDuffie.
