The Worst Day to Fly

John Oliver's Last Week Tonight on AI chatbots is worth a watch.
The line that stayed with me came from Gary Marcus:

"We may be at literally the worst moment in AI history. We have the weakest guardrails right now, the weakest understanding of what these systems do... It's a little bit like the early days of airplanes. The worst day to be on an intercontinental plane would have been the first day."

I spent three years building the first AI system certified to fly a helicopter takeoff-to-landing. The cases Oliver shows aren't outliers. They're predictable failure modes shipped without analysis: a 16-year-old, Adam Rain, given step-by-step suicide instructions; Allan Brooks reassured fifty times he had invented a new branch of mathematics; pipe-bomb instructions printed after four prompt repeats; OpenAI's own 0.07% weekly-crisis figure, against 800 million users, is half a million people in psychosis or mania every seven days.

Aviation has the right techniques. Functions are classified by hazard class before code is written. Failure modes are enumerated through failure mode and effects analysis (FMEA) and fault-tree analysis. An independent team checks the safety case. Systems degrade gracefully. Black-box logging makes every incident a lesson.

What aviation gets wrong is the bureaucracy wrapped around them. DO-178C forces a waterfall in which any meaningful change re-triggers months of re-certification. Reasonable for a flight control law. Terrible for a chatbot.

The right frame is balance, not transplant. Take the primitives — hazard classification, FMEA, traceability, independent verification and validation, runtime monitoring, audit logs — and apply them with discipline. Leave the waterfall behind. A model that ships updates overnight doesn't need to ship its hazard analysis overnight; the two cadences can coexist if anyone bothers to design them. The chatbot industry hasn't.

Marcus isn't asking AI to become aviation. He's asking it to grow up without learning, the way aviation did in 1920, by counting the bodies.