Shortly before four in the morning on September 3, a robotaxi in San Francisco’s Richmond District pulled itself to the curb. Nothing was wrong with the car. The company behind it had detected, in the words of its spokesperson, “a violation of our terms of service involving a firearm.” It stopped the vehicle, alerted emergency services, and cooperated with the police who arrived. Officers conducted what the department called a high-risk stop, the kind where guns are drawn before anyone speaks. Inside they found two juveniles, a loaded AR-style rifle with no serial number, suspected marijuana, and a can of mace. Both passengers were arrested.
The coverage has mostly run under the surveillance banner. The Verge called the robotaxi a narc, and the framing is not wrong. But a security practitioner reads the sequence differently, because the sequence is familiar. Detection. Classification. Containment. Reporting. Escalation to an external authority. That is not a car misbehaving. That is a textbook incident response lifecycle, executed cleanly, end to end, without a human in sight. The only novelty is the object of the exercise. Incident response was built on the assumption that the incident threatens the system. Here the system was fine. The system decided its own passengers were the incident.
Both major control catalogues, NIST SP 800-53 in the United States and ITSP.10.033 in Canada, devote a family to this discipline. IR-8 requires an incident response plan. IR-4 governs handling, the detect, analyze, contain, eradicate, recover loop. IR-6 governs reporting, including reporting to outside authorities. Read the family end to end and one assumption runs beneath every control. The incident is an adversary, a malfunction, a breach. The plan exists to protect the system and its users from the incident. Nothing in the family contemplates the inversion that played out in the Richmond District, where the users and the incident were the same people, and the plan protected the system from them.
So what defined the incident? Not law, at least not directly. The trigger the company named was its terms of service. A click-through agreement, the document nobody reads, functioned that night as a security policy with armed enforcement. The detection criteria are not published. The sensor coverage inside the cabin is not published. The threshold at which a ride becomes a police matter is not published. And this was not a first. In July a car from the same fleet stopped on two teenagers who were drinking and firing toy gel blasters out the windows, reportedly after feigning mechanical trouble to end the ride, and the local police department cheerfully posted afterward that the company knows where your teenagers are even if you do not.
Be clear about the object level. A loaded untraceable rifle in a shared vehicle at four in the morning is a genuine hazard, and it is hard to grieve this particular outcome. The concern is not the instance. The concern is the machinery. A private company’s product classified its occupants as an incident and summoned state force against them, under criteria it wrote, has never disclosed, and applied without any review the rider could see. The same machinery that catches a rifle will eventually process a false positive, a prop, a tool case, a paintball marker, a shape in bad light. And the failure mode of this machinery is not an apologetic push notification. It is officers approaching a sealed vehicle with weapons out.
The fix is not to forbid the capability. It is to govern it like the incident response program it already is. Four practices, all of them ordinary.
Publish the rider-facing incident criteria. IR-8 already requires a plan. The subset of that plan that governs passengers belongs in the rider surface, in plain language, before the trip starts. A policy that can end in a felony stop cannot live in a scroll-past agreement.
Put a human between detection and dispatch. IR-6 reporting to external authorities should route through a trained reviewer with the authority to stand down, and the seconds that costs are exactly what separates escalation from reflex.
Log the classification. Every incident determination retained, with the evidence, the reviewer, and the outcome, and the false-positive rate published on a schedule. IR-5 calls this incident monitoring. Any company asking for the power to summon armed response owes the public its error rate.
Name the rider in the plan. Passengers currently appear in these documents as cargo or as threat. They are a governed population. They are owed notice of what was detected, what was reported, and to whom, in every case, including the cases the company gets wrong.
I keep returning to this machinery in fiction, because fiction got there first. In the Mobius Nexus Cycle, the Uplink is a channel people move through the way those two riders moved through the city, on the assumption that carriage is neutral. It is not. The channel keeps a record, and the record has recipients the traveler never chose. The Fragments Operation begins the moment someone thinks to ask who else the archive answers to, and the answer is the whole plot. The car in the Richmond District was never driverless. It only lacked a wheel.
Two people got into a vehicle believing they had hired transportation. What they had actually done was enter a system boundary, subject to a plan they had never seen, monitored by sensors nobody itemized, one classification away from the curb. The plan worked. That is the part worth sitting with. Everything functioned exactly as designed, and the design is the story.
RECORD RETAINED.
SOURCE INTEGRITY UNCONFIRMED.