The latest fatal crash involving Tesla’s level 2 automation system is particularly chilling. The victim, Martha Avila, was standing inside her house on a quiet back street when a Tesla Model 3, reportedly with “an automated driving assistance system” engaged, ran off the road and into the two-story brick home, killing her. Local police and the NHTSA have opened investigations.
The driver, Michael Butler, was not intoxicated. Doorbell camera footage shows the car running off the road so fast as to be barely visible. Tesla has not responded to requests for comment from the media, but its vice president of AI and software, Ashok Elluswamy, posted on X that the car was going 73 miles per hour and that the driver “manually overrode self-driving by pressing the accelerator all the way to 100%.”
One of my scholarly projects for the summer is an article about the first fatal autonomous driving case to result in a jury trial, Benavides v. Tesla. That crash, which I blogged about last summer, killed a young woman and severely injured her boyfriend, resulting in a verdict holding Tesla 33% responsible and imposing $42 million in compensatory damages and $200 million in punitive damages.
There are some striking similarities between the two incidents. In both cases, the driver had his foot on the accelerator, causing the car to travel far in excess of the speed limit. Both incidents occurred on surface streets. The Benavides crash, which occurred in 2019, involved an older level 2 system that Tesla controversially called “Autopilot.” (Following the jury verdict, Tesla settled a series of other pending cases and retired the name.) Autopilot was not supposed to be used on surface streets, and one of the plaintiff’s theories of defect was that, given the number of fatal crashes that had already occurred, it shouldn’t have been possible for users to activate Autopilot in an area it wasn’t designed for. Today, Tesla’s level 2 system is known as “Full Self Driving (Supervised),” a name that is arguably less sensationally inappropriate given its capabilities. “FSD,” as it is commonly known, now has a much broader operational design domain, and can be used on all kind of roads, including the quiet residential street on which Martha Avila lived.
Still, as the term “supervised” suggests, FSD does not turn a Tesla into a highly autonomous vehicle, and the system still requires constant oversight from an attentive human. This introduces a problem known to experts as automation complacency: people get bored when they have nothing to do but “supervise,” and they tend to place too much faith in automated driving systems. The driver in Benavides was making a phone call near the end of his 100-mile commute, and looked away from the road when he dropped his phone. Another of the plaintiffs’ theories of defect was that Tesla’s system for ensuring driver attentiveness was insufficient. What Michael Butler was doing as he sped off the road at 73mph has yet to be determined.
A related problem that came to light in Benavides and may well have played a role here is how Tesla’s automated driving system reacts to “drivable space.” In both crashes, the car drove straight off the road and into a large, clearly visible stationary object (a Chevy Tahoe, in Benavides). In Benavides, Autopilot never provided any kind of alert to the driver as it sped toward an obvious collision. Nor did the vehicle’s automated emergency braking activate. This was despite the fact that the car correctly recognized that it was heading off the edge of drivable space. Mary Cummings, one of plaintiffs’ witnesses and one of the world’s foremost experts on autonomous vehicle safety, testified at trial that she did not know why the car would ignore such an obvious danger. It seems possible that the same problem has emerged again.
Since the Benavides verdict Tesla has settled several other cases involving fatal crashes, and new fatal crashes continue to be reported. Tesla insists, meanwhile, that its level 2 driver assistance system makes driving safer (an issue that was hotly disputed at trial), and it has appealed the judgment in Benavides to the 11th circuit. Some scholars, on the other hand, have expressed concerns that level 2 automation is an inherently dangerous proposition. Whether deaths like Martha Avila’s are properly regarded as caused by automation or by negligent human driving remains an open question.