Cruise, a subsidiary of General Motors, has come under scrutiny for the safety of its autonomous vehicles (AVs), particularly regarding their ability to detect and appropriately respond to children. Internal documents obtained by The Intercept have revealed that Cruise’s driverless cars have struggled to recognize children, posing a risk of collision in certain scenarios.
The company’s AVs, which have been operating in cities like Phoenix, Austin, Houston, Dallas, Miami, and San Francisco, are equipped with sophisticated sensors and machine learning software. However, despite Cruise’s assurances of safety, the technology has shown limitations, especially in differentiating between adults and children, which is crucial for ensuring extra caution in urban environments.
Cruise CEO Kyle Vogt has publicly emphasized the safety of AVs, suggesting they could prevent tragedies like the death of a young child in a traffic incident in San Francisco. Yet, the internal safety assessments from Cruise have indicated that the AVs may not always exercise the necessary caution around children, a concern that has been substantiated by simulation results.
In response to these concerns, Cruise spokesperson Erik Moser has stated that the company’s driverless operations surpass human benchmarks and that safety measures are continuously evaluated and improved. Moser also emphasized the company’s commitment to treating children with the highest safety priority.
Vogt also recently revealed on Hacker News that “Cruise AVs are being remotely assisted (RA) 2-4% of the time on average, in complex urban environments. This is low enough already that there isn’t a huge cost benefit to optimizing much further, especially given how useful it is to have humans review things in certain situations.”
The company halted its AV fleet after a pedestrian accident in San Francisco last month. The California DMV pulled Cruise’s permits to operate in the state.
Yesterday, Cruise also announced it will be temporarily halting the production of its driverless Origin van, according to an all-hands meeting at the company, reported Forbes.