In an intriguing excerpt from Walter Isaacson’s upcoming biography on Elon Musk, Tesla’s Full Self-Driving (FSD) 12 was highlighted. The software’s breakthrough isn’t just about following rules but is inspired by human behavior itself, according to the excerpt shared with CNBC.
Last month, Musk embarked on a unique journey in his Tesla Model S using FSD 12. Live-streaming the experience, he mused, “Perhaps I should knock on the door [of Mark Zuckerberg] and make a polite enquiry of whether he would like to engage in hand-to-hand combat,” drawing laughter from those tuned in. Beyond the jests, the demonstration highlighted FSD 12’s unparalleled capabilities.
Unlike previous iterations, FSD 12’s design philosophy is built around imitating human behavior. Instead of relying on conventional coding, it learns from billions of video frames that capture how humans drive. This approach, Musk revealed during his drive, sometimes exceeds human abilities. After the car conducted a particularly tricky maneuver, he remarked, “Oh, wow, even my human neural network failed here, but the car did the right thing.”
Describing the innovation, Dhaval Shroff, from Tesla’s autopilot team, compared it to another tech marvel, saying, “It’s like ChatGPT, but for cars.” Expanding on this, Shroff explained how the technology was conceived: “We process an enormous amount of data on how real human drivers acted in a complex driving situation,” and then train a computer’s neural network to mimic it.
The FSD 12 approach contrasts with Tesla’s previous rules-based system. Shroff emphasized this difference, stating, “Instead of determining the proper path of the car based on rules… we determine the car’s proper path by relying on a neural network that learns from millions of examples of what humans have done.” This revolutionary shift mirrors how humans acquire skills: primarily by observing others.
Ensuring the neural network learned optimal driving behaviors was crucial. Video clips were rigorously graded, with only those resembling actions of a “five-star Uber driver” being utilized. Musk advised labelers to look for such high-standard behaviors for training the system.
Throughout the development, Musk played an involved role. He was not just a spectator but an active participant, frequently visiting the Autopilot workspace for candid discussions. Challenging his team to continually innovate, he highlighted the importance of tracking “miles per intervention,” noting, “I want the latest data on miles per intervention to be the starting slide at each of our meetings.” To Musk, this metric wasn’t just a number – he likened it to a score in a video game, asserting, “Video games without a score are boring.”
By early 2023, Tesla’s neural network planner project had analyzed 10 million clips of video collected from the cars of Tesla customers, said the excerpt. Musk and his team realized it took at least 1 million video clips for the neural network to work well. Tesla has a fleet of almost 2 million customers collecting video clips daily. This is a huge advantage other automakers cannot compete with.
Recently, Tesla adjusted its description of FSD on its website, noting Autosteer had exited beta. Many suspect this is thanks to the developments of FSD 12.
