Tesla announced it is building foundation models for autonomous robots, made today via its new Twitter account ‘Tesla AI’ (@Tesla_AI).
The company’s multi-modal neural networks, which are already in customer vehicles, take in various modalities such as camera videos, maps, navigation, IMU (Inertial Measurement Unit), and GPS, etc. Tasks such as Occupancy prediction, which predict the probability that a certain 3D position is occupied, are already quite general in what they represent. This technology can be used for collision avoidance by any robot, showcasing its versatility.
Tesla is building the foundation models for autonomous robots pic.twitter.com/VUES9jU3ze
— Tesla AI (@Tesla_AI) June 21, 2023
The company is leveraging fleet scale auto-labelling, using video data from multiple trips in the same location to reconstruct entire scenes. Tesla is also building off state-of-the-art generative modeling techniques, enabling the prediction of possible outcomes given past observations across multiple camera views. These imagined futures can be action-conditioned to produce different outcomes, with the videos generated entirely by the neural network using different prompts.
These video foundation models are set to serve as the brain of both the car and the Optimus robot. They will learn from a vast set of extremely diverse data from the Tesla fleet and will be trained on enormous amounts of compute.
Tesla has also announced the start of Dojo supercomputer production in July 2023, essentially its generalized neural network training computer. The company projects to be among the top five in the world by February 2024. The total amount of Tesla compute is expected to hit 100 exa-flops by October 2024.
“We just need a crazy amount of training compute to get full self-driving 10X safer than human, but can’t get enough GPUs, plus prices are extremely high!” said CEO Elon Musk today. He also noted Dojo has “been online and running useful tasks for a few months” already.
Essentially, Dojo is a special kind of computer chip made by Tesla. It’s different because it’s designed to work really well with AI. This chip is also unique because it’s made in a new way that helps it work faster and stay cooler.
Because of these innovative features, Tesla will be able to teach its AI to do things ten times faster within a year. This means that Tesla cars will be able to share information with its fleet, as it happens. For example, if one car sees a pothole, it can tell the car behind it to watch out.
Tesla is inviting individuals to join the Tesla AI team to build the future of robotics, marking an exciting new chapter in the company’s journey towards autonomous technology.