Full Self-Driving

By Gideon Mandu

Tesla being arguably the leader in real-world Artificial Intelligence as it applies to real-world applications have revealed great details about their new Artificial intelligence tools being implemented to improve its self-driving capabilities. The lengths those developers go to achieve the feats the vehicle is currently able to perform are outstanding. Imagine a car driving you from home to work when you don't feel like driving but are motivated to head out and break some sweat chasing bitcoins or whatever.

“Full Self-Driving Capability” is an add-on available to new models. The $10,000 package lets the car automatically change lanes, navigate on highways, move into parking spots and emerge from a parking spot to arrive by the driver.

Tesla’s head of AI, Andrej Karpathy, described Tesla’s architecture as “building an animal from the ground up” that moves around, senses its environment and acts intelligently and autonomously based on what it sees. This is achieved by developing an artificial intelligence built on neural networks, the system just relying on 8 cameras as the main sensory input to the computer. The different raw camera footage is fussed up into a 3-dimensional vector space by the neural network looking to behave like a synthetic animal.
Image Credits: Tesla

Raw images go through a rectification layer to correct for camera calibration and put everything into a common virtual camera then, after which they are passed through a residual neural network (ResNet) to process them into several features at different scales. The multi-scale information is fussed by BiFPN or Weighted Bi-directional Feature Pyramid Network.

Then passed through a transformer module to re-represent them into the vector space in the output space, which feeds into a feature queue in time/ space and is then processed by a video module like the spatial recurring network video module, wherein different aspects of the module keep track of different aspects of the road and form a space-based and time-based queue, both of which create a cache of data that the model can refer back to when trying to make predictions about the road. The data is passed into the branching structures of the hydra net with trunks and heads for all different tasks.

The networks learn from the most complicated and diverse scenarios in the world, iteratively sourced from Tesla's fleet of nearly 1M vehicles in real-time. This coupled with their simulation systems builds to make the AI more powerful by implementing “a video game with Autopilot as the player.” This simulation allows for the model to be trained for scenarios that occur less often in the real world or situations that are deemed impossible to happen.

This software updated coupled with their new autopilot chip and Dojo AI training infrastructure hope to improve the companies Self-driving capabilities and also push them to other AI-based ventures one being the introduction of the humanoid robot said to be released next year to handle “the work that people least like to do,” said the CEO.