Finding
The stack is collapsing into one neural network, which moves the hard problem from code to training data and compute
A classic autonomy stack is a chain of engineered modules: perception builds a scene, prediction forecasts other road users, planning picks a path and control executes it. Since Tesla's FSD v12 in 2024, the leading systems have replaced most of that chain with a single network trained on fleet video to map sensor input to driving output. XPeng's VLA (vision-language-action) model, Wayve and Nvidia's Alpamayo follow the same approach.
The shift changes what limits progress. A modular stack improves with engineering hours spent on each block; an end-to-end model improves with more and better training data, larger models and more training compute, which is why training clusters grew from tens of thousands to hundreds of thousands of GPUs. It also makes behavior harder to explain and certify, so production systems keep a separate safety layer - rule checks, redundant perception and a minimal-risk maneuver that brings the vehicle to a safe stop - around the network.
Layers
Stack layers
| Layer | What it does | Typical methods |
|---|---|---|
| Perception | Builds a scene: objects, lanes, free space, occupancy | Cameras, radar, lidar; bird's-eye-view and occupancy networks |
| Prediction | Forecasts how other road users will move | Trajectory models, multi-modal outcomes |
| Planning | Chooses a path and behavior | Cost functions, learned policies, reinforcement learning |
| Control | Turns the plan into steering, braking, acceleration | Model-predictive control; drive-by-wire |
| Localization | Knows where the vehicle is | GNSS, inertial sensors, maps or map-light methods |
| Safety layer | Checks the plan, handles faults, stops safely | Redundant compute, rule monitors, minimal-risk maneuver |
Sources
Sources and scope
- Scope
- Software architecture of autonomous vehicles and robots
- Sources
- Tesla; XPeng; Wayve; Nvidia (CES 2026); ElectronsX analysis
- Updated
- October 2026