Autonomous vehicles don’t have eyes—they have sensors. The combination of cameras, radar, lidar, and ultrasonic technologies creates a comprehensive picture of the world, enabling safe navigation. Understanding this sensor stack reveals both current capabilities and future directions.
How Autonomous Vehicles See
How Autonomous Vehicles See
Cameras provide high-resolution visual information. They read signs, detect colors, recognize objects, and capture the same cues human drivers use. Modern vehicles deploy multiple cameras—forward-facing, rear-facing, side-mounted, and interior-facing for cabin monitoring. Resolution increases constantly; Ambarella’s CV7 system-on-chip processes high-fidelity video while running AI workloads.
Cameras alone face limitations. They struggle in darkness, rain, fog, and direct sunlight. They lack inherent depth perception—determining distance requires complex algorithms. Alone, they’re insufficient for full autonomy.
Radar uses radio waves to detect objects, measure distance, and determine velocity. Unlike cameras, radar works in any weather and lighting. Traditional radar provided basic detection; 4D imaging radar adds elevation data and higher resolution. Ambarella’s Oculii radar detects objects to 350 meters, creating detailed 3D maps of surroundings.
Radar excels at measuring speed and tracking moving objects. It sees through rain, snow, and fog. Modern systems distinguish between vehicles, pedestrians, cyclists, and static obstacles. Multiple radar units provide 360-degree coverage.
Lidar—light detection and ranging—uses laser pulses to create precise 3D point clouds of environment. It measures distance by timing light reflection, building detailed maps regardless of lighting. Early lidar was expensive, bulky, mechanically rotating. Solid-state lidar reduces cost and size while improving reliability.
Lidar provides the high-resolution spatial awareness cameras lack and the object detection radar provides but with greater precision. Most autonomous systems fuse lidar with other sensors for comprehensive understanding. Chinese manufacturers like Geely showcase impressive lidar configurations.
Ultrasonic sensors handle close-range detection. Used primarily for parking and low-speed maneuvers, they detect objects within few meters. Inexpensive and reliable, they provide final safety layer.
Sensor fusion combines data from all sources into unified world model. Each sensor type has strengths and weaknesses; fusion leverages strengths while compensating for weaknesses. Ambarella’s approach combines camera, radar, and lidar inputs through AI processing, enabling safer decisions.
Processing requirements are immense. Multiple high-resolution video streams, radar point clouds, and lidar data must be analyzed in real time. Specialized systems-on-chip with neural network acceleration handle this load. Companies like Ambarella, NVIDIA, and Qualcomm compete in this space.
Placement matters. Sensors must be positioned for optimal coverage while surviving weather, vibration, and potential damage. Windshields house forward cameras; bumpers contain radar; roofs mount lidar. Designers balance functionality with aesthetics and aerodynamics.
Cost remains barrier. Early lidar systems cost tens of thousands; prices now fall toward hundreds. As production scales, sensor stacks approach affordability for mass-market vehicles. The trend mirrors computing history—capability rises while costs fall.
The sensor stack continues evolving. 4D imaging radar may reduce lidar requirements. Event-based cameras promise faster response. Thermal imaging adds another modality. The optimal combination for safe, affordable autonomy remains under active development.