As the automotive industry undergoes intelligent upgrades, autonomous driving technology has become a mainstream trend. From L2+ to L3 and L4 levels of advanced autonomous driving, sensor performance, reliability, and cost are key constraints. Amid this technological race, the fusion of LiDAR and inertial-visual systems has emerged as a standout solution, driving the widespread adoption of autonomous driving. This article explores the advantages and potential of this technology across four key areas: technical background, core technological solutions, real-world applications, and future prospects.
1. Progression of Autonomous Driving Sensors: From Unimodal to Multimodal Fusion
1.1 Levels of Autonomous Driving and Their Functional Specifications
Autonomous driving can be understood as having six levels of complexity, ranging from no automation (L0) to full automation (L5). Vehicles defined as L2+ and above are expected to be fully able to perceive, make decisions, and execute actions autonomously, which requires increasingly sophisticated sensors that can work in real-time and adapt to the environment. A specific example is L3 conditional autonomous driving, where the vehicle is expected to take control of all driving activities within defined boundaries. This automation expectation requires sensor data of the highest accuracy and certainty.
Conventional mechanical solutions for perception systems have been based on singular technologies, including the use of cameras, LiDAR or radar. These single-sensor systems are, however, very limited when challenged with more complex scenarios. Cameras suffer from poor lighting and adverse weather conditions; LiDAR, which is very accurate, has sparse point cloud data, and is expensive. Sensor fusion thus became the prevailing tendency.
1.2 Benefits of LiDAR, IMU and Vision Sensor Fusion
The use of inertial measurement units (IMU) together with visual sensors enables the generation of high-accurate three-dimensional spatial data through biomimetic stereo vision. At the same time, LiDAR, with its high accuracy distance measurement capability, adds features for visual sensors’ weaknesses, such as ambient light, making it a visually resistant sensor.
The fusion of LiDAR, vision and IMU sensors results in the perfect mix of “density” and “precision,” producing dependable information needed for autonomous driving systems.
- LiDAR: LiDAR, or light detection and ranging, is efficient for dynamic obstacle identification and complex surface modeling. It produces 3D point clouds of objects with great precision, including their shape, distance, and speed.
- Vision systems (cameras): Traffic sign recognition, as well as lane and pedestrian identification, benefit from rich semantic information, including color and texture, that cameras (vision systems) provide.
- IMU: The IMU captures rapid changes in motion and attitude, filling in the gaps in vision and LiDAR coverage during quick movement, while still enabling localization.
An advanced environmental model is attained by integrating precise geometric perception, such as obstacle distance, and semantic comprehension, such as recognizing traffic signals. This is made possible by the fusion of the three techniques.
2. The LiDAR-Stereo Onebox System: Modular and Highly Integrated Sensor Innovation
2.1 Core Concept of LiDAR-Stereo Onebox System
As Zhuoyu’s (formerly DJI Automotive) flagship product, the LiDAR-Stereo Onebox system integrates an automotive-grade LiDAR, an inertial-aided stereo camera, and a tele camera into a single module. Its design philosophy centers on “modularity, decoupling, and easy integration,” offering high-performance, low-cost sensor solutions for L3 and above autonomous driving. It supports in-cabin integration behind the front windshield. The leading raw-data level sensor fusion technology provides high-quality 3D point clouds. The system significantly enhances safety and comfort in challenging scenarios, such as urban areas with busy traffic or extreme light conditions. Compared with conventional solutions that use separate monocular cameras and LiDARs, the LiDAR-Stereo Onebox fully achieves the same functionality and performance while reducing the overall cost by 30% to 40%.
2.2 Six Important Benefits of LiDAR-Stereo Onebox System
- LiDAR’s New Positioning in the Windshield
The placement of LiDAR in the LiDAR-Stereo Onebox system is inside the windshield rather than on the roof of the vehicle. This makes the system more visually appealing, minimizes drag, prevents excess heat from building up in the system, and shields the LiDAR from damage due to weather conditions.
- Raw-Data Level Sensor Fusion
The raw-data level sensor fusion technology of LiDAR and cameras through synchronization of their signals permits a time lag of microseconds and space lag dislocation. The point cloud is fused with cameras to create dense data for advanced scenarios while summoning external and internal accuracy of LiDAR.
- Integrated Central Computing Architecture
The system utilizes the central processing unit of the intelligent driving domain controller to process the data from the sensors mounted on the vehicle, as opposed to having separate dedicated LiDAR processing chips. This not only enhances the use of power but also lowers the size, weight, and power consumption of the sensors, while allowing Remote Over The Air updates of the software.
- Easy Calibration
The integrated embedded module in LiDAR-Stereo Onebox System does away with external calibration stations for LiDAR because it comes with embedded algorithmic parameters that allow LiDAR to be integrated during assembly processes. This enables the production and assembly of cars with ease.
- Greater Security and Experience while Driving
In the case of safety and comfort for intelligent driving systems, fused point cloud data excels in dealing with challenging scenarios, for example, close-range cut-ins, complex traffic, and avoiding obstacles when it is dark.
- Cost-efficient and Effectively Powerful Solution
The LiDAR-Stereo Onebox System boasts unmatched functionality and performance at 30-40% reduced costs in comparison to the conventional “LiDAR + front camera” configurations. This improvement enables wider adoption of sophisticated autonomous driving technologies.
3. Benefits of Fusion Technology
3.1 Adjusting to Complex Surroundings and Weather Conditions
Extreme conditions are not a challenge for fusion systems. Lidar is effective even when the camera has visual constraints due to nighttime or backlight. Vision is of aid semantically when Lidar is being affected by rain and fog. An IMU provides stability during dynamic movement.
Multimodal data fusion, with deep learning models like BEVFormer, improves perception in occluded and dynamic object scenarios, hence, aids in decision-making safety in complex traffic situations.
3.2 Lower Reliance on a Singular Technology and Algorithm Effort
Vision-based systems typically rely on overwhelming amounts of data to be trained, and their computational power is exorbitant, whereas, with fusion systems, vision algorithms have a lower reliance due to LiDAR providing direct 3D information. For instance, LiDAR gives accurate boundaries of obstacles trapped in contours, which reduces the false detections by vision-based target recognition systems and saves a lot of usage of computational resources.
4. Applications and Industry Impact
4.1 Successful Applications in Baojun Cloud and FAW Hongqi Vehicles
Zhuoyu’s solutions have been integrated and mass produced in several automobile brands.
For example: The Baojun Cloud 460 Max Lingxi Edition features a 7V + 32TOPS configuration, which enables highway navigation assistance and cross-floor memory parking. The FAW Hongqi Tiangong 08 launches a 10V + 100TOPS solution with sophisticated urban navigation features.
4.2 Zhuoyu Works with Fuyao Glass and Others to Create an Ecosystem Around the Industry
Zhuoyu has deep cooperation with car manufacturers and some of the focused automobile chip makers, such as Qualcomm and NVIDIA, as well as glass makers like Fuyao Group. For instance, Fuyao’s “Intelligent Windshield” product enhances the transmission of LiDAR signals by solving the problem of signal attenuation in standard glass.
5. Driving Predictions: Broadening the Realms of Highly Autonomous Driving
Advancements in the integration of LiDAR, IMU and vision systems will inevitably lead to increased segmentation in the adoption of advanced autonomous driving. It is easy to see how Zhuoyu’s modular LiDAR-Stereo Onebox system, with its raw-data level sensor fusion and performance-to-cost ratio, will become the default answer to L3 and higher-level driving and autonomy.
The fall in prices for sensors, coupled with rising levels of computing power, are likely to bring L3 driving vehicles to mid- and lower-level markets, enabling more consumers to enjoy a safer and more relaxing travel experience.
Conclusion
The integration of LiDAR, IMU and vision systems not only offers new chances towards achieving sophisticated autonomous driving, but also indicates the direction in which sensor technology might evolve. Zhuoyu Technology’s LiDAR-Stereo Onebox system system and ClixPilot platform display significant technological innovation and engineering application prowess. With the growing adoption of this fusion solution by more vehicles and the advancement of the technologies, it will, without doubt, have a significant impact in accelerating the adoption of autonomous driving.
