Lane-Level Localization and Navigation Systems of Autonomous Vehicles

The Lane-Level Localization technology of autonomous vehicles is used to determine the vehicle’s position and pose. It is an important component of the autonomous vehicle localization and navigation system. The autonomous driving system includes multiple subsystems such as environmental perception, decision-making and planning, and motion control^{1 2}. All these subsystems require accurate vehicle pose information to ensure the accuracy of perception, decision-making, planning, and motion control ³. For example, a position error of a few decimeters may cause the vehicle to be localized in the wrong lane, leading to incorrect driving behavior decisions and potentially causing traffic accidents. Therefore, the localization accuracy of autonomous vehicles needs to reach the lane level, i.e., decimeter-level or even centimeter-level accuracy ⁴.

Core Requirements for Lane-Level Localization Systems

In addition to the high accuracy requirement, the localization system must also meet the following requirements:

1. Stability: The localization system must be capable of handling various influencing factors under different road and weather conditions, while maintaining localization accuracy in the presence of signal interference or sudden error surges. 
2. Real-time Performance: The localization system needs to provide real-time vehicle position information promptly to fully enable the instant driving decision-making and control of autonomous vehicles. 
3. Cost-effectiveness: The localization system must meet cost-effectiveness requirements to facilitate the mass production and application of autonomous vehicles.

Based on these requirements for the localization system, the following analyzes common localization methods and their respective advantages and disadvantages.

GNSS/INS Integrated Navigation

The Global Navigation Satellite System (GNSS) ⁵ is currently the most commonly used method for vehicle localization, with the Global Positioning System (GPS) being the most widely applied. Typical GNSS provides an average accuracy from several meters to more than 20 meters ⁶, which puts the limits to the application in autonomous vehicles. Solutions like Differential GNSS (DGNSS) ^{7 8}, Assisted GNSS (AGNSS) ⁹ and Real-Time Kinematic (RTK) positioning ¹⁰ can be used to improve the accuracy of GNSS. DPGS uses the known position of fixed infrastructure units to compute the GNSS measurement errors, which are then broadcasted to the GNSS units mounted in the vehicles to improve the precision, with an average accuracy of 1-2 meters. AGNSS makes use of information provided by the cellular network to minimize the time delay while acquiring the position from the satellites, and increase the signal coverage. However, it offers lower positional accuracy than DGNSS. RTK-GNSS uses carrier phase measurements taken from the GNSS signals in order to estimate the position of the vehicle relative to a base station with known co-ordinates, which can be achieved to centimeter level. However, because of its high cost, it is not possible to employ it on a large scale.

Limitations of GNSS and the Role of INS

Although DGNSS, AGNSS and RTK-GNSS have made the traditional GNSS positioning more accurate, they still basically can be used in open areas. Due to several factors like signal blockage, errors etc., their reliability still remains poor in obstructed environments including environments with high-rise buildings and large trees. Furthermore, the output frequency of GNSS is relatively low while the planning and control subsystems of autonomous driving need high frequency positioning information. Therefore, GNSS localization accuracy on its own cannot always be trusted and typically requires the fusion of other sensors to obtain localization signals.

The Inertial Navigation System (INS) uses the accelerometers and gyroscopes making up an Inertial Measurement Unit (IMU) to measure triaxial acceleration and angular velocity, respectively, and calculates the position of the vehicle in relationship to its original position by integration. However, IMU measurement is sensitive to a number of noise. These noises are magnified in the several levels of integration from acceleration to displacement and angular velocity to attitude angles with errors growing exponentially and accumulating over time.

GNSS/INS Coupling Methods

Integrated navigation technology can be seen as the method of comprehensive processing of multiple sources of navigation information. In order to make full use of the respective benefits of GNSS and IMU, in most cases, both will be mixed to form a GNSS/INS integrated navigation system ^{11 12 13}. GNSS/INS integrated navigation can be divided into three common integration methods: loose coupling, tight coupling and deep coupling.

1. Loose coupling has a plain structure, which combines the results of positioning independently calculated from GNSS and INS.
2. Tight coupling uses the raw GNSS data to make measurement updates for the INS directly, making positioning output even if less than 4 satellites can be seen.
3. Deep coupling is a step further in terms of utilizing GNSS information, it is a method for the integration of the two body position sensors (INS and GNSS) at the hardware level, this method enables GNSS to assist the measurement update of the INS while enabling the calculation results of the INS to assist the calculation of the GNSS receiver tracking loops, in order to improve the performance of the integrated navigation method.

Tight coupling and deep coupling can improve the quality of GNSS positioning to a certain extent, but their mathematical models and system implemented are relatively complicated, etc. Tight coupling and deep coupling are more commonly used in professional equipment.

On the one hand, GNSS/INS integrated navigation uses absolute position information of GNSS to suppress the accumulation of errors of IMU. On the other hand, when the signal quality of the GNSS signals is reduced or lost in obstructed environments such as near high-rise buildings, the INS can guarantee reliable positioning output for a period of time. However, the positioning accuracy of GNSS/INS integrated navigation is absolute, and it is dependent on the positioning accuracy of GNSS. When the GNSS signals are subjected to long periods of degradation or failure, under these conditions, significant errors can be introduced into GNSS/INS integrated navigation as a result of the rapid accumulation of the IMU measurement errors. This situation is rather common in a complex structured road environment.

LiDAR-Based Localization

Another positioning technology that is developed in the recent years is the LiDAR-based localization technology. LiDAR sensor is a device that is used to scan the distance to a target with the help of several laser beams. The distance is calculated by each beam, regarding the time that was spent by the signal to reach the receiver, and also the intensity of infrared of the obstacle. LiDAR system captures much ranging and infrared data, which includes the features of the traffic surroundings like road sides, road features, barriers and the participants of the traffic. These data are proceeded to produce a feature map of the environment. In real-world driving, the LiDAR measures real-time data to extract features which are compared with the pre-defined feature map to calculate the position of a vehicle. One more localization technique that utilizes LiDAR is referred to as laser odometry, and involves comparing the point cloud data between frames with the previous frame to determine the relative pose difference between two frames.

Camera-Based Localization Methods

With the development of computer vision technology, cameras, as low-cost sensors, are widely used in unmanned driving. In this context, how to effectively utilize the perception information from cameras for navigation and positioning becomes particularly important. On one hand, visual SLAM positioning is a commonly used positioning method. It uses the images captured by the camera to match with the pre-collected high-precision visual map to obtain the camera’s positioning in space, and then calculate the vehicle’s position. In addition, visual odometry ^{14 15} estimates the relative pose change of itself by calculating the change between adjacent images.

Localization Using Traffic Markings and Lane Lines

On the other hand, the research focuses on the positioning based on the recognition of traffic markings by cameras. Since lane lines are common traffic markings on structured roads and are excellent sources of positioning information, using the relative lateral distance between the lane lines recognized by the camera and the camera itself (this is referred to as the line-side distance) to assist in positioning is the most common approach. As shown in following Figure, \( \boldsymbol{p_l^1 – p_l^4} \) and \( \boldsymbol{p_r^1 – p_r^4} \) represent discrete points on the left and right lane lines respectively. The projection of the rear axle center of the vehicle on the left lane line is located between \(\boldsymbol{p_l^2}\) and \(\boldsymbol{p_l^3}\). These two points are referred to as the left boundary points, and the line connecting these points is referred to as the left boundary line. Similarly, the right boundary points are \(\boldsymbol{p_r^2}\) and \(\boldsymbol{p_r^3}\), and the line connecting these points is referred to as the right boundary line. When the position coordinates of the left and right boundary points and the linear equation of the left and right boundary lines are known, the vehicle position can be calculated based on the line-side distance and the relative position relationship between the vehicle and the lane lines.

Research on Multi-Sensor Fusion with Cameras

Rose et al. ^{16 17} combined GNSS, IMU, cameras and lidar for positioning. They used LiDAR and cameras to measure the lateral distance to assist positioning, thereby improving the positioning accuracy of GNSS/IMU. However, they lacked consideration for the lane where the vehicle was traveling and the changes in the left and right boundary points. Nevertheless, only by ensuring the accuracy of the current lane and boundary points can the accuracy of positioning using the lateral distance be guaranteed. Tao et al. ^{18 19 20} proposed a camera observation model, integrating L1-GNSS, wheel speed sensor, yaw rate, camera and lane lines in the map to form a positioning system. The difference lies in the different map matching methods adopted.

The map-matching method that takes into account distance, heading and topological connection relationships matches the lanes. By using cameras to observe the equation coefficients and designing matching functions, this method has a high degree of dependence on the accuracy of camera recognition and lacks robustness. Tao et al. utilized the lateral distance to improve the error model of GNSS in the lateral direction and proposed a matching method. Firstly, they considered the types and distances of the left and right lane lines, as well as the determination of some candidate lanes, and then selected the lane with the shortest distance. This matching method relies on the distance factor when determining the final lane, and matching errors may occur when the positioning coordinate error is large.

These studies all involve integrating the lateral distance of the line with other sensing sources for positioning. On one hand, accurately positioning the line using the distance recognized by the camera requires matching the vehicle to the correct lane on the map. When a vehicle changes lanes in the same direction and approaches the middle lane line, even a small positioning error can lead to a lane mismatch. The map-matching methods described in these literatures are difficult to determine the accurate lane. When the lane is mismatched, using the incorrect lane line for assistance positioning not only does not improve the positioning accuracy but also causes positioning errors. On the other hand, most of these studies utilize the Extended Kalman Filter method for integrated positioning, integrating the lane line lateral distance information into the filter, generating a single observation equation based on a single lane line, and the anti-interference ability against large lateral errors is limited.

Wheel Odometry Localization

The wheel odometer measures the number of wheel pulses per unit time through the wheel encoder to calculate the wheel speed and travel distance. Due to the need of the anti-lock brake system (ABS), the wheel encoder sensor is installed on the vehicle, which makes the wheel odometer a universal and cost-effective positioning technology. Compared with the accelerometer sensor of the IMU, the wheel odometer requires fewer integration steps to determine the vehicle position, thereby reducing errors in the integration process and providing a better positioning solution. The wheel odometer was initially widely used in mobile robots and later gradually applied to vehicles to complete the positioning task.

Applications and Challenges of Wheel Odometry

Thrun et al. ²¹ addressed the issue that the GNSS/IMU fusion algorithm had difficulty estimating the vehicle’s posture and IMU deviation when GNSS signals were weak or the vehicle was stationary by considering the wheel odometer. Funk et al. ²² utilized the wheel odometer to complement the deficiencies of the visual-inertial odometer in automotive applications. Moreover, in some specific conditions, such as low-light, low-speed driving, and parking scenarios ^{23 24}, the wheel odometer can serve as the primary positioning method.

During the actual driving of the vehicle, the positioning accuracy of the wheel odometer is affected by various factors. On one hand, even if the model parameters such as tire diameter are calibrated during installation, factors like wheel wear and load changes during vehicle operation will cause these parameters to change dynamically, thereby affecting the output of the odometer. On the other hand, conditions such as wheel slip and uneven road surfaces will result in the odometer’s measured driving distance not being equal to the actual driving distance of the vehicle. Therefore, the measurement error of the wheel odometer has a high uncertainty, and accurately estimating this error is the key to improving the positioning accuracy of the wheel odometer.

Improving Wheel Odometry Accuracy

Fazekas et al. ²⁵ proposed an offline iterative estimation algorithm, and further presented an online estimation method ²⁶. In each iteration cycle, they estimated the wheel circumference using Kalman filtering and least squares method to improve the accuracy of the wheeled odometer. They further used nonlinear least squares method to recursively estimate the wheel circumference to enhance the estimation accuracy of wheel speed ²⁷, and utilized Gaussian-Newton regression and Kalman filtering to calibrate the parameters of the wheeled odometer ²⁸. Welte et al. ²⁹ precisely calibrated the model parameters using the Rauch-Tung-Striebel smoothing method, thereby obtaining a state estimation close to the actual situation. In summary, these studies used traditional state estimation methods to improve the positioning accuracy of the wheeled odometer, but these methods relied on precise parameter adjustments and had poor adaptability to various working conditions.

On the other hand, due to the influence of road conditions and the model’s own parameters on the model’s accuracy, traditional state estimation methods are difficult to obtain an accurate model under the influence of multiple factors. Deep learning is a data-driven method that can learn complex nonlinear characteristics and uncertainties from data. Researchers have proposed a deep learning-based method to explore the error drift characteristics of IMU over time to obtain a better GNSS/INS combined navigation solution ^{30 31 32 33}. Some deep learning methods are also applied to wheeled odometers. The LSTM model was used to learn the uncertainty of wheel odometer measurements ³⁴, and WhONet model was also used and further improved the accuracy compared to the LSTM model ³⁵.

Comparison of Localization Methods

The following table compares the differences among the aforementioned positioning methods in terms of sensor, advantages, and limitations.

Conclusion and the Need for Multi-Sensor Fusion

From the above table, it can be seen that different positioning methods have their own advantages and limitations. In the face of complex and variable structured road environments, a single sensor and positioning source is difficult to ensure continuous, reliable and high-precision positioning requirements. Multi-sensor fusion positioning is a reliable solution. The main sensors used in existing low-cost positioning algorithms are GNSS, IMU, cameras and wheel odometers. The separate application of each sensor lacks adaptability and robustness to the working conditions. How to design a robust and low-cost positioning method to improve the accuracy, stability and reliability of positioning is an urgent problem to be solved.

1. JO K, KIM J, KIM D, et al. Development of autonomous car—Part I: Distributed system architecture and development process[J]. IEEE Transactions on Industrial Electronics, 2014, 61(12): 7131-7140.
2. BADUE C, GUIDOLINI R, CARNEIRO R V, et al. Self-driving cars: A survey[J]. Expert Systems with Applications, 2021, 165: 113816.
3. VIANA K, ZUBIZARRETA A, DIEZ M. A Reconfigurable Framework for Vehicle Localization in Urban Areas[J]. Sensors, 2022, 22(7): 2595.
4. LEVINSON J, MONTEMERLO M, THRUN S, Map-Based Precision Vehicle Localization in Urban Environments[J]. Robotics: Science and Systems, 2007, 3: 121-128.
5. GYAGENDA N, HATILIMAJ V, ROTH H, et al. A review of GNSS-independent UAV navigation techniques[J]. Robotics and Autonomous Systems, 2022: 104069.
6. Tan H S, Huang J. DGNSS-based vehicle-to-vehicle cooperative collision warning: Engineering feasibility viewpoints[J]. IEEE Transactions on Intelligent Transportation Systems, 2006, 7(4): 415-428.
7. REZAEI S, SENGUPTA R. Kalman filter-based integration of DGNSS and vehicle sensors for localization[J]. IEEE transactions on control systems technology, 2007, 15(6): 1080-1088.
8. STATECZNY A, SPECHT C, SPECHT M, et al. Study on the positioning accuracy of GNSS/INS systems supported by DGNSS and RTK receivers for hydrographic surveys[J]. Energies, 2021, 14(21): 7413.
9. DJUKNIC G M, RICHTON R E. Geolocation and assisted GNSS[J]. Computer, 2001, 34(2): 123-125.
10. ZHANG J, WEN W, HUANG F, et al. GNSS-RTK Adaptively Integrated with LiDAR/IMU Odometry for Continuously Global Positioning in Urban Canyons[J]. Applied Sciences, 2022, 12(10): 5193.
11. LYU P, BAI S, LAI J, et al. Optimal time difference-based TDCP-GNSS/IMU navigation using graph optimization[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-10.
12. ADUSUMILLI S, BHATT D, WANG H, et al. A low-cost INS/GNSS Integration Methodology based on Random Forest Regression[J]. Expert Systems with Applications, 2013, 40(11): 4653-4659.
13. NAGUI N, ATTALLAH O, ZAGHLOUL M S, et al. Improved GNSS/IMU loosely coupled integration scheme using two kalman filter-based cascaded stages[J]. Arabian Journal for Science and Engineering, 2021, 46: 1345-1367.
14. PARRA I, SOTELO M A, LLORCA D F, et al. Robust visual odometry for vehicle localization in urban environments[J]. Robotica, 2010, 28(3): 441-452.
15. ENGEL J, KOLTUN V, CREMERS D. Direct sparse odometry[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(3): 611-625.
16. ROSE C, BRITT J, ALLEN J, et al. An integrated vehicle navigation system utilizing lane-detection and lateral position estimation systems in difficult environments for GNSS[J]. IEEE Transactions on Intelligent Transportation Systems, 2014, 15(6): 2615-2629.
17. ALLEN JR J W. Use of Vision Sensors and Lane Maps to Aid GNSS-INS Navigation[D]. Auburn: Auburn University, 2011.
18. TAO Z, BONNIFAIT P, FREMONT V, et al. Mapping and localization using GNSS, lane markings and proprioceptive sensors[C]//2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo, Japan, 2013: 406-412.
19. TAO Z, BONNIFAIT P, FREMONT V, et al. Lane marking aided vehicle localization[C]//16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013). Hague, Netherlands, 2013: 1509-1515.
20. TAO Z, BONNIFAIT P, FRÉMONT V, et al. Road-Centered Map-Aided Localization for Driverless Cars Using Single-Frequency GNSS Receivers[J]. Journal of Field Robotics, 2017, 34(5): 1010-1033.
21. THRUN S, MONTEMERLO M, DAHLKAMP H, et al. Stanley: The robot that won the DARPA Grand Challenge[J]. Journal of field Robotics, 2006, 23(9): 661- 692.
22. FUNK N, ALATUR N, DEUBER R, et al. Autonomous electric race car design[J]. arXiv preprint arXiv:1711.00548, 2017.
23. SCHWESINGER U, BÜRKI M, TIMPNER J, et al. Automated valet parking and charging for e-mobility[C]//2016 IEEE Intelligent Vehicles Symposium (IV). Chicago, USA, 2016: 157-164.
24. BRUNKER A, WOHLGEMUTH T, FREY M, et al. Odometry 2.0: A slipadaptive EIF-based four-wheel-odometry model for parking[J]. IEEE Transactions on Intelligent Vehicles, 2018, 4(1): 114-126.
25. FAZEKAS M, NÉMETH B, GÁSPÁR P. Iterative parameter identification method of a vehicle odometry model[J]. IFAC-PapersOnLine, 2019, 52(15): 199- 204.
26. FAZEKAS M, NÉMETH B, GÁSPÁR P, et al. Vehicle odometry model identification considering dynamic load transfers[C]//2020 28th Mediterranean Conference on Control and Automation (MED). Saint-Rapha, France, 2020: 19- 24.
27. FAZEKAS M, GÁSPÁR P, NÉMETH B. Velocity Estimation via Wheel Circumference Identification[J]. Periodica Polytechnica Transportation Engineering, 2021, 49(3): 250-260.
28. FAZEKAS M, GÁSPÁR P, NÉMETH B. Calibration and improvement of an odometry model with dynamic wheel and lateral dynamics integration[J]. Sensors, 2021, 21(2): 337.
29. WELTE A, XU P, BONNIFAIT P. Four-wheeled dead-reckoning model calibration using RTS smoothing[C]//2019 International Conference on Robotics and Automation (ICRA). Montreal, Canada, 2019: 312-318.
30. MALLESWARAN M, VAIDEHI V, MANJULA S, et al. Performance comparison of HONNs and FFNNs in GNSS and INS integration for vehicular navigation[C]//2011 International Conference on Recent Trends in Information Technology (ICRTIT). Kerala, India., 2011: 223-228.
31. MALLESWARAN M, VAIDEHI V, SARAVANASELVAN A, et al. Performance analysis of various artificial intelligent neural networks for GNSS/INS integration[J]. Applied Artificial Intelligence, 2013, 27(5): 367-407.
32. DAI H, BIAN H, WANG R, et al. An INS/GNSS integrated navigation in GNSS denied environment using recurrent neural network[J]. Defence technology, 2020, 16(2): 334-340.
33. FANG W, JIANG J, LU S, et al. A LSTM algorithm estimating pseudo measurements for aiding INS during GNSS signal outages[J]. Remote sensing, 2020, 12(2): 256.
34. ONYEKPE U, PALADE V, KANARACHOS S, et al. Learning uncertainties in wheel odometry for vehicular localisation in GNSS deprived environments[C]//2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA). Miami, USA, 2020: 741-746.
35. ONYEKPE U, PALADE V, HERATH A, et al. WhONet: Wheel Odometry neural Network for vehicular localisation in GNSS-deprived environments[J]. Engineering Applications of Artificial Intelligence, 2021, 105: 104421.