Intelligent AGVs Based on IMU

1. State Estimation Technology for Intelligent AGVs

Intelligent Automated Guided Vehicles (AGVs) have become a crucial equipment that plays a significant role in enhancing logistics efficiency and reducing production costs. State estimation technology is a prerequisite for the autonomous operation of intelligent AGVs and is the core guarantee for ensuring the safe operation of intelligent AGVs. Through analyzing sensor data, state estimation technology can accurately infer the position, speed, and pose of the intelligent AGV system in three-dimensional space. With this information, one can know the operating status of the vehicle.

2. Current AGV Navigation and the Role of IMU

Currently, AGVs can be classified into:

• Magnetic guidance AGVs
• Free-path AGVs

The paths of magnetic guidance AGVs are simple to lay out, but they have poor flexibility. Therefore, free-path AGVs have a broader market application demand. The Inertial Measurement Unit (IMU) serves as the on-board sensor. It can usually improve the state estimation accuracy of intelligent AGVs by being assisted by sensors such as GPS, vision, and LiDAR. The main methods include GPS/IMU ^{1 2}, vision/IMU, and LiDAR/vision/IMU ^{3 4}, etc.

2.1 Challenges in Dynamic Environments

When the intelligent AGV encounters rapid changes in scenarios such as personnel movement and goods placement within the factory premises, resulting in image inaccuracies and image matching failures for sensors like LiDAR and vision, this can affect the AGV’s state estimation capabilities ^{5 6 7}. When the sensors, such as vision, introduce greater errors due to environmental influences, obtaining accurate state estimation information becomes extremely difficult. Therefore, improving the state estimation technology of the IMU is of great significance for the intelligent driving of AGVs.

2.2 Advantages of IMU for State Estimation

IMU is usually small in size and light in weight, making it easy to be integrated into intelligent AGVs. IMU can output data at a high frequency, providing more detailed and accurate motion information for achieving high-precision state estimation ^{8 9 10}. The IMU coordinate system is usually the same as the carrier coordinate system, which means that obtaining the IMU state values can also obtain the state quantities of the intelligent AGV. However, during the operation of the intelligent AGV, it often exhibits the phenomenon of “moving – stopping”, causing the intelligent AGV to be disturbed by errors in both static and dynamic states. Therefore, processing the static and dynamic data of IMU, implementing the state estimation method of the intelligent AGV in different states based on IMU, and forming the static/dynamic full-process state estimation technology based on IMU through the combination of static and dynamic methods, can enhance the autonomous navigation ability of the intelligent AGV and improve the level of logistics automation ^{11 12 13}.

State estimation is a core technology of intelligent AGVs, playing a crucial role in the operational stability, reliability, and pose accuracy of AGVs. Through obtaining information about the internal state of the system, state estimation enables the understanding and optimization of system behavior. In fields such as engineering control, navigation systems, and target tracking, state estimation is one of the key technologies for achieving automation, intelligence, and efficiency ^{14 15}.

3. Single Sensor Navigation Technologies

Currently, the navigation of AGVs based on a single sensor usually includes magnetic guidance, vision, LiDAR and inertial measurement units, etc., which are the mainstream navigation forms for current AGVs.

3.1 Magnetic Guidance AGVs

Magnetic guidance AGV technology has been widely applied. Magnetic guidance AGVs are equipped with magnetic markers or coils on the ground, and magnetic sensing devices are installed on the AGV vehicles. They achieve AGV state estimation by detecting the ground magnetic field. The magnetic guidance technology has low production costs, is easy to implement, but is not very flexible ^{16 17}.

3.2 Vision-Based AGVs

Visual-based AGV navigation technology is gradually becoming a research hotspot in the field of industrial automation, with its application scope covering areas such as warehouse logistics and intelligent manufacturing. The visual-based intelligent AGV state estimation technology uses visual information for navigation and determines the position of the AGV relative to the environmental map. However, its robustness in different environments is insufficient, the recognition algorithm is complex and significantly affected by lighting conditions. Excessive exposure may lead to the loss of path information, thereby causing navigation failure.

Key research and developments in visual navigation include:

• In response to the problems such as uneven lighting, damaged guiding tape, and obstacle obstruction encountered during AGV actual operation, Yang Lei ¹⁸ proposed a processing scheme for guiding images, extracting the guiding trajectory from the guiding images.
• Mur-Artal ¹⁹ proposed the ORB-SLAM2 method in 2021, which is a complete simultaneous localization and mapping system suitable for monocular, stereo, and RGB-D cameras, including map reuse, loop closing, and repositioning functions.

However, visual technology has issues such as low data update frequency, complex factory environment, and excessive pedestrians, resulting in certain limitations of visual-guided AGVs.

3.3 LiDAR-Based AGVs

LiDAR can provide highly accurate distance information, which is used for environment mapping and obstacle detection. Due to the height limitation of the installation, the field of view of the LiDAR has blind areas. When detecting low objects, it may collide with them.

Research approaches to address LiDAR limitations:

• Xiao Zhicheng ²⁰ proposed a collision avoidance method combining ultrasonic sensors and lidar sensors. Through coordinated control between the top-level and bottom-level programs, it is applied in the local collision avoidance of AGVs, improving the safety of the AGV system during operation.
• Chen Zeyi ²¹ aimed to solve the problem of drift that traditional state estimation methods are prone to, and proposed using a three-dimensional LiDAR as the main sensor to construct a three-dimensional point cloud map to achieve the self-state estimation of intelligent vehicles.

However, in high-speed movement or vibration environments, the LiDAR may cause motion blur, resulting in inaccurate distance measurement.

3.4 Inertial Navigation AGVs (IMU)

The inertial AGV utilizes inertial sensors such as gyroscopes and accelerometers to calculate its position and orientation by measuring the acceleration and angular velocity of the AGV.

Advantages of IMU:

• The IMU can provide high-frequency sensing data, typically reaching hundreds of hertz or even higher, which is crucial for real-time motion tracking and attitude estimation.
• Compared to high-resolution cameras and LiDARs, the IMU is smaller in size, lighter in weight, and thus easier to integrate into small devices.
• The IMU is less susceptible to dust or other harsh environmental conditions, and its cost is usually lower, making it an economically viable choice in many applications.

Disadvantage of IMU:

• Although inertial navigation has the advantages of high accuracy in a short period and low cost, errors accumulate over time, resulting in gradually decreasing navigation accuracy.

3.5 IMU’s Advantage in Indoor Factory Environments

In the indoor environment of a factory, LiDAR and vision systems may be affected by environmental light, structural obstacles, personnel movement, and goods placement, resulting in inaccurate image matching. However, IMU can operate independently of the external environment. Its output data is only related to itself. Compared with LiDAR and vision systems, IMU has lower cost and a simpler structure, making it easier to integrate and maintain. For some application scenarios with restrictions on cost and equipment complexity, choosing IMU as the navigation sensor is a more economical and practical choice.

4. Multi-Sensor Fusion Mode

The intelligent AGV state estimation method based on multi-sensor fusion ¹⁸ has the following common integrated navigation models, such as GNSS/IMU model ¹⁹, visual/IMU model ²⁰, LiDAR/visual/IMU model ^{21 22}, etc., which are the mainstream multi-sensor fusion methods.

4.1 Research in Multi-Sensor Fusion

Key research studies in this area include:

• Wang Xin ²⁶ proposed a real-time state estimation method with multi-view visual-inertial tight coupling. To address the issues of error accumulation and inertial bias drift in the tracking estimation part, a QR code pose correction model was proposed, and periodic compensation was carried out for some key frames.
• Yuan Chuanqian ²⁷addressed the problems of low state estimation accuracy, sensor signal loss, and large navigation trajectory error of AGV in the logistics transportation process. He proposed a fusion positioning framework based on wheel odometer, inertial measurement unit, ultra-wideband positioning system, and LiDAR. The sensor mathematical model was introduced into the extended Kalman filter algorithm, thereby designing an indoor AGV autonomous positioning and navigation algorithm.
• Zhang, M ²⁸ demonstrated that single IMU state estimation can achieve acceptable accuracy and robustness in various use cases. Utilizing multiple IMUs can further enhance the overall performance. Therefore, a lightweight and precise algorithm was proposed to fuse measurements from multiple IMUs and external sensing sensors. This algorithm can achieve significant performance improvement without incurring computational costs.
• For low-light environments without features, Filip ²⁹ proposed an improved SLAM algorithm. This algorithm incorporates the wheel encoder data of the AGV, uses LiDAR SLAM as the baseline algorithm, and integrates the wheel encoder sensor data into the baseline SLAM structure using the extended Kalman filter algorithm. In this algorithm, the prediction steps of the extended Kalman algorithm use wheel encoder and inertial measurement unit data, while LiDAR data is used for the correction steps.

5. Conclusion: Challenges and Core Significance

Although the AGV state estimation technology based on multi-sensor fusion can enhance the performance and robustness of the system, it also faces a series of challenges such as:

• Differences in sampling frequency, accuracy and delay among different sensors
• Data redundancy and complexity
• Increased cost

In the multi-sensor fusion-based AGV state estimation technology, the IMU sensor is the core of the fusion system. Whether in the application of a single sensor or in the application of multi-sensor fusion, improving the accuracy of IMU state estimation has significant research significance.

1. Ullah I, Zaidi B H, Nisar S,et al. An inertial and global positioning system based algorithm for ownship navigation [J]. International Journal of Sensor Networks, 2021, 37(4): 209-18.
2. Zhang J, Wang J, Cui X,et al. BDS/GPS/UWB Adaptively Robust EKF Tightly Coupled Navigation Model Considering Pedestrian Motion Characteristics [J]. Remote Sensing, 2022, 14(8): 1-17.
3. Sharifisoraki Z, Dey A, Selzler R,et al. Monitoring Critical Infrastructure Using 3D LiDAR Point Clouds [J]. Ieee Access, 2023, 11: 314-36.
4. Swearingen T C C, Klaver R W W, Anderson Jr C R R,et al. Influence of camera model and alignment on the performance of paired camera stations [J]. Wildlife Society Bulletin, 2023, 47(2): 1-11.
5. 罗欣. 智能 AGV 系统设计与关键技术研究 [D], 华南理工大学, 2017.
6. 宋永杰. 激光与视觉相结合的智能 AGV SLAM 算法应用研究 [D], 西南交通大 学, 2020.
7. 万好. 基于视觉引导的 AGV 车载系统研究 [D], 南昌航空大学, 2014.
8. Gong Z, Pei L, Zou D,et al. Graphical Approach for MAV Sensors Fusion [C]. proceedings of the 29th International Technical Meeting of The-Satellite-Divisionof-the-Institute-of-Navigation (ION GNSS+), 2016.
9. Berkane S, Tayebi A, de Marco S. A nonlinear navigation observer using IMU and generic position information [J]. Automatica, 2021, 127.
10. Shaukat S A, Munawar K, Arif M,et al. Robust Vehicle Localization with GPS Dropouts [C]. proceedings of the 6th International Conference on Intelligent and Advanced Systems (ICIAS), 2016.
11. Fang T H, Park S H, Seo K,et al. Attitude Determination Algorithm using State Estimation Including Lever Arms between Center of Gravity and IMU [J]. International Journal of Control Automation and Systems, 2016, 14(6): 1511-9.
12. Patel U N, Faruque I A. Multi-IMU Based Alternate Navigation Frameworks: Performance & Comparison for UAS [J]. Ieee Access, 2022, 10: 17565-77.
13. Rodrigo Marco V, Kalkkuhl J, Raisch J,et al. A Novel IMU Extrinsic Calibration Method for Mass Production Land Vehicles [J]. Sensors, 2021, 21(1): 1-30.
14. Chang D, Johnson-Roberson M, Sun J. An Active Perception Framework for Autonomous Underwater Vehicle Navigation Under Sensor Constraints [J]. Ieee Transactions on Control Systems Technology, 2022, 30(6): 2301-16.
15. Dahal P, Mentasti S, Paparusso L,et al. RobustStateNet: Robust ego vehicle state estimation for Autonomous Driving [J]. Robotics and Autonomous Systems, 2024, 172.
16. 李林慧. 智能 AGV 多目视觉导引与惯性测量的复合导航技术研究 [D], 南京航 空航天大学, 2019.
17. 刘朋飞. 智能 AGV 视觉导航控制系统研究及应用 [D], 华南理工大学, 2013.
18. Ye X Y, Song F J, Zhang Z Y,et al. A Review of Small UAV Navigation System Based on Multisource Sensor Fusion [J]. Ieee Sensors Journal, 2023, 23(17): 18926- 48.
19. 叶超. 多信息融合的自动驾驶定位与环境感知系统关键技术研究 [D], 哈尔滨 工业大学, 2022.
20. 岑汝平. 基于视觉与惯导融合的移动机器人定位研究 [D], 重庆大学, 2022.
21. Lu B X, Tsai Y C, Tseng K S. GRVINS: Tightly Coupled GNSS-Range-VisualInertial System [J]. Journal of Intelligent & Robotic Systems, 2024, 110(1): 1-17.
22. Yang C, Shi W Z, Chen W. Robust M-M unscented Kalman filtering for GPS/IMU navigation [J]. Journal of Geodesy, 2019, 93(8): 1093-104.