What is Visual-Inertial Odometry?
Visual-Inertial Odometry (VIO) is an estimation technology in which the position and orientation (pose) of a device is estimated using a combination of information received from a camera (visual) and Inertial Measurement Unit (IMU). This merging enthusiasts devices ranging from smartphones and drones to AR/VR headsets to enable them to interpret their own movement and position within a surrounding in which GPS is either unavailable or unreliable.
Unlike classical methods relying solely on a single sensor modality, VIO makes the best of complementarities in spatial information provided by visual and inertial sensors: cameras provide rich environment cues, but suffer motion blur and low texture; IMUs provide high frequency motion cues, but drift with time lapse.
Core Concepts of VIO
Visual Odometry
Uses camera images and tracks features in your scene and estimates motion based on movement of features from one frame to the next
Inertial Measurement
Uses accelerometers and gyroscopes to monitor motion (acceleration and rotation rates)
Sensor Fusion
Integrates the visual and inertial measurements in an accurate, robust pose estimation, by using algorithms like Kalman filters.
Localization
Senses position of device with respect to its environment, without any external infrastructure such as GPS
How VIO Operates: The Technical Process
Camera
Captures visual features
IMU
Measures motion data
VIO Output
Accurate pose estimation
Feature Detection
The algorithm detects distinctive features (corners, edges) in the images from the camera that will easily track across the frames.
IMU Data Processing
The IMU includes measurements of acceleration and gyro velocity, which are coupled to predict the motion between camera frames at high frequencies.
Data Synchronization
Different sensor frequencies and sensor time-stamps are corrected to time-synchronize visual and inertial data for correct fusion.
Sensor Fusion
Visual and inertial information are fused using state estimation algorithms. For advanced approaches, see our guide on deep sensor fusion such as Kalman filters or optimization-based approaches to provide estimates of a highly accurate pose with minimal drift.
Motion Estimation
The system uses the relative motion of features with the IMU-predicted motion to estimate the three dimensional position and orientation of the device.
Map Maintenance
Some VIO systems construct and update a local feature map for increased accuracy and relocalization in the case of lost track.
Applications of VIO
Drones & UAVs
Allows stable flying and navigation in GPS-denied conditions such as indoors, under forest canopies, etc.
Augmented Reality
Allows virtually entertaining objects to remain accurately positioned in the real world as users move their devices.
Robotics
Equips autonomous robots with the means to navigate and map previously unknown environments, in the absence of GPS.
Autonomous Vehicles
Serves as a backup localisation system when GPS signals are blocked in urban canyons or tunnels.
VIO vs Other Odometry Methods
| Method | Accuracy | Robustness | Hardware Requirements | Processing Needs |
|---|---|---|---|---|
| Visual Odometry (VO) | Medium | Low (fails with motion blur) | Camera only | Medium |
| Wheel Odometry | Low (wheel slip) | Medium | Wheel encoders | Low |
| IMU Only | Very Low (drifts quickly) | High | IMU only | Low |
| Visual-Inertial Odometry (VIO) | High | High | Camera + IMU | High |
The Future of VIO
Near Future (1-3 years)
Better power consumption, low-light performance and wider consumer AR adoption through smartphones implementations.
Mid Future (3-5 years)
Tight integration with semantic understanding via machine learning, better dynamics handling, multi-device collaboration.
Long Term (5+ years)
Widely used by all mobile devices, always on AR worlds, enhanced robotics applications, new forms of human-computer interaction, etc.
Research Frontiers
Neuromorphic vision sensors, event-based cameras, deep learning integration and collaborative vision-inertial (VIO) across device networks.
Conclusion
Visual-Inertial Odometry is an efficient solution to the important issue of localization in non-GPS-based or unstable environments. VIO brings together the complementary capabilities of the visual and inertial sensors to offer robust and precise pose estimation capable of supporting drone navigation to immersive augmented reality experiences.
With sensor technology becoming more advanced and algorithms becoming more efficient, we may well find VIO used in more and more applications and eventually assisting machines in understanding and navigating our world in a much better way.