Based on IMU Real time Motion Capture Technology

1. Introduction to Motion Capture Technology

Motion capture technology is a method for tracking and recording the movement process of the human body. Motion capture technology generally refers to the process where sensors and signal capture devices track the movements of specific parts of the human body, and then the motion data is transmitted via a transmission device to the host computer to drive the model, enabling it to naturally reproduce the movements.

2. Applications of Motion Capture Technology

Human motion capture technology is widely applied in fields such as human-computer interaction, rehabilitation medicine, and sports fitness. In the field of human-computer interaction, taking interactive games as an example, the motion capture system tracks and records the physical data of human movement, which is used to generate 2D or 3D models and restore human actions in a virtual environment, making the movements of the virtual models more natural and flexible, and enhancing the user experience.

In the field of rehabilitation medicine, this technology can be applied in medical processes such as epilepsy, rehabilitation examinations, and rehabilitation therapy, allowing for a more accurate understanding of the patient’s physical activity ability and optimizing the treatment plan. In the field of sports and fitness, it enables coaches to more accurately evaluate the athletes’ abilities and formulate more appropriate training plans. In daily life, virtual reality can also be combined with sports and fitness, and the system can intelligently formulate fitness plans based on the analysis of human movement, adding fun while promoting exercise.

3. MEMS IMU Characteristics

The MEMS (Inertial Measurement Unit, IMU) captures the motion by obtaining three-dimensional spatial information to determine the attitude angles. It is easy to wear, has low cost，has a high degree of freedom, good real-time performance, is relatively less susceptible to environmental interference, but the sensors are prone to drift and other problems. However it can also be mitigated or even eliminated through subsequent mathematical methods.

4. Technological Development in Motion Capture

Based on the research of inertial sensors, the research on the application of inertial sensing in motion capture systems has begun to develop in the following directions:

• By leveraging the complementary characteristics of IMU and optical sensors such as cameras, they can be combined for self-motion tracking ^{1 2 3}.
• Since the motion tracking system based on IMU, when using the magnetometer for heading estimation, the magnetometer signal is inevitably subject to magnetic interference, which seriously reduces the accuracy of heading estimation. Some methods have been proposed to eliminate the estimated heading angle offset^{4 5}.
• The cost of motion tracking systems based on inertial sensors in the market is still relatively high, typically reaching USD1,400. Therefore, research on low-cost IMU motion tracking has also become a major direction ^{6 7}.
• To meet the current increasingly stringent requirements for the restoration of human body postures, systems and research methods specifically targeting certain parts of the human body have been proposed ^{8 9 10}.

5. Application case

In the application of MEMS IMU human pose tracking, the currently well-known brand is the MVN system developed by Xsens.

Xsens Link systems

The Xsens Link is designed for ultimate freedom of movement, accurately capturing even the most dynamic actions. With a large wireless range, magnetic immunity, and On-Body Recording, you can capture without boundaries or limitations – powering your animation, research, and humanoid robot training¹¹.

The Xsens Link components includes, Battery life: 4 hours internal + 8 hours external hot-swappable battery Casing: Rugged pelican case Charging: Charging station for battery pack Clothing: Full body eSuit Finger tracking compatibility: Stretchsense, Xsens Metagloves by Manus Hardware synchronization: Optional, with aditional Awinda station Latency: 20 ms On-body buffering: 10 minutes Sensors: 17 click-in sensors (plus 1 prop sensor) Update rate: 240 Hz Wireless communication: Wi-Fi Wireless datalink: Access point (multiple persons) Wireless range: 150 m.

6. IMU Algorithm Overview

The core components of the IMU include single, double and triple axis combinations of the accelerometer, gyroscope and magnetometer, which measure the wearer’s motion posture data through inertial force. The composition of IMU can be roughly divided into three categories: IMU based on acceleration and magnetometers, IMU based on gyroscopes, and IMU based on multi-sensor fusion.

IMU based on accelerometers and magnetometers can generally accurately calculate attitudes only in static conditions. This is because both accelerometers and magnetometers are mainly used to measure orientation sensors and are very susceptible to nonlinear acceleration and magnetic fields in dynamic conditions. Therefore, they are often used in gyroscopes rather than independently in motion capture systems.

7. Multi-Sensor Fusion and Attitude Calculation

The IMU based on gyroscope calculates real-time attitude changes through angular velocity integration. During the measurement process, it is not easily disturbed by other external environmental factors, but angular velocity integration is accompanied by error accumulation. Although the dynamic performance was good in the initial stage of measurement, the error gradually increased over time, causing the attitude solution to diverge accordingly.

Combining the above two types of IMU, based on their complementary advantages, the multi-sensor fusion IMU, that is, the IMU that outputs nine-axis sensing information, is widely used in current inertial motion capture systems. In a static state, the attitude orientation is output by an accelerometer and a magnetometer. At this time, the gyroscope ideally cannot measure the change in angular velocity. When in motion, the gyroscope accurately and quickly responds to changes in angular velocity. Although errors accumulate over time, the orientation can be corrected to a certain extent through the accelerometer and magnetometer. Ultimately, both good static measurement values and relatively real-time and accurate dynamic change values were obtained.

How to obtain high-precision dynamic and static attitudes by using the IMU of multi-sensor fusion has become a key issue to be considered in the process of attitude calculation. Motion capture technology has developed from the last century to the present. The relatively mature and widely applied multi-sensor compensation fusion algorithms can be roughly divided into two categories: kalman filtering and complementary filtering.

8. Kalman Filtering in Motion Capture

Kalman filtering is most widely applied in systems that require yaw calculation. Many commercial IMU and systems use attitude solution optimization methods, such as Xsens, micro-strain, Intersense, Crossbow and other commercial devices, all of which employ Kalman filters. It uses dynamic state equations to represent estimated and measured values, and updates the estimated measured values at the present moment through the covariance matrix of the final result at the previous moment and the filtering error, thereby estimating the optimal solution. However, since the Kalman filter cannot be applied to nonlinear systems, the extended Kalman Filter (EKF) was proposed to linearize nonlinear systems. In the current attitude solution methods, the extended Kalman filter can meet the requirement of high precision ¹², but it is computationally complex and requires a large number of recursive iterations. Although the use of quaternions can relatively reduce the computational scale ¹³, it still cannot meet the requirement of high real-time performance. In response to this, some people have proposed a method of implementing EKF circuits through hardware design to enhance efficiency ¹⁴.

9. Complementary Filtering Algorithm

The logic of the complementary filtering algorithm is concise and clear, with high execution efficiency. It has relatively low requirements for the accuracy of the sensor and the computing power of the processor, and is widely used in low-cost IMUs. Complementary filtering algorithm improves the accuracy of the solution by fusing and processing the data of inertial sensors with high-frequency and low-frequency signals, eliminating noise compensation errors. Mahony ¹⁵ proposed a separate algorithm using complementary filtering processing. By observing the vector changes of the accelerometer and magnetometer, the random drift generated by the gyroscope during the integration operation was calibrated. Then, the gyroscope was input as the final measurement value into the attitude matrix to achieve real-time attitude calculation. This algorithm achieves effective filtering correction performance with relatively small computational overhead. Subsequently, Madgwick ¹⁶ proposed using the gradient descent algorithm to calculate the motion posture, which uses quaternions for representation and supports efficient computation. The gradient descent algorithm is derived through the analysis of the raw data from the accelerometer and magnetometer. The gyroscope error is estimated by differentiating quaternions.

1. Malleson C , Gilbert A , Trumble M , et al. Real-Time Full-Body Motion Capture from Video and IMUs[C]// 2017 International Conference on 3D Vision (3DV). IEEE Computer Society, 2017.
2. Akbari A , Thomas X , Jafari R . Automatic noise estimation and context-enhanced data fusion of IMU and Kinect for human motion measurement[C]// 2017 IEEE 14th International Conference on Wearable and Implantable Body Sensor Networks (BSN). IEEE, 2017.
3. Ziegler J , Kretzschmar H , Stachniss C , et al. Accurate human motion capture in large areas by combining IMU- and laser-based people tracking[C]// 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2011.
4. Jung Keun Lee, Tae Hyeong Jeon。IMU-Based but Magnetometer-Free Joint Angle Estimation of Constrained Links[C] //2018 IEEE SENSORS。 28-31 Oct. 2018.
5. P. Raghavendra，M. Sachin，P. S. Srinivas, at el. A Novel Biomechanical Model-Aided IMU/UWB Fusion for Magnetometer-Free Lower Body Motion Capture[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2016, 47(6):1-12.
6. Raghavendra P , Sachin M , Srinivas P S , et al. Design and Development of a Real-Time, Low-Cost IMU Based Human Motion Capture System[M]// Computing and Network Sustainability. 2017.
7. Lee D , Lee S , Park S , et al. Test and error parameter estimation for MEMS — based low cost IMU calibration[J]. International Journal of Precision Engineering & Manufacturing, 2011, 12(4):597-603.
8. Jae Kim K , Agrawal V , Bennett C , et al. Measurement of Lower Limb Segmental Excursion using Inertial Sensors during Single Limb Stance[J]. Journal of Biomechanics, 2018:S0021929018300794.
9. Seel T , Raisch J , Schauer T . IMU-Based Joint Angle Measurement for Gait Analysis[J]. Sensors, 2014, 14(4):6891-6909.
10. Hamdi M M , Awad M I , Abdelhameed M M , et al. Lower limb motion tracking using IMU sensor network[C]// Biomedical Engineering Conference (CIBEC). IEEE, 2014.
11. Next-Generation Xsens Link, shoppage
12. Ligorio G , Sabatini A . Extended Kalman Filter-Based Methods for Pose Estimation Using Visual, Inertial and Magnetic Sensors: Comparative Analysis and Performance Evaluation[J]. Sensors, 2013, 13(2):1919-1941.
13. Abbate N , Basile A , Brigante C , et al. Development of a MEMS based wearable motion capture system[C]// Human System Interactions, 2009. HSI ’09. 2nd Conference on. IEEE, 2009.
14. Mie S , Okuyama Y , Sato Y , et al. Real-Time UAV Attitude Heading Reference System Using Extended Kalman Filter for Programmable SoC[C]// 2017 IEEE 11th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC). IEEE, 2017.
15. Mahony R , Hamel T , Pflimlin J M . Nonlinear Complementary Filters on the Special Orthogonal Group[J]. IEEE Transactions on Automatic Control, 2008, 53(5):1203-1218.
16. Madgwick S O H , Harrison A J L , Vaidyanathan A . Estimation of IMU and MARG orientation using a gradient descent algorithm[J]. IEEE International Conference on Rehabilitation Robotics : [proceedings], 2011, 2011:5975346.