Combining passive visual cameras and active IMU sensors for persistent pedestrian tracking

Vision based pedestrian tracking becomes a hard problem when long-term/heavy occlusion happens or pedestrian temporarily moves out of the visual field. In this paper, a novel persistent pedestrian tracking system is presented which combines visual signal from surveillance cameras and sensor signals from Inertial Measurement Unit (IMU) carried by pedestrians themselves. IMU tracking performs Dead Reckoning (DR) approach utilizing accelerometer, gyroscope and magnetometer. IMU tracking has nothing to do with visual occlusion, so it keeps working even when pedestrians are visually occluded. Meanwhile, visual tracking assists in calibrating IMU to avoid the bias drift during DR. The experimental results show that the IMU and visual tracking are complementary to each other and their combination performs robust pedestrian tracking in many challenging scenarios.