TY - GEN
T1 - Vi-Fi
T2 - 21st ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2022
AU - Liu, Hansi
AU - Alali, Abrar
AU - Ibrahim, Mohamed
AU - Cao, Bryan Bo
AU - Meegan, Nicholas
AU - Li, Hongyu
AU - Gruteser, Marco
AU - Jain, Shubham
AU - Dana, Kristin
AU - Ashok, Ashwin
AU - Cheng, Bin
AU - Lu, Hongsheng
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - In this paper, we present Vi-Fi, a multi-modal system that leverages a user's smartphone WiFi Fine Timing Measurements (FTM) and inertial measurement unit (IMU) sensor data to associate the user detected on a camera footage with their corresponding smartphone identifier (e.g. WiFi MAC address). Our approach uses a recurrent multi-modal deep neural network that exploits FTM and IMU measurements along with distance between user and camera (depth information) to learn affinity matrices. As a baseline method for comparison, we also present a traditional non deep learning approach that uses bipartite graph matching. To facilitate evaluation, we collected a multi-modal dataset that comprises camera videos with depth information (RGB-D), WiFi FTM and IMU measurements for multiple participants at diverse real-world settings. Using association accuracy as the key metric for evaluating the fidelity of Vi-Fi in associating human users on camera feed with their phone IDs, we show that Vi-Fi achieves between 81% (real-time) to 91% (offline) association accuracy.
AB - In this paper, we present Vi-Fi, a multi-modal system that leverages a user's smartphone WiFi Fine Timing Measurements (FTM) and inertial measurement unit (IMU) sensor data to associate the user detected on a camera footage with their corresponding smartphone identifier (e.g. WiFi MAC address). Our approach uses a recurrent multi-modal deep neural network that exploits FTM and IMU measurements along with distance between user and camera (depth information) to learn affinity matrices. As a baseline method for comparison, we also present a traditional non deep learning approach that uses bipartite graph matching. To facilitate evaluation, we collected a multi-modal dataset that comprises camera videos with depth information (RGB-D), WiFi FTM and IMU measurements for multiple participants at diverse real-world settings. Using association accuracy as the key metric for evaluating the fidelity of Vi-Fi in associating human users on camera feed with their phone IDs, we show that Vi-Fi achieves between 81% (real-time) to 91% (offline) association accuracy.
UR - https://www.scopus.com/pages/publications/85135959158
U2 - 10.1109/IPSN54338.2022.00024
DO - 10.1109/IPSN54338.2022.00024
M3 - Conference contribution
AN - SCOPUS:85135959158
T3 - Proceedings - 21st ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2022
SP - 208
EP - 219
BT - Proceedings - 21st ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2022
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 4 May 2022 through 6 May 2022
ER -