TY - GEN
T1 - Leveraging earables for unvoiced command recognition
AU - Srivastava, Tanmay
AU - Khanna, Prerna
AU - Pan, Shijia
AU - Nguyen, Phuc
AU - Jain, Shubham
N1 - Publisher Copyright:
© 2022 Owner/Author.
PY - 2022/6/27
Y1 - 2022/6/27
N2 - We demonstrate an ear-worn technology that recognizes unvoiced human commands by tracking jaw motion. The ear-worn system is designed to achieve continual unvoiced command recognition for robust human-computer interaction (HCI) applications. First, the system reliably extracts the jaw motion signals buried under the noise caused by head motion, walking, and other motion artifacts to track single secondary voice articulator (i.e., word). Then, learning from linguistics and human speech anatomy, we design a novel algorithm that localizes the phonemes in the command, and reconstructs the word. We evaluate the proposed system in real-world experiments with 15 volunteers. Our preliminary results show that the proposed system obtains a word recognition accuracy of 95.6% in noise-free conditions and 93.2% and 91.6%, while head nodding and walking.
AB - We demonstrate an ear-worn technology that recognizes unvoiced human commands by tracking jaw motion. The ear-worn system is designed to achieve continual unvoiced command recognition for robust human-computer interaction (HCI) applications. First, the system reliably extracts the jaw motion signals buried under the noise caused by head motion, walking, and other motion artifacts to track single secondary voice articulator (i.e., word). Then, learning from linguistics and human speech anatomy, we design a novel algorithm that localizes the phonemes in the command, and reconstructs the word. We evaluate the proposed system in real-world experiments with 15 volunteers. Our preliminary results show that the proposed system obtains a word recognition accuracy of 95.6% in noise-free conditions and 93.2% and 91.6%, while head nodding and walking.
KW - earable
KW - IMU sensing
KW - unvoiced speech recognition
KW - wearable devices
UR - https://www.scopus.com/pages/publications/85134058686
U2 - 10.1145/3498361.3538665
DO - 10.1145/3498361.3538665
M3 - Conference contribution
AN - SCOPUS:85134058686
T3 - MobiSys 2022 - Proceedings of the 2022 20th Annual International Conference on Mobile Systems, Applications and Services
SP - 622
EP - 623
BT - MobiSys 2022 - Proceedings of the 2022 20th Annual International Conference on Mobile Systems, Applications and Services
PB - Association for Computing Machinery, Inc
T2 - 20th ACM International Conference on Mobile Systems, Applications and Services, MobiSys 2022
Y2 - 27 June 2022 through 1 July 2022
ER -