TY - GEN
T1 - Push the Limit of Acoustic Gesture Recognition
AU - Wang, Yanwen
AU - Shen, Jiaxing
AU - Zheng, Yuanqing
N1 - Funding Information:
Vision based gesture tracking are well-studied [20, 28, 31]. Microsoft HoloLens [20] uses specialized cameras to provide contact-free human gesture tracking. Sony PlayStation VR [31] require users to wear helmets and controllers, which are cumbersome compared to contact-free systems. DigitEyes [28] can model hand movement from ordinary gray-scale images. However, vision based methods require good light conditions, which limits their applications. VI. CONCLUSION This paper presents a holistic design and implementation of an acoustic based gesture recognition system that can identify 15 types of gestures with high robustness and accuracy. In order to alleviate frequency selective fading, this paper adopts frequency hopping and carefully designs down-conversion and demodulation to avoid inter-subframe interference. Based on the insights obtained in the initial experiments, this paper conducts data augmentation on raw CIR data to synthesize new augmented data, which is used to effectively train neural network models. In particular, the augmented data captures different variations in practical scenarios such as different gesture speeds, distances to transceiver, and signal attenuation. The experiment results show that RobuCIR substantially outperforms state-of-the-art work and achieves an overall accuracy of 98.4% under different usage scenarios. ACKNOWLEDGEMENT This work is supported in part by the National Nature Science Foundation of China under grant 61702437 and Hong Kong GRF under grant PolyU 152165/19E. Yuanqing Zheng is the corresponding author.
Publisher Copyright:
© 2020 IEEE.
PY - 2020/7
Y1 - 2020/7
N2 - With the flourish of the smart devices and their applications, controlling devices using gestures has attracted increasing attention for ubiquitous sensing and interaction. Recent works use acoustic signals to track hand movement and recognize gestures. However, they suffer from low robustness due to frequency selective fading, interference and insufficient training data. In this work, we propose RobuCIR, a robust contact-free gesture recognition system that can work under different usage scenarios with high accuracy and robustness. RobuCIR adopts frequency-hopping mechanism to mitigate frequency selective fading and avoid signal interference. To further increase system robustness, we investigate a series of data augmentation techniques based on a small volume of collected data to emulate different usage scenarios. The augmented data is used to effectively train neural network models and cope with various influential factors (e.g., gesture speed, distance to transceiver, etc.). Our experiment results show that RobuCIR can recognize 15 gestures and outperform state-of-the-art works in terms of accuracy and robustness.
AB - With the flourish of the smart devices and their applications, controlling devices using gestures has attracted increasing attention for ubiquitous sensing and interaction. Recent works use acoustic signals to track hand movement and recognize gestures. However, they suffer from low robustness due to frequency selective fading, interference and insufficient training data. In this work, we propose RobuCIR, a robust contact-free gesture recognition system that can work under different usage scenarios with high accuracy and robustness. RobuCIR adopts frequency-hopping mechanism to mitigate frequency selective fading and avoid signal interference. To further increase system robustness, we investigate a series of data augmentation techniques based on a small volume of collected data to emulate different usage scenarios. The augmented data is used to effectively train neural network models and cope with various influential factors (e.g., gesture speed, distance to transceiver, etc.). Our experiment results show that RobuCIR can recognize 15 gestures and outperform state-of-the-art works in terms of accuracy and robustness.
UR - http://www.scopus.com/inward/record.url?scp=85090269422&partnerID=8YFLogxK
U2 - 10.1109/INFOCOM41043.2020.9155402
DO - 10.1109/INFOCOM41043.2020.9155402
M3 - Conference article published in proceeding or book
AN - SCOPUS:85090269422
T3 - Proceedings - IEEE INFOCOM
SP - 566
EP - 575
BT - INFOCOM 2020 - IEEE Conference on Computer Communications
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 38th IEEE Conference on Computer Communications, INFOCOM 2020
Y2 - 6 July 2020 through 9 July 2020
ER -