In this paper, we propose ThuMouse, a novel interaction paradigm aimed to create a gesture-based and touch-free cursor interaction that accurately tracks the motion of fingers in real-time. ThuMouse enables users to move the cursor using frequency-modulated continuous-wave (FMCW) radar. While previous work with FMCW radar in human-computer-interfaces (HCI) has focused on classifying a set of predefined hand gestures, ThuMouse regressively tracks the position of a finger, which allows for finer-grained interaction. This paper presents the gesture sensing pipeline we built, with regressive tracking through deep neural networks, data augmentation for robustness, and computer vision as a training base. We also report on a proofof-concept demonstration shows how our system can function as a mouse, and identify areas for future work. This work builds a foundation for designing finer micro gesture-based interactions, allowing the finger to emulate external input devices such as a joystick and touch-pad.
Li, Z., Lei, Z., Yan, A., Solovey, E., & Pahlavan, K. (2019). ThuMouse: a micro-gesture cursor input through mmWave radar-based interaction. Semantic Scholar. https://pdfs.semanticscholar.org/524d/d51d09eb6d15fdca0abd7e1e5f419969d714.pdf
*denotes a WPI undergraduate student author