Enabling Fine-grained Finger Gesture Recognition on Commodity WiFi Devices

Sheng Tan, Jie Yang, Yingying Chen

Research output: Contribution to journalArticlepeer-review

Abstract

Gesture recognition has become increasingly important in human-computer interaction and can support different applications such as smart home, VR, and gaming. Traditional approaches usually rely on dedicated sensors that are worn by the user or cameras that require line of sight. In this paper, we present fine-grained finger gesture recognition by using commodity WiFi without requiring user to wear any sensors. Our system takes advantages of the fine-grained Channel State Information available from commodity WiFi devices and the prevalence of WiFi network infrastructures. It senses and identifies subtle movements of finger gestures by examining the unique patterns exhibited in the detailed CSI. We devise environmental noise removal mechanism to mitigate the effect of signal dynamic due to the environment changes. Moreover, we propose to capture the intrinsic gesture behavior to deal with individual diversity and gesture inconsistency. Lastly, we utilize multiple WiFi links and larger bandwidth at 5GHz to achieve finger gesture recognition under multi-user scenario. Our experimental evaluation in different environments demonstrates that our system can achieve over 90% recognition accuracy and is robust to both environment changes and individual diversity. Results also show that our system can provide accurate gesture recognition under different scenarios.

Original languageEnglish (US)
JournalIEEE Transactions on Mobile Computing
DOIs
StateAccepted/In press - 2020

All Science Journal Classification (ASJC) codes

  • Software
  • Electrical and Electronic Engineering
  • Computer Networks and Communications

Keywords

  • Bandwidth
  • Cameras
  • Gesture recognition
  • Mobile computing
  • Performance evaluation
  • RF signals
  • WiFi
  • Wireless fidelity
  • channel state information
  • finger gesture

Fingerprint

Dive into the research topics of 'Enabling Fine-grained Finger Gesture Recognition on Commodity WiFi Devices'. Together they form a unique fingerprint.

Cite this