Cyrus Guo | WaveNav: PC Navigational Accessibility with LP Machine Learning

 
WaveNav is a ML powered wearable accessibility system that allows users to utilize customizable gestures to complement KBM (Keyboard and Mouse) navigation of PC navigational interfaces. It is powered by TinyML, which includes edge impulse model trainings and an Arduino Nano 33 BLE Sense to provide fast and accurate gesture classification, and AutoHotKey (AHK), a Windows scripting language. To be more specific, the Edge Impulse model is run through an arduino sketch with extra code that set certain outputs to respective simulated keypresses, which is then picked up as an input by AHK and AHK then performs the task associated with that key combination. For example, tilting your head to the left could trigger the browser’s “back” button, and the opposite for “forward”. Another example would be kicking your left foot out three times to open Google Chrome and a specific webpage. The small form factor allows users to wear the device on any dextrous parts of the body, and everything from the gestures themselves to the AHK outputs can be customized to individuals’ specific needs. While the system was inspired by the concept of easing computing for individuals with physical disabilities, anyone can take advantage of navigational gestures. Academic research has shown that while utilizing gestures for very specific inputs (similar to KBM) is not efficient or useful enough, gesture based navigational control, which has more tolerance for different movements, has shown great user feedback. Overall, this project is intended to be a proof-of-concept that TinyML along with open source scripting languages like AHK can be a powerful tool for accessibility, or even music or performance arts.