An Accessible Digital Musical Instruments operated through gaze pointing and head rotation
<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/1b5d0097-8357-4aa0-a47a-723e23488434/github_white.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/1b5d0097-8357-4aa0-a47a-723e23488434/github_white.png" width="40px" /> Github repository
</aside>
Netychords is a software Accessible Digital Musical Instrument controlled by gaze pointing and head rotation. It is suited for playing chords. The instrument was co-developed during my Ph. D. studies period by Matteo De Filippis (as part of his bachelor thesis) and me. It’s developed as a fork of “Netytar”, with which it shares part of the interface and development framework. It is suitable for musicians having a quadriplegic disability, since it can be played without using their hands. It’s a polyphonic instrument, meant to play chords.
https://github.com/LIMUNIMI/Netychords/releases
Netychords is FOSS (Free Open-Source Software)! Download it and try it from Netychords’s GitHub Releases (link above). Right now, it is only compatible with Windows operating systems, although in the future we’d like to make it compatible with others. Just download the latest Release and run “Netychords.exe”. In order to run it correctly and enjoy it fully, you may need:
I’ve prepared a video tutorial on how to download our instruments and setup all the software you’ll need in your PC in order to play. You can find it on YouTube:
Tutorial - How to download and setup our accessible instruments
Gaze point controls chord selection through a virtual keyboard drawn on the screen. Gaze point is detected through an eye tracker, a sensor which detects eye features through infrared cameras. Moreover, a blink-based chords stop method is implemented, in order to control chords duration. Head movements around the yaw axis controls note dynamics through an head tracking sensor, mimicking the strumming movements of the hands in a guitar. Chords strumming happens when the rotation direction is inverted, and intensity is proportional to movement strength (acceleration).
In order to play Netychords properly, you’ll need an eye tracker and NeeqHT head tracker sensor. You can also try it with your mouse and keyboard, but it’s really meant to be played by gaze pointing and head rotation.
Netychords is compatible with Tobii Eye Trackers such as Tobii Eye Tracker 5 (which have a cost of around 250€) or old, out of production Tobii 4C and Tobii EyeX.
NeeqHT head tracker can be easily built DIY for around 30€, following the guide you will find on the NITHsensors and NITHwrappers page.
The virtual keyboard have differently colored keys to indicate different chords. Each color corresponds to a different root note, while different color shades indicate different chord types.
