An Accessible Digital Musical Instruments operated through gaze pointing and breath pressure, based on smooth pursuit and infinite keyboards
<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/1b5d0097-8357-4aa0-a47a-723e23488434/github_white.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/1b5d0097-8357-4aa0-a47a-723e23488434/github_white.png" width="40px" /> Github repository
</aside>
Kiroll is a Accessible Digital Musical Instrument that can be controlled through gaze pointing and breath. This instrument is particularly suited for playing melodies and was developed by Luca Valente as part of his thesis project, which I supervised, to achieve a degree in Music Informatics at the Università degli Studi di Milano. The instrument is specifically designed for musicians with quadriplegic disabilities as it can be played without the use of hands.
Being a monophonic instrument, Kiroll is capable of playing only one note at a time. Its unique design is based on an "infinite roll of monodimensional keyboards”, which users can navigate using their smooth pursuit eye movement abilities. Consequently, this instrument is not suitable for operation with hands and fingers, as it is intentionally engineered to leverage eye movements for optimal performance.
https://github.com/LIMUNIMI/Kiroll/releases
Kiroll is FOSS (Free Open-Source Software)! Download it and try it from Kiroll’s GitHub Releases (link above). Right now, it is only compatible with Windows operating systems. Just download the latest Release and run “Kiroll.exe”. In order to run it correctly and enjoy it fully, you may need:
To fully experience Kiroll, you will require an eye tracker and a NithBS breath pressure sensor. While it is possible to use a mouse and keyboard, the software is specifically designed to be played using gaze pointing and breath control.
Kiroll is compatible with Tobii Eye Trackers, including the Tobii Eye Tracker 5 (priced at approximately €250) and older, discontinued models such as the Tobii 4C and Tobii EyeX.
The NithBS breath pressure sensor can be conveniently constructed as a DIY project for around €30 by following the instructions provided on the NithSensors page.
Gaze pointing is used for note selection and is detected via an eye tracker, which employs infrared cameras to identify eye features. Breath control, on the other hand, manages note dynamics through a low-pressure sensor. In this regard, Kiroll can be likened to a digital flute.
The instrument's keyboard layout has been specifically designed for gaze interaction, leveraging the eyes' smooth pursuit capabilities. The vertically arranged keys in the moving keyboards are played by moving the gaze horizontally from left to right, selecting and playing one note per keyboard. As a note is played, the keyboards gently and smoothly move to reveal new ones on the right edge of the screen. This movement is intended to prevent the Midas Touch issue, which is common in gaze-based interfaces.
Keyboards must be played in sequence; if an incorrect keyboard is played, the interface will provide feedback indicating the correct keys to play.
Breath control is utilized for note onsets, dynamics, and intensity, similar to a flute. Alternatively, the spacebar can be used instead of breath, but this will not allow for control over dynamics.
By default, notes are arranged vertically, but can be flipped horizontally using a button in the interface. The keys are colored with high-contrast shades to exploit the human eye's peripheral vision sensitivity to color. A single octave of the C major scale is displayed by default; however, the scale and octave can be adjusted in the Settings.
Green lines and circles show the allowed gaze movements, while red lines and circles indicate forbidden movements
The uppermost part of the interface as well as the Settings can be operated through gaze pointing and short bursts of breath in the breath sensor. The last gazed button will be colored in yellow, meaning that it’s selected and ready to be activated.
Before playing Kiroll you must press Start to init the instrument. You can then Pause the interaction whenever you want. Exit closes the program.
