Using technology today often requires individuals to be quite dexterous, from controlling drone remote controllers to pressing and swiping buttons on a smartphone. These high-tech advancements don’t often take into account those who are disabled.

Now, the Performance without Barriers research group is designing digital musical interfaces for disabled musicians. The research group collaborates with disabled performers from the very start of the design phase and looks at specific abilities the individuals have when creating the new musical technology.

Currently, VR technology is designed for mainly passive interactions such as “riding” a rollercoaster or “attending” a concert. The group wanted to utilize VR technologies in an active and performative way for disabled musicians.

The integration of VR and design capabilities was a collaborative effort among electronic engineers, computer scientists, sonic arts researchers, immersive content designers, a soloist ensemble, and a local group that currently helps disabled musicians perform and compose their own music. In addition to these individuals and musicians, they also joined forces with a software developer who was in the works of creating a VR musical instrument called the Infinite Instrument, which runs on a 360-degree VR headset known as HTC VIVE.

The Infinite Instrument, originally designed with able-bodied musicians in mind, was reconfigured to take into account different types of mobility. In turn, a cerebral palsy musician played a VR instrument that was designed to cater to her expressive upper body movements. It did not require her to use fine motor precision in her arms or fingers, because she did not possess that type of mobility.

Although VR headsets are what we normally see when referring to a VR experience, the researchers found that the tactile feedback from hand-held controllers, with which the user operates to press commands, access the menu, and reach content, allowed the musician using the device to play the instrument by feeling and hearing it, instead of seeing it through a headset. By not relying on a headset, the musician could still see other musicians around her during the performance.

The research group also worked with a blind performer, in which they built from his expertise with the clarinet and used VR to enhance his musical performance. In this instance, a VR headset is clearly useless, but they wanted to know how they could use other immersive qualities of a physical space for a music performance using VR.

“VR experiences are so concentrated on the visual experience that often they disregard the inherently immersive nature of sound, but it’s all around us, a proper 360° experience,” said Franziska Schroeder, senior lecturer at Queen’s University Belfast, and Matilde Meireles, postdoctoral research fellow, according to their article on The Conversation.

The team is located at Sonic LAB in Queen’s, which is an immersive, 3D sound space, also referred to as “the iMAX for the ears.” Using this space, the researchers adapted the VIVE technology to the context of the blind musician.

They focused on VIVE controllers that were more tangible rather than visual. They could also track physical positions in a space, so the clarinetist could position sounds in the 3D sonic LAB by using the VIVE controller that was strapped to his instrument.

All of the VR instrument designs and musicians were featured in a concert, where disabled musicians and the Hard Rain Soloist Ensemble performed side-by-side.

“Our approach reveals how new technologies can be developed, that actively engage disabled musicians in music making and demonstrate a commitment to quality of life for disabled musicians,” said Schroeder and Meireles, according to their article on The Conversation. “ We will continue to design instruments that can be used in VR, but we will now focus on using more affordable systems, with a view to creating a virtual reality ensemble of disabled and able-bodied musicians.”

---

Filed Under: Product design, Virtual reality

 

---