Immersive live monitoring options

At PXL-Music Research, we consistently hear from musicians that they would have liked to experience the immersive sound themselves during their immersive concert. If we disregard an additional immersive speaker setup directed at the musicians on stage (a major investment and much extra complexity), two possibilities remain.

Stage in the middle of the hall

One option that sometimes works is placing musicians in the audience within the immersive speaker setup. Naturally, this entails risks and is not possible in every context or venue. In an immersive venue, there is a distance from the speakers to the center of the stage. Due to the speed of sound, this physical distance results in a delay in monitoring. In critical contexts where timing is extremely important and there are few acoustic sounds audible to other musicians without delay, this will likely not be the best option. 

However, we have already tried this outside of this research project at the Club Wintercircus with the band SCHNTZL. The musicians—a drummer with a predominantly acoustic kit and a keyboardist with all digital sounds—venerated playing in the center of the hall (about 10 meters from the furthest speakers, nearly 30ms) and did so without monitoring. They listened to the same speakers as the audience around them, and in this case, it worked for them. The speakers directly above them arrived first, and consequently, more emphasis was placed on those. Due to the positioning in the center of the hall, there was more freedom in how the immersive sound was constructed—the amplified sound ultimately came from behind and above in front of the audience—and it all contributed to a totally new experience.

Binaural

A more versatile option is binaural monitoring, which simulates how we experience sound around us via in-ear monitors or headphones. Binaural solutions often rely on the principle that the differences in loudness and time with which sounds reach our two ears enable us to locate sound sources around us. With stereo monitoring, something located entirely on the left side of the stereo image will only reach the left ear, and something entirely on the right will reach the right. A sound that is exactly equally loud on the left and right is perceived as if it were in the center. With binaural monitoring, sounds located entirely on the left will still be audible on the right, albeit with a delay. That is, after all, how it is in real life. In this way, musicians can also have an immersive experience on stage via in-ear monitoring.

Amptec provided a KLANG:konductor binaural system for the immersive concerts by WITT and Pothamus at the Ancienne Belgique Club, allowing the musicians to experience binaurally through their in-ear monitors.

Customer

Depending on the KLANG device, there is a fixed number of monitor mixes that can be created. In the case of the KLANG:konductor, this is 16. If you have 16 or fewer musicians, a separate instrument balance can be created for each musician, and everyone receives a personalized mix.

In such a mix, it is possible to choose for each audio source how and where it is positioned around the musician, and how loud it is. This can be based on realism, where the positions correspond to what the musician sees around them. It can also be musical, where instruments important to the musician are given a clear place in the center front, while unimportant instruments are placed at the back left or right.

This is also visually represented in KLANG. Stereo positioning is achieved along a line from left to right through an image of a head. Binaural positioning is achieved on a full circle around that head. This provides an overview of your sources via icons, and their position relative to the listener.

It is not yet frequently used at KLANG, but with head tracking you can achieve a dynamic binaural mix. Your perspective of the mix will then change as you turn your head, just as happens with loudspeakers. Head tracking was not used at the WITT and Pothamus concerts.

Experience

How do the musicians experience the difference? These kinds of answers are always a bit abstract because of the descriptions.

The goal of a binaural mix is to make sounds via in-ears/headphones sound less like they are 'in your head' (as with stereo), but rather as if they are coming from further away. It is as if there is more 'air' surrounding the sound, which is often perceived as less fatiguing. With a spread of sounds around you, there are fewer overlapping sounds, which can sound less oppressive.

That is also the primary goal of KLANG: not the hyperrealistic experience as if you are listening to speakers all around you, but rather a more pleasant experience that emphasizes the important sounds. The experience therefore transcends the immersive aspect to some extent, as KLANG is already frequently used at concerts that are not immersive.

Naturally, we asked for the opinions of the musicians from WITT and Pothamus. With Pothamus, the effect was clearly perceived as comfortable. With WITT, it wasn't immediately clear to some musicians what the binaural was doing until we switched to stereo. Then, they very quickly asked to switch back. We would never recommend doing this yourself, but we felt it was okay to switch from binaural to stereo just once as a test during a WITT soundcheck without notifying them. That was immediately followed by angry, confused looks, and after the song, the question was directed to the monitor mixer as to what had briefly changed. The binaural remained on.