Microphone technology at an immersive concert

In this article, we use the terms “immersive audio”, “immersive”, “spatial audio”, “3D audio”, “spatial”, and “spatial audio” interchangeably. In our view and based on our experience, these terms essentially mean the same thing. They all refer to technologies and techniques that create a three-dimensional sound experience, where sounds appear to come from different directions and distances, in order to provide as realistic an experience as possible. This type of audio offers a more immersive listening experience, regardless of the specific term used.

June 24–28, 2024, Ancienne Belgique - Brussels.

PXL-Music Research and DAE are joining forces to organize an immersive concert featuring two bands, exploring the boundaries of what is possible with game technology and how far we can go with our immersive audio setups. This involves examining audio in the context of live amplification via our immersive sound system, as well as the possibilities for recording such spectacles in the future. Finally, we bridge the gap between the two and allow audio and game technology to communicate with each other.  This article focuses specifically on microphone technology in the context of recording and amplification at an immersive concert.

Immersive microphone technology at a live music concert can be deployed in two ways:

1. Recording or capture: The concert is recorded using a 3D microphone setup, after which the recording is mixed in the studio, for example in Dolby Atmos – an immersive audio format. This finished product can then be placed on a streaming platform such as Apple Music, Tidal, or Amazon Music. An immersive concert recording consists, on the one hand, of all the individual tracks and microphones being captured, and on the other hand, of an immersive microphone setup that accurately captures the ambiance and the 360-degree soundscape of the concert. With this 3D array, you aim not only to record the audience and the ambiance, but also, during this concert, a portion of the sound reproduced via the 360° PA systems. 

2. Live amplification: Another possibility is the use of advanced spatial microphone technology to amplify the band's instruments during the concert, thereby creating an immersive sound experience for the audience.

However, it does not have to be a choice between these two options. The ideal scenario combines both technologies to achieve the ultimate result. This is exactly what we have attempted.

WITT and Pothamus – two different immersive cases:

The WITT concert, featuring frontwoman Tine Theré , offers a different immersive musical experience than that of Pothamus. Tine resolutely opts for vocal layering—stacking different voices—to create impressive immersive choirs via backing tracks, supplementing this live with six solo singers. Constructing a 3D experience from the creation of a song is unique and something that occurs too rarely.

Pothamus, a doom metal band, works from a more traditional music production style, primarily using stereo and mono source material. They translate their philosophy and tracks into an immersive experience by, for example, using custom microphone technology.

Immersive microphone technology for concert capture:

If you want to record immersive music, you usually do so with the goal of transporting the listener to the recording location. We are immersive beings by nature. In English, 'immersive' means submerged. Sound comes from all sides in our spatial, three-dimensional world: from the front, back, left, right, and above and below us. There is nothing as enchanting as being transported during a recording to the actual acoustics and musical experience of a concert hall.

So, to capture a live concert by WITT or Pothamus at the AB in immersive audio, you use custom microphone setups. You want the audience to feel the atmosphere and experience the sounds in detail.

The idea behind the choice between microphone setups can be manifold. However, for us, they can largely be divided into three different basic concepts.

1. Binaural recording

   
   

In binaural audio listening, sounds are recorded using two microphones positioned to mimic the position and distance of human ears. This is often done using a dummy head or an artificial head, in which the microphones are placed at the position of the ear canals. As a result, not only is the direction of the sound captured, but also how the sound is affected by the anatomy of the head and ears, such as the way sounds are filtered and attenuated.

When playing through headphones, the listener can experience a very realistic, spatial sound reproduction, making it seem as if sounds are coming from different directions and distances—just like in the real world.

Unfortunately, such recordings can only be played on headphones. However, they have been made for a very long time, and usually with the same microphone, the Neumann KU-100. It is a benchmark test and provides a good comparison with other technologies.

2. Non-coincident microphone setup

   
   

The second microphone setup is a non-coincident microphone setup. We opted for a modified version of the OCT setup, an OCT look-alike that we adapted based on our own findings and needs.

OCT stands for Optimized Cardioid Triangle and was introduced by Theile and Wittek. It involves a setup of three microphones positioned at specific distances from each other. Later, others created a variant with two additional microphones at the rear to enhance the immersive effect.

Non-coincident microphone setups aim to match microphone placement with the speaker arrangement in a 3D listening environment. Systems such as Dolby Atmos, Auro 3D, and Sony 360 each have their own specific speaker configurations; however, the bottom layer of these setups typically follows a similar pattern, with speakers placed at the left, right, center, and rear left and right. In our view, these are the most important channels to capture. Furthermore, for a fully immersive speaker setup, additional speakers are required.

However, due to the compact space of the AB Club, the low ceiling, and ergonomic considerations, we have not included height channels in our microphone setup. In larger venues, such as the Sportpaleis or the main hall of the AB, there would be sufficient space and reason to place four additional microphones at height.

Why did we choose OCT and not other non-coincident microphone setups? With non-coincident microphone capsules, all setups look somewhat alike, as does the final auditory result. From experience, we have found that the difference between our OCT-3D look-alike and other methods such as the 2L-Cube, PCMA-3D, or Hamasaki Square is not significant.

Setup:

The OCT is designed to minimize interchannel crosstalk for accurate localization of the frontal sound image. In our setup, we implemented the frontal triangle using two cardioid microphones (Schoeps MK4) and one omni-microphone (Sennheiser MKH8020) in the center. The OCT was originally developed to use cardioid microphones, but our experiments show that adding an omni-capsule in the center significantly improves the spatial aspect and frequency response.

The choice of the two supercardioid microphones (Schoeps MK 41) on the left and right rear sides also stemmed from experiments. The supercardioid microphone proved excellent at minimizing interchannel crosstalk and therefore offers better separation between the front and back than, for example, a standard cardioid microphone.

For the recording of the WITT and Pothamus concerts , we used the following 3D array: 

L = cardio Shoeps MK4

C = Omni Sennheiser MKH 8020

R = cardio Shoeps MK4

LS = Super Cardio MK41

RS = Super Cardio MK41

3. Coinciding microphone technology

   
   

The counterpart to non-coincident microphone technology is a coincident system or coincident microphone technology. Ambisonic is one such technology and another way of recording and reproducing three-dimensional sound. It originated in the 1960s and 70s and was conceived by the English mathematician Michael A. Gerzon and the physicist Peter Fellgett. Ambisonics is a 3D audio technology that captures and reproduces sound from all directions, creating an immersive listening experience. Unlike traditional stereo or surround sound, which focus on specific channels (such as left, right, front, back), ambisonics captures the entire sound field as a sphere. This means that you hear sounds not only around you, but also above and below you, making it ideal for applications such as VR, AR, and immersive music. If you want to convert this microphone technology to, for example, Dolby Atmos, you need a plugin that translates the entire sound environment into the specific channels of that format. The major advantage of Ambisonics is that you can flexibly manipulate the recorded sound. You record the sound space in RAW format (A-format), which you can adjust afterwards. This allows you, for example, to amplify or adjust specific parts of the sound image, such as the sides or the rear, as desired. 

A-format:

When recording in Ambisonic, you can use soundfield microphones. These are microphones that record in A-format. The A-format is the RAW format that comes out of the microphone's capsules unedited.

For first-order soundfield microphones, we use four cardio capsules arranged in a tetrahedron.

The simplest soundfield microphones record in first-order Ambisonic. "First order" indeed means that higher orders also exist, in which space is subdivided even further, resulting in more detailed information. Higher-order Ambisonics require a different approach and operate according to a mathematically different model than that of first-order Ambisonics. We did not use them for this recording, and they are therefore not relevant in this case.

First-order soundfield microphones are, for example, the RØDE' NT-SF1 and the Sennheiser AMBEO. VR. For our recording at the AB, we use the AMBEO VR.

B-format:

The B format is obtained by adding the various microphone capsules in a matrix (plugin) in such a way that you get the B format with the components W, X, Y, Z. 

W = LF + LB + RF + RB

X = LF − LB + RF − RB

Y = LF + LB − RF − RB

Z = LF − LB − RF + RB

Just like with MS (mid-side technology), we work with sound field synthesis. We do not use a channel-oriented transmission method like with Dolby Atmos or Auro-3D, but divide the space by means of bidirectional polar characteristics.

In first-order Ambisonic, space is divided into three directions: from front to back via an X-axis, left to right via a Y-axis, and top to bottom via a Z-axis.

The aim of the process is to reconstruct the direction of the sound for the listener.

There is also a fourth W component, a 360° sphere or an omnipolar characteristic.

Sensitivity:

The four components W, X, Y, and Z together form the B-format, an intermediate form from which you can create various other polar information. By adding W to one of the X, Y, and Z components, the sensitivity of the axes can be changed, as you can see below. Add an omni to a bidirectional microphone, and you get a cardio. We also use this technology in other microphones to achieve specific polar characteristics.

Direction:

By summing the X, Y, and Z axes relative to each other, we can also change the direction of rotation.

If we combine the direction of rotation and the sensitivity as discussed above, then any physical positioning in a sphere can be obtained from the B-format by adding and subtracting the various components W, X, Y, Z.

Translation to Immersive:

If you want to translate Ambisonic audio to Dolby Atmos or another multi-speaker format, you must use a plugin or manually add and subtract the various directional components. This requires converting specific components of the Ambisonic B format to the layout of a multi-speaker setup.

To drive the front left speaker, for example, you use the sum:

W + X + Y 

For the rear right speaker, the following applies:

W - X + Y

In this way, you can create all the different positions by means of adding and subtracting signals. Adding means combining components, while subtracting involves joining two components and placing one of them out of phase.

Translation plugins are available, such as those from RøDE and dearVR, that allow you to convert the B format to formats like Dolby Atmos. At PXL Music Research, we are also developing our own plugin, which offers specific benefits for immersive mixing both live and in the studio and aligns with our own research. Finally, it is of course also possible to achieve the same result via aux buses, both live and in the DAW.

You can find more information in the manual for the Josephson 700 microphone. https://www.josephson.com/pdf/srs7ug.pdf

A preliminary conclusion?

Why all these different setups? To us, it is clear that there is a significant difference in sound and atmosphere between coincident and non-coincident microphone setups. Testing and listening to these different setups offers incredibly interesting study material. The intention here is not to express a strong preference for one over the other; the differences are clearly present. Certain characteristics, such as the more coherent and compact soundstage of AMBEO, can sometimes offer advantages, while the lush, spatially more pronounced character of the OCT will appeal to many people. As always, it is important to experiment yourself and listen to the microphone within the specific context. We often find that the rich sound of a non-coincident setup initially captures the imagination, but that you can only make a choice once you listen to the microphone setup in its context.

A caveat here is the footprint during a recording and the speed of operation. With an ambisonic microphone, you can be operational within 15 minutes with the right preparation, which is virtually impossible with non-coincident microphone setups due to the complexity of the microphone setup and the it's size of the space it occupies?

An Immersive Kit Microphone

And then finally, what do you do if you want to use an immersive microphone setup to amplify a drum kit live, separate from the individual mono sources of the different drums?

We experimented with Ambisonic technology within another PXL-Music Research project and came up with the idea to replace the B-format components, as described above in the Ambisonic section, with high quality microphones possessing the same polar characteristics. The X, Y, Z components were replaced by bidirectional microphones, and the W component by an omnidirectional one. Subsequently, using the plugins discussed, we can translate this into a multi -speaker system. After testing this setup in the studio, we concluded that this approach could also work well as a microphone setup for a live drum kit. The Z component has limited impact and is difficult to control. Therefore, we decided not to it use is during the live concert. Converting the B-format system into actual microphones is an experiment that others have already performed, but using it during a live concert is a truly unique event.

The major advantage of this microphone setup is that you can use high-quality microphones, instead of the small electret capsules often used in standard ambisonic microphones. Additionally, this setup offers the possibility of manipulation; during the mix, you can make various choices, such as emphasizing spatial information over frontal information.

Was this experiment successful? During the Pothamus concert, the WXY kit microphone was active throughout the entire live show, to a greater or lesser extent. Crosstalk between the various instruments was manageable, partly because the guitar amplifiers were not on stage. However, the fact that the addition and subtraction work of the decoding had to be done via aux buses in the mixing console and not via a plugin made everything quite complex. As a result, for example, manipulations during the concert were difficult, which sometimes led to the feeling that the overheads were coming from the rear speakers.

For the recording, this kit microphone is of course a huge asset and significantly enhances the live experience.