Toward live recorded 6DoF audio – Unity and ZM-1 microphone arrays (2018)

6DoF (six-degrees-of-freedom) refers to the possibility of a rigid body (or a representational device of it) to move freely in three transitional and three rotational degrees.

Designing sound for immersive VR 6DoF experiences is usually done by using mono or stereo sound artificially spatialised through software processing the seeks to replicate reverberation, diffusion and other acoustic behaviors.

Now, doing a live sound recording, in a physical acoustic space so it can be used in the context of 6DoF VR is said to be impossible or, at least, very hard.

Setup of 7 ZYLIA ZM-1microphone arrays.PNG

To try and achieve just that, I used simultaneous recordings made with 7 ZYLIA ZM-1 microphone arrays, placing multiple ambisonics sound files in a scene I built in Unity 3D (visual concept by Anna Czerwoniec). The sound sound sources in those recordings are 3 musicians that were sitting among microphones. More information can be found on this blog post from ZYLIA.

The following video is a screen-capture of an animated path in Unity 3D with the binaural decoding (listen on headphones) of the ambisonic cluster, demonstrating how it is possible to walk among the musicians as if the listener were there in the room.

This initial proof-of-concept was recently presented at Game Sound Con in Los Angeles, LA and AES Convention in New York, NY – USA.

AES New York - October 2018
AES New York – October 2018
GameSoundCon Los Angeles - October 2018
GameSoundCon Los Angeles – October 2018