Submergence (2018)
Detail
Submergence is a 3D spatialized composition and sound design for National Film Board’s Interactive Studio VR film Tidal Traces by Nancy Lee and Emmalena Fredriksson. The 360 dance film explores movement choreography located in the inter-tidal mudflats near Vancouver. For more information on the film, visit the NFB’s webpage here
Artistic Context
I intended to influence the gaze of the viewer through two overlapping-independent sound spheres, which occupied the 3D space: 1) Environmental Sound 2) Action Sound. The former relates to the overall ambience of the film, and the latter to active moments in the choreography which I wanted to emphasize in order to gain the viewer’s attention. The film score and sound design was composed, mixed and spatialized using Facebook’s 360 Spatial Workstation (Version 1.3) within the Reaper DAW. The Environmental Sound resulted in the introductory ambient backdrop.
A sound mass of found sounds (field recordings) and digitally created sounds representing the elemental sounds of the water, wind and land acted as an aural primer for the viewer to habituate to, and be able shift their attention to differing timbres presented throughout. I wanted the viewer to feel as though they were present at the location without it being an exact representation of what it would sound like. This sound sphere gradually transformed by applying micro pitch and filtering effects which kept the sense of temporality of the work.
Mixing audio through Facebook 360’s Spatial Workstation
The compositional proccess; using a monoscopic version of the film
The Action Sound sphere was influenced by the choreography and scene changes. During the beginning of the work, a violin melody entered in order to produce a sensation in response to the expansiveness of the environment, and to the distance and motion of the dancers. Violin plucks and other percussive sounds acted as moving trajectories, which acted as aural cues for the viewer to follow and shift their gaze. In specific, when two different choreographic scenes were located in opposite angles of the 360° environment concurrently, these sounds emerged towards them in order to guide the movement of the participant. Additionally, water splashes were designed and accompanied the movements of the dancers with changing attenuation based on how close they were to the viewer. The addition of this organic sound constructed a more authentic experience, which coalesced with the musical score
Technology
The film score was mixed into a 3D sound space using Facebook’s 360 Spatial Workstation (Version 1.3), Reaper DAW and tested with the Oculus Rift. Multiple audio formats were generated to accommodate different delivery platforms. The binaural audio was encoded in 1st-order AmbiX as well as 8-channel TBE and 2-channel head-locked stereo.
Creative Team