A brief collaboration over a hackathon with Borut Kumperscak, Will Gallia, and Matthias Moos, organized by Nuno Correia at Goldsmith University London.
I designed a simple granular engine, which maps objects visual properties, using different weightings, on the synthesis parameters. I aimed to create meaningful connections between both auditory and visual percepts. In this video, a single computer is processing both video and audio. As you might hear, the processing load affects the sound quite a lot when using several objects. I will soon post a new video based on an improved version of the patch and a better data flow management, hopefully, in 3D HOA/binaural. This will be a good exercise for me to implement creation with sound in space through the tools I am currently developing.
These works are first attempts of composition with image and visual programming tools, frameworks for further developments.
This first use of Processing allowed me to define a framework for the use of sources into a three dimensional space.
As a musician involved into spatial composition and sensible to visual arts, I am interested in the interdependency of sources and space. From an aesthetic standpoint, I am also interested in slow transformation processes, perpetual changes from underlying patterns. This requires attention, and patience, which are actually essential to contemplation. I am also concerned with the interaction of auditory and vusual perception and cognition. I also am interested in integration of predominant visual stimuli, and in the ability of auditory stimuli to convey complexity in an integrative way. I aim to find consistent strategies to associate sound and vision at a different level of representation. I don't aim to provide an accurate representation of information, since this information doesn't exist, but to build a perceptually meaningful object. Working at two different perceptual and cognitive levels requires experience and practice. Three-dimensional visual rendering principles are widely mastered, but there is not yet such a pendant in the domain of sound, which is highly dependent on technologies, but also on what can be considered a relative weakness of auditory perception.
A First Approach
With my modest programming skills, I have used elementary visual criteria and processes, while having in mind future interpretation of input data to simulate displacements of sources into space, and the coexistence of auditory signals. These processes denote acoustic or synthesis concepts, namely sine wave decomposition and manipulation, grains, sources, auditory space, reverberation, and their interaction. They are also influenced by compositional processes, such as cycles – that is, synchronous or asynchronous periodicities –, scattering or clustering of multiple sources. By doing so, I aimed to play with the interdependency of sources and space. The use of periodicities is meant to generate multiple combinations of features that will lead to slow transformation from space to object, and drive perception to seek for gestalts that it can recreate, although patterns evolve in a rather obvious and rapid fashion, for practical puroposes.
This work was composed in two steps. First, a raw pipe recording was processed and sequenced in Audiosculpt and Live, using effects such as sonogram filters, pitch shifting, clipping filters, granular synthesis. Then, three audio tracks were extracted and mapped on three elementary shapes in After Effects. The compositional processes and succession or superimposition of materials are very straightforward, from a musical and visual standpoint, and echo the roughness of the primary sound.
This is the skeleton for a project, born from the contemplation of the landscapes reflected by the surface of a small river that ran near my appartement in Le Mans. I was stricken by the beauty and the presence of those reflections, which transcended the objects and the support, but also the very way I was looking at them, and how these had been able to immerse me in this contemplative state. I then sought for means to express this aesthetic emotion by integrating those entities, the objects, their representation, the support and the look in a plurisensory installation. This installation can be schematically described by the projection on images on resistive fabric panels with various textures and their sonification in a scenographic space, allowing for the public to interact with images, sounds and spatialization.
The scenography was initially conceived within the space of the Ecole des Beaux Arts du Mans, with the help of Florian Behejon.