Audiovisual Theory

Drew Hemment, On magazine (5.3 Winter Solstice 1998)

Sound and vision used to be inextricably linked. In musical contexts, not only did you see who was producing the sound, but that sight was crucial to the meaning of the musical event. Sound was first technically dislocated from vision when Wagner hid the orchestra in the pit, so as to increase the mystery and drama of its affect. Later recording technology (not to mention radio and telephone) initiated a rupture that was to isolate them in separate media, a situation that is celebrated in the acousmatic music of Pierre Schaeffer (50s) where sound is free to roam without being anchored to a visual representation of its source. Sound and vision were to occupy common ground once more when technological developments yielded the first ‘multimedia’ format - the synchronicity of the ‘talkies’. But it is in recent ventures onto the plane of digital convergence that a multimedia artform has come into its own. Evolving primarily from club visuals where video mixing supplies a silent soundtrack to the DJ’s adventures in sound (in a peculiar inversion of the pianist accompanying silent cinema), it has developed from a simple multiplication of layers into a genuine synergy with the live sound and vision workouts of Hexstatic and Deconstructed Cinema and the audiovisual artworks of Hexstatic and Locust.

As well as a pastiche of different sensations that instil their effects through the juxtaposition of different forms, multimedia is also celebrated for the potential for hybrid synergetic sensations that float above the individual parts. In Audiovision Michel Chion elucidates the idea of synchresis as ‘the spontaneous and irresistible weld produced between a particular auditory phenomenon and visual phenomenon' that can form ‘monstrous yet inevitable and irresistible agglomerations in our perception.’ But in addition to fusions between the senses across the audiovisual canvas we can also detect transfusions within the particular media, as in what we might call sonovision. Chion identified the infusion of a sonoric sensibility within the image itself in the films of Orson Welles and Ridley Scott, as well as in more recent cinema where we see ‘a cinematic rhythm composed of multiple fleeting sensations, of collisions and spasmodic events.’ Here it has heightened a palpable sensuousness and materiality. 'In these movies matter - glass, fire, metal, water, tar - resists, surges, lives, explodes in infinite variations, with an eloquence in which we can recognise the invigorating influence of sound on the overall vocabulary of modern-day film language.'

Audio sense is registered as textural or rhythmic surface, as found in non-western music forms such as West African polyrhythmic drumming as well as genres such as drum 'n' bass and ambient, but less so in western tonal music with its developmental figures and arguments. This extension of the auditive may be recorded in the way that a scene is suffused with effects of colour or texture, or that image and movement become cut up and repeated, forming dense rhythmic textures separated from signification, narrative or perspective.

One revealing site where sonovision may be found is music video. What is crucial here is the way that the images are broken into fragments and repeated, and used to build dense textures. ‘Music video editing returns repeatedly to the same motifs, typically playing on four or five basic visual themes. Rather than serving to advance action, the editing of music videos turns the prism to show it facets. The rapid succession of shots creates a sense of visual polyphony and even of simultaneity, even as we see only a single image at a time’ (Chion). A special case is dance music, which itself presents an intensive, rhythmic plateau that is designed for the participatory context of the dancefloor and made to be played in the mix, rather than as a unified 'song' with clear beginning or end. Music video has always enjoyed an uneasy relationship with such a sensibility of sound as it lacks the counterpoints and dramatic intervals that might supply the synchronisation points needed for ‘audiovisual phrasing’. Without these the video either tends to totally dislocate from the music such that the two float in parallel indifference, or to impose some kind of theatrical resolution on the track so as to return it to a meaning it never possessed. Another interesting case is presented by breakbeat and drum 'n' bass which are essentially musics of rupture and dislocation. This aesthetic has been brought to the audiovisual canvas in the Reprazent 'Brown Paper Bag' video where Roni Size is able to sample, pause, repeat and timestretch life of the streets of New York, opening a space in-between time and choreographing broken motion into an inadvertent dance in a kind of plunder-kinetics. This is still subordinated to the cinematic convention of linear narrative, however, and the video contrasts the break/flow decomposition of the image with the journey of the musicians to the airport as they wander through the streets both at one with and apart from the scene they observe (compare the modernist Flâneur). The analogy in music would be less the floating thread of the jungle MC (which does not direct interpretation so much as add an extra layer to decode) than the return of the track to some kind of harmonic cadence - a closure which is only resisted by the chance that Roni might miss the flight back home.

The promise of sonovision is realised in the simultaneous audiovisual sculpting of Locust which applies the soundbyte science of the breakbeat to the cinematic source, transforming both audio and image from original footage of jazz greats and rock ‘n’ roll stars into a montage which is intensified by the fragmentation of the cinematic scene (frame) into multiple, alternating parts. Similarly in Hexstatic’s Lifeforms series the motion and sound of chirping tropical frogs, buzzing insects and splashing water are sampled and choreographed around rhythms supplied by renegade breakologists Coldcut, whilst other outings combine the effusive and strongly textural choreography of 50s Hollywood musical with live footage that has been transformed in post- production into a tapestry of colour and light, with individual scenes often broken into fragments that are juxtaposed or recombined. It is here that we truly hear ‘a transposition of sonic velocity into the order of the visible.'