Corpus Of Sound: : Bodycoder and the Music of Movement

Drew Hemment
Mute (Issue 10, 1998, pp: 34-9)

The advent of electronic technologies has reduced the traditional exuberance of musical performance to the minimalist gestures of knob twiddling and button pushing. But there are those who are re-closing the gap between sound, body and image. Drew Hemment reports.

Most music technology has been developed with only the studio in mind, and has not yet been translated into forms appropriate for live performance. Music technologies were designed primarily for cleaning up and enhancing previously recorded material - when they came to be used for original sound generation, 'live' music production was subsumed by the studio environment without a corresponding evolution occurring in the technologies employed to perform it. The future of electronic music lies in designing interfaces that allow a more intuitive relation with sound. Part of this future lies in software development and the technical research being done into MIDI formats etc., but they will remain only part of the solution if they aren't situated within a culture of artistic competence. In particular it is the study of music in motion, or what I term kinesonics, that promises to break open the black boxes of the techno fetishists, and it is this horizon which is explored by Bodycoder, an interface and instrument developed by Julie Wilson and Mark Bromwich.

Bodycoder transforms the performing body into a virtual instrument linked by radio transmitters to sound and video projection systems. A cybernetic suit of MIDI-linked movement sensors digitally encodes movement, triggering and modulating samples and sounds. The result is simultaneously dance and composition - the dancer's contortions modeling an original score in a performance that (rarely for this type of electronic experiment) coheres motion, sound and vision into one, 'smooth' experience.

Ordinarily a performance to music, dance tends to be seen and not heard. But Bodycoder makes music out of the body's most subtle and intimate motions, transcoding gesture and physical expression into sound. The performance occurs in the contortions of the dancer's body, but also in the modulations in sound as well as in the morphing images which dance to the body's tune across the video screen enclosing the performance space. As Wilson states: "These are characters in the complex landscape of performance which is both real and hyperreal, which is something to do with interiors and surfaces and the other spaces which are opened up through performance". For Wilson and Bromwich, Bodycoder's developers, inspiration was borne of frustration at, firstly, the sterile and distanced nature of most electro-acoustic performance, and secondly, the way that technology all too often constrains rather than liberates performance.

Electronic technologies might have freed music from the parameters of the instruments, but this has, so far, occurred at the price of the kind of engaging visual performances which draw audiences in to the musician's world. In improvised performance in particular, what the musicians do
not play is sometimes considered just as important as what they do play. In bearing witness to the drama that unfolds between performers and their instruments, the audience is engaged in ways that a vista of anonymous audio-technicians hiding behind banks of machines rarely achieve. Whilst some maintain that even the pushing of a button or the turn of a dial incorporates the body and consequently the artists' emotional involvement, arrays of black boxes still tend to mystify the production process and it has only really been in dance music - where the action moves to the floor - that the relationship has been a happy one.

Bodycoder, on the other hand, seeks to animate the human form within the virtual environment. Its power is raised not compromised by entering into negotiation with the 'machinic assemblage'. In ways similar to those by which the potentiality of the violinist may be raised when subsumed within the machinic assemblage of the orchestra, the Bodycoder system aims to raise the expressive potential of both human and machine.

With its emphasis on the synergy of live performance, Bodycoder departs from the recording aesthetic that marks most forms of contemporary music, even those such as rock or folk which edify an embodied authenticity. Driven as it is by the tripartite axis of phonograph-tape-sampler, its reinvigorated instrumentalism might seem at odds with digital-electronic innovation, but Bodycoder allows the performer full instrumental control over sound. Within Bodycoder's performance suit Julie Wilson's arm is transformed into a modulation wheel, as might be found on a synthesiser, allowing far greater expressive range than simple triggering does. Bodycoder's system uses the powerful EMu E4X Sampler for real-time additive synthesis and frequency modulation, which is further controlled by twenty forms of real-time filtering including, for example, morphing between filter profiles. So while there are no acoustic instruments used - or 'played' - at source, the sound states produced are in every sense live.

KINESONICS

In importing the motion of the body into the sonic sensorium, Bodycoder provides an example of choreography in sound or kinesonics - the study of movement in music. There is a perennial philosophical thesis, revitalised by discussions on cybernetics and currently enjoying a certain vogue, which states that intelligence is embodied rather than just a result of abstract cerebral contemplation. Theorists such as David Porush (MIT) see the 'brain' as infusing the whole body in such a way that creative engagement comes from the whole body's physical encounter with sound and instrument (a view that would be supported by anyone working with a tactile art form). In this light most electronic interfaces (mouse, keyboard, switches) may be seen to seriously restrict the body's potential whereas kinesonic systems become particularly interesting.

Whilst a direct parallel may be identified in Rolf Gehlaar's work with ultrasonic sensors in the seventies (which developed the notion of body-as-instrument), Wilson and Bromwich locate themselves specifically in a tradition inaugurated by Leon Theremin. His instrument, the Theremin,
though difficult and cumbersome to master, allowed the movement of the body to be converted into 'music' by registering the interference patterns induced into an electric field. It initiated an exploration of virtual worlds - one which does not trace out the parameters of an instrument so much as map an ideal sonic space. The ethereal, meandering sounds it produced abducted the ear of western
music, which were conditioned by tonal harmonics, and are still used in contemporary music (Portishead and Scanner) to instil effects of otherworldly transcendence.

Theremin's initiative was expanded upon through such instruments as Don Buchla's Lightening instrument and Walter Fabeck's Chromosone. The Lightening (Mark 2) works on similar principles to the Theremin, except where the Theremin registers pitch and volume only, it maps the position and velocity of infrared wands in 3D space and converts this information into any parameter. It can also 'learn' from other instruments via a separate MIDI input (and so respond to a keyboard if you are jamming) and has switches which allow the operator to scroll through menus and initiate, or delay, effects. The Chromosone, like Bodycoder, was developed at STEIM. It's a visual feast, incorporating an LED-lit perspex 'keyboard' which functions as a physical reference for the movements of two datagloves in three perpendicular ultrasound fields corresponding to pitch, volume and timbre. The keyboard can also be tilted round a pivot to move the X plane and so realign one pitch field in relation to the other two (a gestural method of scaling), while foot peddles and switches in the fingers of the gloves control yet more parameters. This is all connected by an umbilical chord to the complex brain of a Sensorlab unit where all the processing takes place.

Whilst these projects show how adaptable such instruments can be, all of them share limitations in the time and effort required to master them. Indeed, it is often only the designers themselves that ever become proficient. Mark Bromwich is clear about the political dilemma and potential
elitism this presents; arguably the more complex the instrument the less accessible it becomes. One project which tackles this problem head on is the Soundbeam instrument. It emits an ultrasonic beam whose variable thresholds form an elastic and invisible keyboard in space, allowing the slightest movement to trigger an infinite array of prerecorded sounds which can be passed through a number of effects. One particularly effective application of this has opened worlds of sound to people with special needs by situating the operator within a localised beam so that any part of the body is given the capacity to compose music by moving.

Another innovator in this field is Alexi Blinov, who is currently working with multimedia misfits AudioRom. Since escaping post-Glasnost Russia with a bag full of tricks and a head full of ideas, Blinov has developed his twin interests - lasers and opto-electronics - in ways which are at once esoteric and accessible. Laser art presents a unique possibility to operate with true colour, and facilitates the creation of optic-objects without the metaphysical inconvenience of having to give colour to a previously constructed form. But whilst it is the potential of laser projection for creating animated, interactive sprites that truly excites Blinov, the inhibitive costs involved have forced him to focus on working with more straight-forward infra-red and visible light sensors. Much of this interface technology is not revolutionary, but his great achievement is in making practical and effective devices such as the compact LAT unit.

There is less scope here for empathy between the morphology of movement and the sounds produced. When compared to, say, the subtle and sophisticated translations of a violin bow such devices might seem to provide limited expressive potential. But they compensate for this by maximising the sensitivity and adaptability of the interface. Andre Ktori of AudioRom explains that in all their work, this loose collective of artists aim to develop interfaces which can be highly responsive to physical gesture and so provide maximum scope for the player to feel a direct participation in the audiovisual action. Interactive audio environments such as Trigger Happy fall short of the potential for sophisticated sound generation offered by Bodycoder, and yet they offset this limitation in the capacity they offer for involving performers and audience within a kinesonic musical event.

KINESONICS AS SONOMOTIVE

There is nothing new in the idea that transference occurs between the senses. Indeed the boundaries are rarely as distinct as we might like to think. The notion of a one-to-one correspondence between the senses and the sense organs does not stand up to scrutiny. Instead, we can talk of a 'calibration' of the senses, such that the visual, audio or tactile modes may spread across the perceptive field. This also becomes a political question when it is claimed, as it was for example by McLuhan, that we are leaving a literate or visual epoch epitomised by a relation to 'linear' print running across a page, and entering a tactile or audio epoch characterised by immersive textures and open polysemy. If 'audiovision' denotes the synergetic fusion of disparate media, then perhaps 'sonovision' might denote the infusion of a sonorous sensibility into the image, and, similarly, 'sonomotion' the extension of 'audio sense' into the sphere of movement - kinesonics. What is important to 'audio sense' is immediate affect rather than narrative progression or perspectival depth. Sonority is registered as textural or rhythmic surface, as seen in non-western music forms such as West African polyrhythmic drumming or genres such as dance
and ambient, but less so in western tonal music with its developmental figures and arguments. This extension of the auditive may be manifested in the way that a scene is suffused with effects of colour or texture, or that image and movement become cut up and repeated, forming dense rhythmic textures separated from signification, narrative or perspective - as witnessed in the music in motion of Bodycoder.

One site where sonovision may be found is music video. What is crucial here is the way that the images are broken into fragments and repeated, rather than building a narrative whole within linear or 'dramatic time'. Music video editing returns repeatedly to the same motifs, typically playing on four or five basic visual themes. Rather than serving to advance action, the editing of music videos turns the prism to show its facets. Their rapid succession of shots creates a sense of visual polyphony, even of simultaneity, although we see only a single image at a time.

Another special case is dance music, designed for the participatory context of the dancefloor and made to be played in the mix, rather than as a unified 'song' with clear beginning or end. Music video has always enjoyed an uneasy relationship with such sensibilities as it lacks the
counterpoints and dramatic intervals that might supply the synchronisation points needed for 'audiovisual phrasing'. Without these the video either tends to totally dislocate from the music such that the two float in parallel indifference, or to impose some kind of theatrical resolution on the track so as to ally it to a meaning it never possessed. Another interesting example is presented by breakbeat and drum'n'bass which are essentially musics of rupture and dislocation. In Reprazent's Brown Paper Bag video, for example, Roni Size is able to sample, pause, repeat and timestretch life of the streets of New York, opening a space in between time and choreographing broken motion into an inadvertent dance in a kind of plunder-kinetics. However, this is still subordinated to the cinematic convention of linear narrative, and the video contrasts the break/flow decomposition of the image with the journey of the musicians to the airport as they wander through the streets both at one with, and apart from, the scenes they observe. The analogy in music is less the floating thread of the jungle MC (who adds layers for decoding rather than directing interpretation further) than a return to some kind of harmonic cadence - a closure the narrative only resists via the possibility that Roni might miss the flight back home.

In the audio-visual realm, only AV-artesans such as Hexstatic and Locust have properly broken down the visual frame. In Hexstatic's Lifeforms series the motion and sound of chirping tropical frogs, buzzing insects and splashing water are sampled and choreographed around rhythms supplied by renegade breakologists Coldcut. Similarly, Mark van Hoen (aka Locust) works on the principle of constructing sound and vision out of the same original material. His AV-works apply the lore of the breakbeat to original footage of jazz greats and rock'n'roll stars, sculpting dance tracks in sync with visual montages of riffs and licks. This synthetic realism does not reassure the listener that everything is in its proper place so much as startle and perplex them with the juxtaposition of irreconcilables. Similarly, the audio sense that sometimes resonates through music videos comes into its own in the live video mixing of the likes of Hexstatic and Motion Graphics. These might combine, say, the effusive and strongly textural choreography of '50s Hollywood musicals with live footage - transformed in post-production into a tapestry of colour and light - and individual scenes broken into fragments, then juxtaposed or recombined.

It is primarily in installations (such as Trigger Happy) and club-based audiovisual outings that the visual frame is ruptured and the screen displaced from centre stage. In such cases not only is the music free to roam but the visual action itself seeps out from its neat circumscription and connects with the wider environment. A recent Deconstructed Cinema collaboration with film-maker Tony Woodhead at a Raya night at the ICA saw projections sent flying round a dizzying array of moving screens including the windows of the venue, found objects, mirrors placed on revolving turntables, and a disco mirror-ball suspended from the ceiling. Meanwhile Bodycoder aims to further sunder the constrictions of standard 'multimedia' formats and is developing an ambitious production that will fragment the performance space by splitting the screen into many translucent layers (made up of waterfalls, water jets, dry ice and smoke cascades) to create a shifting space of visual subterfuge and audio deceit.

Info on Bodycoder and Electronic Dance Theatre from smusmab@pegasus.hud.ac.uk