PGPD essay

October 25, 2008 at 3:00 pm (Further Research & Contextualisation)

*note: this is the incomplete version, the final version is on my stolen computer – there may be mistakes!

How does the relationship between music and image making in the works of Kandinsky and Klee compare to composer’s working within the medium of Electroacoustic music?

The aesthetic of the Russian abstract painter Wassily Kandinsky (1866-1944), was greatly influenced by his sensitivity towards music throughout his life. In his seminal text Concerning the Spiritual in Art, Kandinsky constructed a treatise for composition that aspired to cause vibrations in the viewers soul, stemming from the expression of the artists ‘inner need’; an art that rejected the ‘outer’ materialism of the times through the purely representational. (Kandinsky. 1977) For Kandinsky, these ‘vibrations’ could ‘…only be achieved through a visual language independent of the forms of reality.’ (Dabrowski. 2003: p79) He perceived music as one such language, capable of reaching an abstract form of expression that could resonate deep within the listener’s being.

Kandinsky greatly admired the works of Austrian composer Arnold Schoenberg, whom he befriended shortly after attending a concert of his works in 1911. (Wasserman. 2003: p23) Schoenberg rejected the traditions and rules of harmonic resolution (consonance) within the western tempered scale, and thus developed the chromatic scale, a 12-note system that freed dissonance from its obligation to be resolved. His controversial ‘emancipation of dissonance’, which removed the recognition of the key of a piece of music, sought to breakdown the hierarchy of notes within traditional tonality. This was something Kandinsky could understand and relate to within his own work. The use of a ‘single pictorial key’ in Composition V (1911) rendered ‘in muted greys with only a few touches of brighter colors [sic]’ could be seen as a move by Kandinsky to disrupt any hierarchical relationship between the colours. (Dabrowski. 2003: p90)

Kandinsky and Schoenberg shared similar compositional objectives in their respective arts; exploring new relationships between colour and form away from the traditional languages of expression. Schoenberg’s early atonal pieces, based on his theory of emancipation of dissonance, allowed him to treat timbre, texture and the colour of tones as equally fundamental to music as pitch (Ronsen. 1975: p107). Likewise, Kandinsky integrated ideas of harmonic dissonance within his oeuvre, in compositions such as Impression III (Concert) (1911). Based on one of two sketches made during a Schoenberg concert, Kandinsky’s piece may be viewed as a visualisation of dissonance. The lurid discord of vibrant yellow juxtaposed against the abstracted form of the black piano morphs ‘the instrument that created the sound into an embodiment of the sound itself.’ (Wasserman. 2003: p23)

Soon after the concert Kandinsky wrote a letter to Schoenberg, empathising with his work. Kandinsky praised his treatment of sonic forms saying that the freedom inherent in ‘“…the independent life of the individual voice in your compositions, is exactly what I’m trying to find in my paintings’”. (as cited in Wasserman, 2003: p.25) Perhaps this individual voice that Kandinsky relates to his own work is the voice of colour and form; the constituent parts of composition that he attempted to free from representation through his development of an individual abstract language. (Kandinsky. 1977)

It has been documented that Kandinsky was able to visualize colour and form vividly whilst listening to music. One account of this was his description of experiencing Wagner’s Lohengrin in 1896: ‘“I saw all the colors [sic] in my mind; they stood before my eyes. Wild, almost crazy lines were sketched in front of me.”’ (as cited in Dabrowski. 2003: p83) Some attribute his overt sensitivity towards sound to synaesthesia, a neurological condition that takes the input of one sensory experience and translates it into another. For example, a synaesthete who has ‘colour hearing’ may respond to certain spoken words (auditory information) by seeing a specific colour. According to case studies in the book Synaesthesia: the Strangest Thing, this seeing is not necessarily an image in their mind’s eye, but rather the feeling or sense of that image. True synaesthetes develop a strong idiosyncrasy with their stimulus trigger (e.g. the name Lucy) and their particular linked response (e.g. the colour blue). This link never changes and many synaesthetes describe having had these links as far back as they can remember. (Harrison. 2001) It’s possible that Kandinsky did have the colour hearing, as hinted at in Concerning the Spiritual in Art. He forms strong relationship between sound and colour in terms of frequency and timbre; linking low frequency tones, like deep bass to darker colours and high frequency tones, such as a shrill trumpet-note to brighter colours like lemon yellow. (Kandinsky. 1977: p49) Kandinsky’s sensitivity also extended towards ‘the haptic (prickly versus smooth) and thermal (cold versus warm) properties of colour’ (Dann. 1998: p56), as seen in his schematics within the chapter The Language of Form and Colour. All these compositional approaches sought to propel his work towards an abstract fusion of the senses in visual form, modelled on music’s non-material ability to express inner need:

Colour is the keyboard, the eyes are the hammers, the soul is the piano with many strings. The artist is the hand which plays, touching one key or another, to cause vibrations in the soul” (Kandinsky, 1914: 52)

Paul Klee (1879-1940) was another painter who, like Kandinsky, explored a compositional approach that stemmed from ‘the condition of music.’ (Kagan, 1983: p155) Originally trained in music, Klee had aspirations of becoming violinist, which he later abandon in favour of visual art. (Hall, 1992: p5) Music was still very important to him though, citing composers of the classical tradition, like Mozart, as a great influence. During his time teaching at the Bauhaus he developed a visual language that reflected upon ideas of musical harmony, polyphony and rhythm. (Zilcher, 2005: p55) He termed the coin ‘structural rhythm’ to describe how several parallel lines can function together to create patterns and attempting to ‘translate temporal elements (rhythms) from music into painting.’ (Duchting, 2002: p35) In a study of Bach’s Sonata No. 7 in G Major he made a schematic for the registration of pitch. He used this as a means of explaining music’s ‘structural’ functionality (larger units that could be divided due to repetition without variation) and ‘individual’ functionality (irregular units that don’t repeat), ideas that he then incorporated into his painting. (Duchting, 2002: p36)

He also developed the idea of polyphony (multiple voices) within works such as Three Subjects, Polyphony (1931). To my eyes, the three overlapping visual forms seem to build a harmonized relationship like a chord in music. However, it seems as though their relation to one another needs time to unfold. The interweaving lines, shadowed in brown and pink help to define a more complex form of ‘endotopic’ and ‘exotopic’ space, which Hall (1992: p.92) equates to the time needed to understand the harmonic relationships and overall timbre of the notes that make up a chord.

Kandinsky and Klee appropriated different schools of musical thought into their visual language; the former finding an analogy between visual abstraction and atonal music, and the later incorporating the rules of harmony and rhythmic structure into his image making. However, it could be said that in both cases, their musical understanding was confined to the traditions of Western tonality; a vernacular comprised of ‘metrically organized harmonic and melodic relationship.’ (Smalley. 1986: p61) Even though Schoenberg rejected traditional notions of tonality, he still utilized an intuitive approach to harmony, leading to a music whose harmonic tension was still reminiscent of tonal music. (Wishart. 1996: p40) It wasn’t until the later developments of experimental music in the 1950’s, that musical understanding moved away from its traditions within tonality. Influenced by Zen Buddhism, composers such as John Cage attempted to free music from the constraints of its heritage simply by letting a ‘sound exist, itself, in a changing sonorous environment.’ (as cited in Nyman 1999: p50)

This notion of liberating sound from causality was an idea later explored in the work of Pierre Schaeffer, the founder of acousmatic music. (Wishart. 1996) Schaeffer’s oeuvre strove to draw the listener’s attention to the intrinsic properties of sound through the method of reduced listening, ‘the apprehension of a sound without relation to its source.’ (as cited in Wishart. 1996: 129) Schaeffer treated sounds, once removed from their context through recording and played back over the loudspeaker, as sound-objects and developed an approach that explored the intrinsic (e.g. textural, timbral and transformational) possibilities of sound within musical composition. (Wishart: 1996)

These new methods of analysing the ‘life’ of a sound, were later adopted in the compositional approaches of composers working within the genre of electroacoustic music, who recognized ‘the inherent musicality in all sounds.’ (Smalley. 1986: p61) A wider palette of sounds had become relevant and readily available to these composers through technology and a new aesthetic approach to music was developed. Electroacoustic composer Trevor Wishart, incorporated the Schaefferian notion of the sound-object into his music. (Wishart. 1996) However, Wishart and other composers within the genre, felt that solely focusing on the intrinsic characteristics of a sound was too narrow, since most peoples natural response to hearing a sound is to relate it to its source. (Smalley: 1986) This lead Wishart to develop a method of using the extrinsic (e.g. the cultural, metaphorical and associative) aspects of sound along with the intrinsic. (Wishart. 1996) He coined the term sound-image to describe his compositional approach; transforming the visualization of a sound in the minds eye and utilizing its extrinsic features to create symbolic links between these conjured images. In his composition Red Bird (1978) he transformed the voice of someone saying ‘Liss…’ (from the word listen) into the sound of birdsong (a metaphor for flight), allowing the voice to ‘take flight’ and suggesting a ‘metaphorical link with the concept of ‘imagination’’. (Wishart: 1996: p166) Though Wishart concentrated on the metaphorical links between evoked sound-images in Red Bird, the wider aesthetic electroacoustic music was to explore ‘how the imagery evoked interacts with more abstract aspects of musical composition.’ (Emmerson. 1986: p7)


To me the ability of the Electroacoustic composer to visualise sound shares common ground with the work of Kandinsky and Klee. Electroacoustic compositions may explore a combination of representational and non-representational sound-images through intrinsic and extrinsic approaches to material. The piano in Kandinsky’s Impression III (Concert), could be seen as both as the extrinsic, representational form of the instrument itself (the source of the sound and it’s wider symbolic meaning), and a manifestation of the behaviour of the sound (the intrinsic). I perceive the piano as somewhere in between the two, since although it is an abstract form, it may still be recognised as a piano due to its contextual relationship to other forms. This mirrors some of Wishart’s ideas about giving clearer meaning to sounds of a more abstract nature, through the juxtaposition of more representational elements. (Wishart. 1996: p157)

In contrast, Klee’s piece Three Subjects, Polyphony, which plays off the functionality of rhythm and polyphony in tonal music, can be seen as an examination of the possible relationships between sounds in the creation of a chord. Rather than hinting at the source of any single sonority, as Kandinsky’s piano does in Impressions III, this piece uses the three interlocking lines to hint at the gestural movement of three separate tones over time. Klee’s approach to incorporating musical ideas in his painting seems more centred around structural and temporal relationships of music within the Western tonal tradition, with perhaps less of a focus on the intrinsic properties of sound. On the other hand, Kandinsky’s synaesthetic experience of music leads me to believe he did have an understanding of intrinsic properties of sound and how it can behave, as described previously in his response to Wagner’s Lohengrin.

Whilst Kandinsky and Klee used music as a catalyst for the creation of a new visual language, electroacoustic composers used the imagery evoked by sound to develop new approaches to musical discourse. Although the methodology differs in technique, what is central to both, is the objective of forming a new coherent language based on the relationship between sound and image.


Brougher, K. Mattis, M. Strick, J. & Zilcher, J. (2005) Visual Music: Synaesthesia in Art and Music Since 1900 London: Thames & Hudson
Cavendish, M. (1975) A Popular History of the Arts, Severn Valley Press Limited
Cox, C. & Warner D. (2004) Audio Culture: Readings in Modern Music London: Continuum
Costa Meyer, E. & Wasserman, F. (2003) Schoenberg, Kandinsky, and the Blue Rider London: Scala Publishers ltd
Dann. T (1998) Bright Colours Falsely Seen: Synaesthesia and the Search for Transcendental Knowledge New Haven: Yale University Press
Duchting, H (2002) Paul Klee: Painting Music London: Prestel
Emmerson, S (1986) The Language of Electroacoustic Music London: Macmillan Press
Hall, D. (1992) Klee London: Phaidon Press ltd
Harrison, J (2001) Synaesthesia: The Strangest Thing Oxford: Oxford University Press
Kagan, A. (1983) Paul Klee: Art & Music, Cornell University Press
Kandinsky, W. (1977) Concerning The Spiritual in Art, Dover Publications
Moritz, W. (2004) Optical Poetry: The Life and Works of Oskar Fischinger Indiana: Indiana University Press
Napier, E. R. (2006) Wassily Kandinsky: Sounds New Haven: Yale University Press
Nyman, M. (1999) Experimental Music: Cage and Beyond 2nd Edition Cambridge University Press
Ronsen, C. (1976) Schoenberg London: Marion Boyars
Schafer, R. M. (1977) The Soundscape: Our Sonic Environment & the Tuning of the World Vermont: Destiny Books
Smalley, D. (1997) Organising Sound, Cambridge University Press
Wishart, T. (1996) On Sonic Art, London: Routledge
Whitney, J. (1980) Digital Harmony: On the Complementarity of Music and Visual Art, Kingstonport Press

Costa Meyer, E. & Wasserman, F. (2003) Schoenberg, Kandinsky, and the Blue Rider London: Scala Publishers ltd
Wishart, T. (1996) On Sonic Art, London: Routledge

Fischinger, O. (2006) Ten Films Centre for Visual Music

Permalink 1 Comment

Motion Tracking: David Rokeby’s “Very Nervous System”

October 8, 2008 at 11:12 am (Further Research & Contextualisation)

I wanted to look further into the possibilities of translating movement into sound using motion tracking via a video camera. I came across this work by interactive artist David Rokeby:

 I’m actually pretty blown away by the amount of gestural control he seems to have over the sound! It appears as though the music has some kind of linearity that only occurs when movement is detected. Individual sounds (or clusters of sounds) seem to be positioned at different points in the space and can be triggered when the body moves into that space; hence (in the second video) he repeats movements in a certain space, and the same sound occurs. The velocity at which the movement travels seems to determine the speed at which the sequence of sounds are played through – notice the big sweeping motions making large gestural sweeps with the sounds. However, I may be (completely) wrong! I will research further into this.


I was wrong (in part)! As Rokeby puts it:

“The installation is a complex but quick feedback loop. The feedback is not simply ‘negative’ or ‘positive’, inhibitory or reinforcing; the loop is subject to constant transformation as the elements, human and computer, change in response to each other. The two interpenetrate, until the notion of control is lost and the relationship becomes encounter and involvement.” (

Rokeby is describing a kind of intelligent system that alters the way it behaves as the user interacts with it more. I would like to find out in more detail how he implented this feedback mechanism. What amazes me is that this piece was created in 1986. The man is obviously a genius! Seeing this is really encouraging, as it highlights the possibility of me achieving (with enough work) the translation of movement through a space into sonic experience. I must research this further through practical work in MaxMSP and Jitter. I’m excited!

The theoretical concerns regarding user interaction are certainly interesting to me. I plan to research into this further as I get deeper into motion tracking. Through my practical work, I’ve decided that I want my interface to be hidden and intuitive like Rokeby’s. However, this is not completely set in stone. As I begin unlock the possibilities of motion tracking and MaxMSP and Jitter I may discover another solution.

In Very Nervous System, Rokeby’s reasoning for creating a dynamic, hidden and intuitive interface stems from a need to counteract the language of the computer:

“Because the computer is purely logical, the language of interaction should strive to be intuitive. Because the computer removes you from your body, the body should be strongly engaged. Because the computer’s activity takes place on the tiny playing fields of integrated circuits, the encounter with the computer should take place in human-scaled physical space. Because the computer is objective and disinterested, the experience should be intimate.” (

Permalink Leave a Comment

Big Ben in Max (update)

October 7, 2008 at 10:25 pm (Big Ben Composition - Prototype 2)

Today I returned to the patch and started to tidy it up in order to make room for the changes I wanted to try out. Instead of going back to Logic and editing sounds there, I thought I’d do as much as I could in Max – after all, one of the key objectives for my project is to get a better understanding of its potential!

The changes I have made / effectiveness / notes, etc (for my reference) ;

1) A sub-patch which randomises pitch between -1 (in reverse at orig. speed) and 1 (forward at orig. speed).

Helps create some interesting variation in the sounds, making the sample layering more dynamic both texturally and in terms of frequency (e.g. Samples pitched very low, bring out the lower end of the frequency spectrum). However, these very low pitch samples may have lost their clarity, as it is unclear what may have caused the sound (e.g. seagul, gong, siren). Some level of listener recognition is important. This can be rectified in the sub-patch without too much trouble (hopefully!)

notes: Using the groove~ object probably isn’t the best option, because if a sample plays backwards (e.g. is less then 0) you need to tell it to play the sample from the end! Anyway I figured it out, so it will suffice.

2) Randomised volume for both Left and Right channels of each sample.

This worked very well I feel. The ever changing position of the sound-objects in space is exciting!

3) Used a fade out and EQ sweep when the ‘darkness’ audio is finished.

This tiny gesture helps to create a smoother transition from the absence of image to its presence.

notes: The volume fade out isn’t smooth, you can here it clicking through different levels from 1, 0.9, 0.8, etc. Very irritating (to me), but probably not hugely noticeable.

Here is 1 minutes worth of my patches ‘dark’ audio:


These last few days in Max has been great! My initial fear of using it has been conquered and I feel I am making progress fast. What I might try and do next is do something more interesting with the seagul sounds and then perhaps after that begin thinking about whether I can involve interaction through a web cam!

New patch pic, isn’t it pretty!

Permalink Leave a Comment

Big Ben in MaxMSP

October 5, 2008 at 10:20 pm (Big Ben Composition - Prototype 2)

After finishing some variations on my sound material these last few days, I decided to piece together my idea in Max.

When I attempted this previously I struggled finding the right objects to use, but today I was determined to find a solution. I found that where I went wrong before was not referencing the name of the audio sample in the buffer correctly.

Creating the rest of the patch was fairly straight forward from this point. Max Help was useful, as was referencing and taking parts from previous patches I had.

Here is a screen shot:

I started by creating a timing mechanism (counter/metro) that would trigger 1 of 4 prepared samples at random for each type of sound (seaguls, gongs, sirens) when specified.

After doing this and sync the timing with the video I sat and listened. I found the relationship between sounds got a little boring after a while. It was too repetitive! (then again I’ve been working on it for a while…) To make it a bit more interesting I randomized the volume of each sample when triggered, to alter the dynamics each time. This could perhaps be extended to EQ and maybe playback speed of each sample.

At this current point I very happy with my progress, but I need to work a bit further on the sounds to keep the listener interested in the ‘dark’ moments. Perhaps making a set of 6-8 for each sound and exploring more elaborate variations and contrasting sonic activity will help me achieve this.

Permalink Leave a Comment

A few ideas…

October 1, 2008 at 5:36 pm (Experiments, Reflections & Planning)

Clap Space (part 2)

I started working some more on this piece, which is composed using 9 single claps that I made during my journeys. If I were to take it further I’d elaborate on the rhythmic element, as well as working hard on the material to create more dynamic gestures and relationships between sounds.

The Why of Ray Bedingo

Me and Chris put ours heads together, as well as several pots and pans, and created all the sound material used in this organic offkey-techno groove!

Permalink Leave a Comment

Window Music Revisited

October 1, 2008 at 3:29 pm (Window Music - Prototype 1)

Window Music (part 2)

Whilst sitting by my window a couple of weeks ago my ears perked up to the exuberant sounds of a dancehall a capella being blasted down the street. It wasn’t coming from someone’s sound system though; the rambunctious vocal rhythms were leaping forth from the mouth of a local Jamaican man, marking his presence in the soundscape with his thick patois accent:

Permalink Leave a Comment