Synaesthesia

October 23, 2007 at 4:07 pm (MADA UNIT 1)

“Colour is the keyboard, the eyes are the hammers, the soul is the piano with many strings. The artist is the hand which plays, touching one key or another, to cause vibrations in the soul” (Kandinsky in “Concerning the Spiritual in Art”)

I’ve been pursuing my interest in synaesthesia, reading this book by John Harrison. My attraction to the subject stems from my own experiences with music and how it links in with my visual perception. In moments of deep immersion within a piece of music, I can start to ‘see’ it. That is, the life of a sound-object can take on a visual form; it’s evolution, gestural and dynamic qualities can become a reality in my imagination. This sensory translation intrigues me, but is it a true synaesthetic experience? What exactly is synaesthesia?

People who have synaesthesia respond differently to sensory information. For example, a synaesthete who has “colour hearing” may respond to certain spoken words (auditory information) by seeing a specific colour. According to case studies this seeing is not necessarily an image in their mind’s eye, but rather the feeling or sense of that image. True synaesthetes develop a strong idiosyncrasy with their stimulus trigger (e.g. the name “Lucy”) and their particular linked response (e.g. the colour “blue”). This link never changes and many synaesthetes describe having had these links as far back as they can remember.

Thus far the book puts forward a few theories on synaesthesia and why it occurs. Some of these have derived from electrocephalography (EEG): a scientific practice that measures activity in different regions of the brain using electrodes that are placed at certain points around the human head. Results from tests with neonates (infants under 3 months) showed that their brain activity is cross-wired. So for example, a visual stimulus may trigger activity in the part of the brain the deals with auditory information. Supposedly synaesthetes retain this cross-wired ability throughout their life, whilst the rest of us learn to make the ‘right’ connections.

Whilst it’s a very interesting condition, I think it would be difficult to base my project on something so subjective (scientifically speaking). However, the link between synaesthesia and musical composition is something I will pursue. I don’t believe I have synaesthesia (at least not to the extent that’s spoken of in this book), but I think there are ways to cultivate a kind of natural understanding of sound in visual terms. Of course this works the other way round and one can interpret an image sonically. This technique was used a lot in the interpretive ‘open works’ of Cornelius Cardew:

I’d like to incorporate these ideas into my work somehow.

Advertisements

Permalink Leave a Comment

Audio-Visual map of London

October 9, 2007 at 3:27 pm (MADA UNIT 1)

For my project I intend to create a series of audio-visual compositions based on footage I record around London. I wish to develop an interface that allows the audience can use to explore these compositions and navigate their own route through the work.

I have started looking at interfaces and have come across some interesting technologies, most notably the Multi-Touch developed by Jefferson Y. Han.

It seems quite intuitive and user friendly, allowing simple gestures and movements on the screen surface to control different parameters. Multi-touch sensor technology can register more than one contact point at a time, which means that you can program a lot more responses to a wider combination of user gestures.

Another touch-screen interface, which Tim found, was a lot more interesting:
http://www.k2.t.u-tokyo.ac.jp/members/alvaro/Khronos/

The Khronos Project allows the user to have temporal control over video clips, depending on the position and the amount of pressure the user applies to the lycra screen. It uses an infrared source reflected off the screen and picked up by a CCD camera (with an IR filter), to read any changes in the surface.

I like the idea of using this kind of technology, but I don’t want the interface to take over the work. Restricting the audiences experience to a screen may detract from the affect I want my final piece of work to have.
I want to create an immersive environment, an engaging experience of re-synthesized London.
Perhaps the interface could be hidden, using motion sensor technology.

Permalink Leave a Comment