Composing Portals
Portals was composed as a video score designed to elicit a fully prepared improvised musical response based solely on the information in the score. The video score would be projected to an audience, creating a real-time audiovisual piece.
The initial concept was a partial response to Pancake’s (2011) use of abstract imagery in video scores which required performers to create spontaneous improvisations in the performance space. The composition of Portals also drew on Cardew’s Treatise which was itself designed ‘as a consistent system of symbolic elements that could be used to inform a performance of agreed-upon meanings for each symbol’ (Anderson, 2006, p. 1). My concept was further reinforced upon hearing Gaffney (2015) discuss her historical encounter with Richier’s L’Eau in the symposium Thought Positions In Sculpture:
The initial concept was a partial response to Pancake’s (2011) use of abstract imagery in video scores which required performers to create spontaneous improvisations in the performance space. The composition of Portals also drew on Cardew’s Treatise which was itself designed ‘as a consistent system of symbolic elements that could be used to inform a performance of agreed-upon meanings for each symbol’ (Anderson, 2006, p. 1). My concept was further reinforced upon hearing Gaffney (2015) discuss her historical encounter with Richier’s L’Eau in the symposium Thought Positions In Sculpture:
‘Through the form of the work I encounter woman-ness which is my own recognition of living in a female body. The highly resolved areas of tightly modelled clay around the sternum, shoulder and breast areas of the body effect an experience of embodied woman, which holds in material form what I know’ (Gaffney, 2015, p. 2).
I aimed to compose a video score which would guide the intuition of a performer, revealing their interpretations and experience of the video score on the instrument. My intention was to use bodily motion in an attempt to realise sounds which may not be easily communicated to a performer through a conventional gridded system of notation (Wishart, 1994; Wishart, 1996). Unfortunately, upon reflection I found that my approach to the visual stimulus led to a literal duplication of the movements depicted on screen rather than eliciting a search to find possible sonic interpretations through the instrument. My decision to use footage depicting violin performances inevitably provided rich and detailed pitch and rhythmic information which resulted in a literal translation of the visual image and a piece which sounded as though composed using a gridded system of notation (Wishart, 1994; Wishart, 1996).
Building on Previous Experiences
Questioner, 1863 was my first attempt at composing a video score and informed a great deal of my approach towards the intention for Portals. Unfortunately, Questioner, 1863 did not use footage with Creative Commons licenses and cannot be included. However, it was performed and recorded in a workshop by Ruffer in 2015 at the University of Huddersfield.
Composing the piece for Ruffer’s workshop was very much a daunting leap into the unknown, encouraged by my teacher, which was no doubt a significant, crucial and defining episode. One highly informative point of the workshop was Ruffer’s tumultuous interpretation of a close-up shot from a particularly frenetic Jethro Tull performance. However, Ruffer commented that she recognised several musicians in the score and made attempts to replicate their style in her performance (Ruffer, personal communication, March 3, 2015), which I considered to be problematic and informed my decision to remove any images of the performer from the video when composing the score for Portals.
Composing the piece for Ruffer’s workshop was very much a daunting leap into the unknown, encouraged by my teacher, which was no doubt a significant, crucial and defining episode. One highly informative point of the workshop was Ruffer’s tumultuous interpretation of a close-up shot from a particularly frenetic Jethro Tull performance. However, Ruffer commented that she recognised several musicians in the score and made attempts to replicate their style in her performance (Ruffer, personal communication, March 3, 2015), which I considered to be problematic and informed my decision to remove any images of the performer from the video when composing the score for Portals.
Influences
The composition of Portals called on a network of nodes and influencing forces (Adkins & d’Escriván, 2013; Adkins, 2014), from Duchamp’s development of the readymade, Schwitters’ Merz, Oswald’s Plunderphonics, Kreidler's Compression Sound Art and remix culture, manifesting in the use of found footage. All video files were acquired from sources with an appropriate Creative Commons license: CC BY, with links and more information available in the performance notes. The visual characteristic of the score was reached through a need to remove each image from its original context by appropriating only that which was required to transmit the desired message, an approach informed by the work of Vostell’s television décollage and Paolozzi's collage-based silkscreen prints. While the treatment of image was influenced by visual art, composers working with graphic scores such as Cage, Cardew, Brown and particularly Wolff, who ‘places the pianist and the instrument right at the heart of the work itself’ (Thomas, 2008), provided musical contexts for my thinking.
Initial Stages of Composition
Finding suitable footage and removing unwanted images was the first stage in composition, achieved using video editing software. Unwanted elements of the image were first removed through a form of digital décollage and collage, with several clips being consolidated into a new video file. A circular mask allowed the entirety of the left hand and wrist to be visible, while an oval mask preserved the vertical movements of the bowing, with both being presented as positive spaces set within a black negative space. Two reflective reports, written after the completion of the composition and following the performance, explain post-production effects in more detail.
An underpinning premise for Portals was to engage with the performer and audience’s ability to retain and recall audiovisual information within a temporal perspective (Snyder, 2001). Composing with an economy of material would increase conscious awareness by reducing memory load (Snyder, 2001): by selecting only close-up and extreme close-up shots it was possible to reduce the unmodified video file significantly. The remaining chunks were refined through cropping before being arranged into short phrases, as can be seen in videos 1-12 of the drafts and experiments playlist. This approach helped me to develop a familiarity with the footage which aided the composition of phrases, particularly a 30-second bowing cell which underpinned much of the piece. Looping of the bowing cell in the final score acted as an amplifier in bringing attention to shorter clips of left hand footage (Thompson, 1979; Glover & Harrison, 2013). This was intended to further reduce memory load and allow the performer to make connections between left and right hand materials more effectively (Snyder, 2001).
I arranged cells and phrases by observing and aligning the movements depicted in the images without any concept of an ideal or desired sound. I felt that to do otherwise would have been inimical to the premise of the piece. The audio tracks were removed from each video file before commencing with deconstruction, editing and reconstruction. My initial approach to the composition was to simply push and pull the short clips around using software, attempting to make connections between them. Some outcomes of this approach can be seen in videos 13-16.
An underpinning premise for Portals was to engage with the performer and audience’s ability to retain and recall audiovisual information within a temporal perspective (Snyder, 2001). Composing with an economy of material would increase conscious awareness by reducing memory load (Snyder, 2001): by selecting only close-up and extreme close-up shots it was possible to reduce the unmodified video file significantly. The remaining chunks were refined through cropping before being arranged into short phrases, as can be seen in videos 1-12 of the drafts and experiments playlist. This approach helped me to develop a familiarity with the footage which aided the composition of phrases, particularly a 30-second bowing cell which underpinned much of the piece. Looping of the bowing cell in the final score acted as an amplifier in bringing attention to shorter clips of left hand footage (Thompson, 1979; Glover & Harrison, 2013). This was intended to further reduce memory load and allow the performer to make connections between left and right hand materials more effectively (Snyder, 2001).
I arranged cells and phrases by observing and aligning the movements depicted in the images without any concept of an ideal or desired sound. I felt that to do otherwise would have been inimical to the premise of the piece. The audio tracks were removed from each video file before commencing with deconstruction, editing and reconstruction. My initial approach to the composition was to simply push and pull the short clips around using software, attempting to make connections between them. Some outcomes of this approach can be seen in videos 13-16.
Use of Screen Space
Rather than creating static portals operating from fixed points within a negative space, movement around the screen space generated parameters for performer interpretation. A reduction of explicit pitch information within the image was intended to guide the performer to develop their own conclusions regarding pitch but also promote engagement with the motion of the image within the overall space. The instructions for interpretation provided a degree of direction but connected the horizontal movement of the bowing portal as relative to the position of the bow on the string. Several factors including vibrato and changes to bow pressure were suggested to be attributable to the size and location of the left hand portal.
Working with Structure
An outline of three large sections began to emerge through the process of editing out unwanted parts of the image and arranging clips based on the visual content, rather than by thrusting a conscious consideration of form upon the materials (Ingold, 2010). Recognising that form was beginning to emerge I drew influence from Cage's First Construction (In Metal) as I was aware that his use of the square root or micro-macrocosmic form had been solidified in the composition (Guessford, 2004). Further influence came from a compositional strategy often employed by 8-bit video game composers as a response to the constraints on system memory in which larger structures were constructed by looping short phrases (Collins, 2007; Fox, 2016). I did not want to determine any musical parameters using Cage’s approach as I was confident with the content of the clips. However, acknowledging accumulative and micro-macrocosmic forms as potential methods for establishing form jolted my imagination towards a method of organising the bowing cells. Decisions relating to the frequency, spacing and order of the bowing cell were reached through intuition, experimentation and constant re-drafting rather than being governed by a system of ratios (Guessford, 2004).
Employing a ternary form was intended to advertise the contrasts between the visual content of the clips for the left hand: section A (00:10 - 02:48) featured a small series of distinct stationary positions within a restricted range on the fingerboard; section B (02:58 - 05:06) only used footage of glissandi articulation over a wider range of the neck; and section C (05:16 - 07:46) combined stationary and glissandi articulations.
Creating subsections enabled a more successful arrangement of discrete phrases featuring left hand footage by specifying duration, providing cues and a classification of visual information. Subsections were labelled as A1 (00:10 - 01:28); A2 (01:30 - 02:48); B1 (02:58 - 03:45); B2 (03:47 - 05:07); C1 (05:17 - 06:01); C2 (06:03 - 06:33); C3 (06:36 - 07:07); and C4 (07:09 - 07:46). Each subsection acted as a container for a specific type of image: A1, B1, C1 and C2 contained still images while A2, B2, C3 and C4 contained moving images.
The diagram below may provide a visual example of the overall form and structure of Portals. The large sections (A,B and C) are coloured green while still and moving images, which make up the subsections, are coloured blue and red respectively. Subsections C1-4 are coloured purple to represent the amalgamation of still and moving images.
Employing a ternary form was intended to advertise the contrasts between the visual content of the clips for the left hand: section A (00:10 - 02:48) featured a small series of distinct stationary positions within a restricted range on the fingerboard; section B (02:58 - 05:06) only used footage of glissandi articulation over a wider range of the neck; and section C (05:16 - 07:46) combined stationary and glissandi articulations.
Creating subsections enabled a more successful arrangement of discrete phrases featuring left hand footage by specifying duration, providing cues and a classification of visual information. Subsections were labelled as A1 (00:10 - 01:28); A2 (01:30 - 02:48); B1 (02:58 - 03:45); B2 (03:47 - 05:07); C1 (05:17 - 06:01); C2 (06:03 - 06:33); C3 (06:36 - 07:07); and C4 (07:09 - 07:46). Each subsection acted as a container for a specific type of image: A1, B1, C1 and C2 contained still images while A2, B2, C3 and C4 contained moving images.
The diagram below may provide a visual example of the overall form and structure of Portals. The large sections (A,B and C) are coloured green while still and moving images, which make up the subsections, are coloured blue and red respectively. Subsections C1-4 are coloured purple to represent the amalgamation of still and moving images.
Structural Diagram of Portals
An understanding of the mechanics and physical properties of the violin was essential when arranging the clips of both hands to ensure that changes in left hand position would be in sync with a bowing direction (Adler, 2002).
Possible Approaches to the Score
Whiten’s (2000) study into primate culture and social learning indicates that as individuals who develop our cognitive abilities within a society, we are guided by strong cultural codes and practices which are subsequently imprinted upon us by this society and the various practices within it. It can be argued that we learn partly through observing and repeating the successful behaviour exhibited by others, and that we develop our own sense of successful practice which can further be recognised in the practices and behaviours exhibited by other individuals. Portals was an attempt to apply learned and codified instrumental frameworks to a piece of music by calling on a performer’s experience of the instrument and knowledge a posteriori through representative images.
Portals presented information to the performer through static and moving images which depicted conventional and idiomatic hand and bowing movements, albeit somewhat disguised and altered. This information was revealed gradually through a temporal perspective which required the performer to analyse, memorise, recall previous events and predict repetition (Snyder, 2001). The gradual revealing of information called on the performer's memory as well as analytical and creative skills which were underpinned by their detailed knowledge of the instrument (Merleau-Ponty, 1962; Krathwohl, 2002). It may therefore be appropriate to suggest that an improvisatory form of exploration could be employed when approaching the score (Smith & Dean, 1997; Griffiths, 2010).
The principle that a mediated physical gesture would be interpreted and recoded into sonic musical gestures was reached by considering the function of conventional systems of notation within the Western musical tradition. The five-line staff has become an almost universally accepted codified system which guides performers via their experience of working with the notational system of predetermined rules and principles from which pedagogies, performance traditions and issues relating to physical enactment of the notation have developed, including musical gesture (Godlovitch, 1998). Moving images, taken from video files which originally documented live performances, were re-appropriated to form a score intended to create a new performance. Therefore, live performances of Portals produce both physical and musical gestures (Argyle, 1988; Leman & Godøy, 2010).
Portals presented information to the performer through static and moving images which depicted conventional and idiomatic hand and bowing movements, albeit somewhat disguised and altered. This information was revealed gradually through a temporal perspective which required the performer to analyse, memorise, recall previous events and predict repetition (Snyder, 2001). The gradual revealing of information called on the performer's memory as well as analytical and creative skills which were underpinned by their detailed knowledge of the instrument (Merleau-Ponty, 1962; Krathwohl, 2002). It may therefore be appropriate to suggest that an improvisatory form of exploration could be employed when approaching the score (Smith & Dean, 1997; Griffiths, 2010).
The principle that a mediated physical gesture would be interpreted and recoded into sonic musical gestures was reached by considering the function of conventional systems of notation within the Western musical tradition. The five-line staff has become an almost universally accepted codified system which guides performers via their experience of working with the notational system of predetermined rules and principles from which pedagogies, performance traditions and issues relating to physical enactment of the notation have developed, including musical gesture (Godlovitch, 1998). Moving images, taken from video files which originally documented live performances, were re-appropriated to form a score intended to create a new performance. Therefore, live performances of Portals produce both physical and musical gestures (Argyle, 1988; Leman & Godøy, 2010).
Critical Engagement
Kanno’s skilled, prepared and dedicated performance of Portals not only provided great satisfaction through a talented professional interpreting my work but also enabled further discussion and understanding of the composition of video scores.
One major concern surrounding the performance of the piece was the use of a transcription during performance. Although this allowed Kanno to explore ideas for rehearsal, the use of the transcription during performance unfortunately nullified several aspects of the intended aesthetic. As previously stated, an ideal performance would have fully engaged with the temporality of video and the retentive and protentive capabilities of the human performer and audience (Snyder, 2001; Kelly, 2016). For example, performing Wolff's Duo for Pianists II demands that performers be constantly operating on the edges of action and restraint (Thomas, 2010). In the case of Portals, the reactions of the human performer and their attempts to keep up with, recall and predict the next sequence in an unstoppable and unyielding stream of information were significantly altered by the use of a transcription. During rehearsal it became clear that Kanno attempted to create a faithful response to the motions she had witnessed in the video performer’s skilled and precise bowing technique (Kanno, personal communication, January 20, 2016). Due to the level of detail in the images the movement of the portals around the screen space were largely unused except for the changes to size of the left hand portal being attributed to degrees of vibrato: the larger the portal, the wider the vibrato as can be heard at 00:45-54.
Kanno’s performance revealed that the explicit rhythmic and pitch information presented by the score was too strong to yield performances based on anything other than a reproduction of the images within a literal and finite set of parameters, rather than encouraging an internal discussion of potential realisations (Mooney, 2011). Furthermore, due to the use of a music stand and transcription during performance, the desired connections between the score and Kanno’s interpretation were rendered unavailable to the audience: it may have been unclear that the video was the score. The use of a transcription in live performance may have skewed audience perception of the audiovisual contract (Chion, 1994) closer towards a cinematic context where sound is typically subservient to image, usually supporting visual cues and narrative (Maloney, 2005; Fox, 2016; Westgeest, 2016). It could be further argued that the projection of the video score diminished the performer’s role while exacerbating the issue of a hierarchy between visual and auditory forms (Prévost, 1995; Kahn, 2001).
The literal interpretation of the score points to it being too prescriptive, able to ‘be compared to a kind of instruction manual’ (Kanno, 2007, p. 235). It may be possible to attribute the literal interpretation to the limited range of instrumental information in the score affording ease of access through Mooney’s (2011) frameworks and affordances model. Engaging with Mooney’s (2011) spectrum after the fact of composition showed that affordances at the bottom range may only sufficiently engage a performer in the task of recalling the sequences of events, which in the live performance of Portals was rescinded by the use of a transcription. A low resistance in the score created a scenario in which literal movements became the currency of the piece, resulting in music with a clear grid-like approach to pitch and rhythm. Further obfuscation of pitch and rhythmic information may work to de-emphasise the grid: showing only the back of the fingerboard or removing the left hand entirely. Bowing could even be represented by a dot or line, indicating bowing direction and rhythm without suggesting string interaction.
Baldwin’s ]HoldingOn[ actively engages the performer in the production of the score, placing them at the centre of the piece. Baldwin’s interest in the performing body and performance space provided an interesting perspective on video which may produce a platform from which a performer can ‘interpret the score rather than solely reproduce the score’ (Baldwin, 2016 More Precise - Points of Contact, and Abstraction in Video Scores section, para. 4): something which Portals failed to accomplish.
By not providing the performer with enough resistance and making the information too obvious, Portals resulted not in an open score as initially intended, but a rather closed and complete one (Eco, 2004). Shlomowitz’s Letter Pieces exemplify the organisation of material within an open score. Creating a piece in which the performer were able to attribute sounds and actions within a determined structure may engender space for investigation and introspection. Interpretations may differ widely between performances of the work, potentially engaging with and exposing the performer’s knowledge a priori. Considering similar approaches towards composing video scores may also enable a more interesting use of the temporal medium through play speed and variations in metre, including arresting or advancing the frequency of images.
Despite my criticisms of Portals in relation to an improvised audiovisual piece for live performance, there may be a positive direction for the approach. Considering Whiten’s (2000) research in which observation and copying are seen as central to cognitive and social development, literal information may be useful in composing pieces for people with little or no prior musical experience, such as young children. Composing videos depicting movements on an instrument may even be tailored to provide instruction on posture and technique. The internet has provided a platform for tutors and musicians to disseminate instrumental tuition videos for several years.
One major concern surrounding the performance of the piece was the use of a transcription during performance. Although this allowed Kanno to explore ideas for rehearsal, the use of the transcription during performance unfortunately nullified several aspects of the intended aesthetic. As previously stated, an ideal performance would have fully engaged with the temporality of video and the retentive and protentive capabilities of the human performer and audience (Snyder, 2001; Kelly, 2016). For example, performing Wolff's Duo for Pianists II demands that performers be constantly operating on the edges of action and restraint (Thomas, 2010). In the case of Portals, the reactions of the human performer and their attempts to keep up with, recall and predict the next sequence in an unstoppable and unyielding stream of information were significantly altered by the use of a transcription. During rehearsal it became clear that Kanno attempted to create a faithful response to the motions she had witnessed in the video performer’s skilled and precise bowing technique (Kanno, personal communication, January 20, 2016). Due to the level of detail in the images the movement of the portals around the screen space were largely unused except for the changes to size of the left hand portal being attributed to degrees of vibrato: the larger the portal, the wider the vibrato as can be heard at 00:45-54.
Kanno’s performance revealed that the explicit rhythmic and pitch information presented by the score was too strong to yield performances based on anything other than a reproduction of the images within a literal and finite set of parameters, rather than encouraging an internal discussion of potential realisations (Mooney, 2011). Furthermore, due to the use of a music stand and transcription during performance, the desired connections between the score and Kanno’s interpretation were rendered unavailable to the audience: it may have been unclear that the video was the score. The use of a transcription in live performance may have skewed audience perception of the audiovisual contract (Chion, 1994) closer towards a cinematic context where sound is typically subservient to image, usually supporting visual cues and narrative (Maloney, 2005; Fox, 2016; Westgeest, 2016). It could be further argued that the projection of the video score diminished the performer’s role while exacerbating the issue of a hierarchy between visual and auditory forms (Prévost, 1995; Kahn, 2001).
The literal interpretation of the score points to it being too prescriptive, able to ‘be compared to a kind of instruction manual’ (Kanno, 2007, p. 235). It may be possible to attribute the literal interpretation to the limited range of instrumental information in the score affording ease of access through Mooney’s (2011) frameworks and affordances model. Engaging with Mooney’s (2011) spectrum after the fact of composition showed that affordances at the bottom range may only sufficiently engage a performer in the task of recalling the sequences of events, which in the live performance of Portals was rescinded by the use of a transcription. A low resistance in the score created a scenario in which literal movements became the currency of the piece, resulting in music with a clear grid-like approach to pitch and rhythm. Further obfuscation of pitch and rhythmic information may work to de-emphasise the grid: showing only the back of the fingerboard or removing the left hand entirely. Bowing could even be represented by a dot or line, indicating bowing direction and rhythm without suggesting string interaction.
Baldwin’s ]HoldingOn[ actively engages the performer in the production of the score, placing them at the centre of the piece. Baldwin’s interest in the performing body and performance space provided an interesting perspective on video which may produce a platform from which a performer can ‘interpret the score rather than solely reproduce the score’ (Baldwin, 2016 More Precise - Points of Contact, and Abstraction in Video Scores section, para. 4): something which Portals failed to accomplish.
By not providing the performer with enough resistance and making the information too obvious, Portals resulted not in an open score as initially intended, but a rather closed and complete one (Eco, 2004). Shlomowitz’s Letter Pieces exemplify the organisation of material within an open score. Creating a piece in which the performer were able to attribute sounds and actions within a determined structure may engender space for investigation and introspection. Interpretations may differ widely between performances of the work, potentially engaging with and exposing the performer’s knowledge a priori. Considering similar approaches towards composing video scores may also enable a more interesting use of the temporal medium through play speed and variations in metre, including arresting or advancing the frequency of images.
Despite my criticisms of Portals in relation to an improvised audiovisual piece for live performance, there may be a positive direction for the approach. Considering Whiten’s (2000) research in which observation and copying are seen as central to cognitive and social development, literal information may be useful in composing pieces for people with little or no prior musical experience, such as young children. Composing videos depicting movements on an instrument may even be tailored to provide instruction on posture and technique. The internet has provided a platform for tutors and musicians to disseminate instrumental tuition videos for several years.