Augmentation and Improvisation

Abstract

Preamble: Medium/Format/Marker Medium/Format/Marker (M/F/M) was a visual-aural improvisational performance involving myself, and musicians Joe Sorbara, and Ben Grossman. It was formed through my work as a PhD candidate at the Improvisation, Community, and Social Practice research initiative at the University of Guelph. This performance was conceived as an attempted intervention against the propensity to reify the “new.” It also sought to address the proliferation of the screen and question how the increased presence of screens in everyday life has augmented the way in which an audience is conceived and positioned. This conception is in direct conversation with my thesis, which is a practice-based research project exploring what the experimental combination of intermediality, improvisation, and the cinema might offer towards developing a reflexive approach to "new" media, screen culture, and expanded cinemas. One of the ways I chose to explore this area involved developing an interface that allowed an audio-visual ensemble to improvise with a film's audio-visual projection. I experimented with different VJ programs. These programs often utilize digital filters and effects to alter images through real-time mixing and layering, much like a DJ does with sound. I found a program developed by Chicago-based artist Ontologist called Ontoplayer, which he developed out of his practice as an improvisational video artist. The program works through a dual-channel interface where two separate digital files could be augmented, with their projected tempo capable of being determined by musicians through a MIDI interface. I conceptualized the performance around the possibility of networking myself with two other musicians via this interface. I approached percussionist Joe Sorbara and multi-instrumentalist Ben Grossman with the idea to use Ontoplayer as a means to improvise with Chris Marker's La Jetée (1962, 28 mins). The film itself would be projected simultaneously in four different formats: 16mm celluloid, VHS, Blu-ray, and Standard Definition video (the format the ensemble improvised with) projected onto four separate screens. From left to right, the first screen contained the projected version of La Jetée that we improvised with, next to it was its Blu-ray format, next to that, a degraded VHS copy of the film, and next to that, the 16mm print. The performance materialized through performing a number of improvisatory experiments. A last minute experiment conceived a few hours before the performance involved placing contact microphones overtop of the motor on a Bell & Howell 16mm projector. The projector was tested in the days leading up to the performance and it ran as smoothly as could be expected. It had a nice cacophonous hum that Ben Grossman intended to improvise with using some contact mics attached directly over the projector’s motor, a $5 iPad app, and his hurdy-gurdy. Fifteen minutes before the performance began, the three of us huddled to discuss how long we'd like to go. We had met briefly the day before to discuss the technical setup of the performance but not its execution and length. I hadn't considered duration. Joe broke the silence by asking if we'd be "finding beginnings and endings." I didn't know what that entailed, but nodded. We started. I turned on the projector and it immediately started to cough and chew on the 40 year old 16mm print I found online. My first impulse was to intervene, to try to save it. The film continued and I sat frozen for a moment. Joe started playing and Ben, expecting me to send him the audio track from La Jetée, prompted me to do so. I let the projector go and began. Joe had a digital kick-drum and two contact mics on his drum kit hooked into a MIDI hub, while Ben's hurdy-gurdy had a contact mic inside it, wired into the hub. The hub hooked into my laptop and allowed for an intermedial conversation to emerge between the three of us. While the 16mm, VHS, and Blu-Ray formats proceeded relatively unimpeded alongside each other on their respective screens, the fourth screen was where this conversation took place. I digitally reordered different image sequences from La Jetée. The fact that it’s a film (almost) comprised entirely of still images made this reordering intriguing in that I was able control the speed of progressing from each image to the next. The movement from image to image was structured between Ben and Joe’s improvisations and the kind of effects and filters I had initialized. Ontoplayer has a number of effects and filters that push the base image into more abstract territories (e.g.: geometric shapes, over pixelation) I was uninterested in exploring. I utilized effects that to some degree still kept the representational content of the image intact. The degree to which these effects took hold of the image were determined by whether or not Ben and Joe decided to use the part of their instrument that would trigger them. The decision to linger on an image, colour it differently, or skip ahead in the film’s real-time projection destabilized my sense of where I was in the film. It became an event in the sense that each movement, both visual and aural was happening with an indeterminate duration.  La Jetée opens with the narrator proclaiming: “this is the story of a man marked by an image from his childhood.” The story itself is situated around a man in a post-apocalyptic world, haunted by the persistent memory of a woman he saw as a child while standing on the jetty at Orly Airport in Paris. The man was a soldier, now captured, and imprisoned in an underground camp. The prison guards have been conducting experiments on the prisoners, attempting to use the prisoner’s memories as a mechanism to send them backwards and forwards in time. The narrator explains, “with the surface of the planet irradiated … The human race was doomed. Space was off limits. The only link with survival passed through time … The purpose of the experiments was to throw emissaries into time to call the past and future to the aid of the present.” La Jetée is visually structured as a photomontage, with voice-over narration, diegetic and non-diegetic sound existing as component parts to the whole film. I decided to separate these components for the sake of isolating them before the performance as instruments of the film to be improvisationally deployed through the intermedial connection between Ben, Joe, and myself.   The resulting projections that emerged from our interface became a kind of improvised "grooving" to La Jetée that restricted the impulse to discriminately place sound beneath and behind the image. I selected images from different points in the film that felt "timely" given the changing dynamic between the three of us. I remember lingering on an image of the woman's face, her hand against her mouth, her hair being blown back by the wind. I looked and listened for the moment when the film would catch and then catch fire. It never came. We let the reel run to the end and continued on improvising until we found an ending. But the sound of that film catching but never breaking, the intention and tension of the film being near death the entire time made everything we did more precious, teetering on the brink of failure. We could never have predicted that, and it gave us something I continue to ponder and be thankful for. Celluloid junkies in the room commented on how precipitous the whole thing was, given how rare it is to encounter the sound of celluloid film travelling through a projector inside a cinematic space. An audiophile mused over how there wasn’t any document, his mind adequately blown by how “funky” the projector sounded. With there being no document of the performance, I'm left with my own memories. In mining the aftermath of this performance, I hope to find an addendum that considers how improvisation might negotiate with augmentation in ways that speak to Walter Benjamin's assertion that the "camera, the film, on the one hand, extends our comprehension of the necessities which rule our lives; on the other hand, it manages to assure us of an immense and unexpected field of action” (Benjamin 236-7).Images to be Determined I got a job working in a photo lab eight years ago, right around the time digital cameras started becoming not only affordable, but technologically-comparable alternatives to film cameras. The photo printer in the lab was setup to scan and digitize celluloid filmstrips to allow for digital “touchups” by the technician. It was also hooked into touchscreen media stations that accepted a variety of memory card formats so that customers could “touchup” their own images. Celluloid film meant that as long as their format was chemical, touching up their images remained the task of the technician. Against the urging of the lab’s manager, I resisted altering other people’s images. It felt like a violation, despite the fact that almost every customer was unaware of this process. They assumed a degree of responsibility for a chemically-exposed image. I still got blamed for a lot of bad photography, but an image chemically under or overexposed was irreparable. Digital cameras changed all of that. I still preferred an evenly exposed celluloid print to a digital, but the allure was the ability for these images to be augmented. Augmentation is synonymous with "enhancement," "prosthesis," "addition," "amplification," "enrichment," "expansion,” and "extension" (to name a few). For the purpose of this essay, I am situating augmentation as an agential act engaging with a static form to purposefully alter its aesthetic and political relation to a reality. To what extent can we say that the digital image is itself, an augmentation? If Instagram is any indication, the digital image's existence is bound by its perpetual augmentation. A digital image is only as good as its capacity to be worked on. The ubiquity of digitally applying lomographic filters to digital images, as a defining step in their distributive chain, is indicative of the discursive impact remediating the old into the new has on digital forms. These digitally-coded filters used to augment “clear” digital images are comprised of exaggerated imperfections that existed to varying degrees, as unforeseen side effects of working with comparatively more unstable celluloid textures. The filtered images themselves are digital distortions of a digital original. The filters augment this original through obscuring one or a number of components. Some filters might exaggerate the green values or sharpen a particular quadrant within the frame that might coincide with the look of a particular film stock from the past. The discourse of “film” and “vintage” photography has become a synonymous component of the digital aesthetic, discursively warming up what is often considered to be a cold, and disembodied medium. Augmentation works to re-establish a congruous relationship between the filmic and the digital, attempting to reconcile the aesthetic distance between granularity and pixelation. This is ironic because this process is encapsulated through digitally encoding and applying these filters for the sake of obscuring clarity. Thus, the object is both hailed as clear and clearly manipulable. Another example a bit closer to the cinema is the development of digital video cameras offering RAW, or minimally compressed file formats for the sole purpose of augmenting the initial recording in post-production workflows in an attempt to minimize degradation in the image. The colour values and dynamic range of these images are muted, or flattened so that the human can control their elevation after the fact. To some degree the initial image, in itself, is an augmentation of its filmic relative. From early experiments with video synthesizers to the present digital coding of film effects, digital images have tantalized video artists and filmmakers with possibility shrouded in instantaneity and malleability. A key problem with this structure remains the unbridled proliferation and expansion of the digital image, set free for the sake of newness. How might improvisation work towards establishing an ethics of augmentation? An ethics of this kind must disrupt the popular notion of the digital image existing beyond analogical constraints. The belief that “if you can imagine it, you can do it” obfuscates the reality that to work with images, whatever their texture, is a negotiation with constraint. Part of M/F/M’s fruition emerged from a conversation I'd had with Canadian Animator Pierre Hébert last summer. Now obvious, but for Hébert, the first obstacle he needed to overcome as an improviser was developing an instrument that he could gig with. Through the act of designing an instrument I immediately became aware of what wasn't possible, and so the work leading up to the performance involved attempting to expand the possibilities of that instrument. How might I conceive of my own treatment of images simultaneously treated by Joe and Ben as a kind of cinematic extended technique we collaboratively bring into being? Constraint necessitates the need for extension, for finding new ways to sound and appear. Constraint is also consistently conceived as shackling progress. In scientific methodologies it is often arbitrarily imposed to steer an experiment into a desired direction. This sort of experimental methodology is in the business of presupposing outcomes, which I feel is often the case with what ultimately becomes the essay of end result in Humanities research. Constraint is an important imposition in improvisation only if the parties involved are willing to find new ways to move in consort with it. The act of improvisation is thus an engagement with the spatio-temporal constraints of performance, politics, memory, texture, and difference. My conception of the cinema is that of an instrument, whose past is what I work with to better understand its future. Critic Gene Youngblood, in his landmark book, Expanded Cinema, theorized a new conception of the cinema as a global planetary phenomenon suffused inside a space of intermedia, where immersive, interactive, and interconnected realms necessitated the need to critically conceptualise the cinema in cosmic terms. At around the time of Youngblood's writing, another practitioner of the cosmic way, improviser and composer Sun Ra was staking a similar claim for music's ability to uplift the species cosmically. Ra's popular line “If we came from nowhere here, why can’t we go somewhere there?” (Heble 125), articulated the problematic racial politics in post-WWII America, that fixed African-American identity into a static domain with little room to move upward. The "somewhere there" to Ra was a non-space, created from "a desire to opt out of the very codes of representation and intelligibility, the very frameworks of interpretation and assumption which have legitimated the workings of dominant culture" (Heble 125). Though Youngblood's and Ra's intellectual and creative impulses formed from differing political circumstances, the work and thinking of these two figures remain significant articulations of the need to work from and towards the cosmic. In 2003, Youngblood published a follow-up essay in a reprint of Expanded Cinema entitled Cinema and the Code. In it, he defines cinema as a “phenomenology of the moving image.” Rather than conceiving of it through any of its particular media, Youngblood advocates for a segregated conception of the cinema: Just as we separate music from its instruments. Cinema is the art of organizing a stream of audiovisual events in time. It is an event-stream, like music. There are at least four media through which we can practice cinema – film, video, holography, and structured digital code—just as there are many instruments through which we can practice music. (Youngblood cited in Marchessault and Lord 7) Music and cinema are thus conceived as the exterior consequences of creative and co-creative instrumental experimentation. For Ra and Youngblood, the planetary stakes of this project are infused with the need to manufacture and occupy an imaginative space (if only for a moment) outside of the known. This is not to say that the action itself is transcendental. But rather this outside is the planetary. For the past year I've been making a documentary with Joe Sorbara on the free improv scene in Toronto. Listening to musicians talk about improvisation in expansive terms, as this ethereal and ephemeral experience, that exists on the brink of failure, that is as much an act of memory as renewal, reverberated with my own feelings surrounding the cinema. Improvisation, to philosopher Gary Peters, is the "entwinement of preservation and destruction", that "invites us to make a transition from a closed conception of the past to one that re-thinks it as an endlessly ongoing event or occurrence whereby tradition is re-originated (Benjamin) or re-opened (Heidegger)” (Peters 2). This “entwinement of preservation and destruction” takes me back to my earlier discussion of the ways in which digital photography, in particular lomographically filtered snapshots, is structured through preserving the discursive past of film while destroying its standard. The performance of M/F/M attempted to connect the augmentation of the digital image and the impact this augmentation had on conceptualizing the past through an improvisational approach to intermediality. The issue I have with the determination of images concerns their technological standardization. As long as manufacturers and technicians control this process then the practice of gathering, projecting, and experiencing digital images is predetermined by their commercial obligation. It assures that augmenting the “immense and unexpected field of action” comprising the domain of images is itself a predetermination. References Benjamin, Walter. Illuminations. New York: Schocken Books, 1985.  Heble, Ajay. Landing on the Wrong Note. London: Routledge, 2000. Marker, Chris, dir. La Jetée. Argos Films. 1962. Marchessault, Janine, and Susan Lord. Fluid Screens, Expanded Cinema. Toronto: University of Toronto Press, 2007. Peters, Gary. The Philosophy of Improvisation. Chicago: University of Chicago Press, 2009.