Digital Transformations

The Media Is the Mix

1In the age of digital transformations of images, communications, and storytelling, Marshall McLuhan's insight that "the medium is the message" can be augmented with the corollary that the media is the mix. Digital forms of narrative are not only characterized by their mixed, hybrid forms and content, but their recombinations 1 draw the spectator into the mix in unforeseen ways. By mixing varying degrees of non-linearity and interactivity in what are ultimately animations, digital narratives create new kinds of digital spectatorship.

2The examples I'll explore here are three films, Conceiving Ada, Gamer, and Time Code. In different yet interconnected ways, each privileges the mix of media over content, or rather, foregrounds the mix as content. These digital narratives divert the reader/spectator/ participant from the traditional ways of making meaning--or at least sense--of narrative.

3One way to illuminate the examples is to explore how they mix linearity and interactivity. In teaching The Theory and Practice of Digital Narrative, my students and I developed a model for analyzing how different works create new modes of storytelling, and fresh relations of looking at and within the frame.2 Extending some of the terms that Janet Murray develops in Hamlet on the Holodeck: The Future of Narrative in Cyberspace, the diagram posits an x axis of linearity and a y axis of agency.3

Click here for an animated version of this graph.

4The linearity axis spans narratives from most linear (one plot line that progresses chronologically through time and space to one outcome through a series of cause and effect occurrences represented from a singular point of view) to non-linear (narratives that could be circular, rhizome-shaped, achronological, synchronic, multiple plotlines, multiple points of view, multidimensional).4 The agency axis spans media that call for very little active participation from the reader/spectator/listener to media that demands a high level of interaction. Of course, all "reading" is in some way active, both in the physical acts of seeing and turning pages/clicking a mouse 5 and in comprehending, imagining, remembering, and making meaning. Nevertheless, there is a distinction between page-turning and making choices in a hypermedia work, and Murray uses the term agency to suggest an "active creation of belief" (as opposed to Coleridge's "willing suspension of belief"). Digital environments require that belief be created and reinforced; as William Gibson posited in Neuromancer, the imagined place of cyberspace is a "consensual hallucination," a social agreement to act as if the places in cyberspace exist. The willing creation of belief is a social agreement that relies on a like-minded community. The more self-reflexive and unconventional the narrative, the more it calls for the will to believe.

5These examples of digital cinema "interpellate," or hail, their spectators as willing agents in the common project of the creation of belief. The subjectivity that these films seek to create for their viewers is one of being an active, technologically-savvy spectator. Instead of encouraging participation in, to use Guy Debord's phrase, the society of the spectacle, these digital narratives perform a Brechtian function in a distinctly technological manner that derives from the mix of media that is digital cinema. The term "digital cinema" has acquired many meanings, ranging from movies shot on digital video in a manner we associate with film (i.e.: single camera) to digital exhibition of media in digitally-equipped movie theaters and streaming video on the web. Lev Manovich defines digital cinema as "a particular case of animation which uses live action footage as one of its many elements."6 Manovich's historical argument suggests commonalities between the earliest moving images and current developments in digital media; animation was marginalized in the development of cinema, and although some of its techniques were adopted by the avant garde, only now animation returns at the very center of digital cinema.7

6Lynn Hershman Leeson's film Conceiving Ada explores digital media in both form and content. In the film Emmy, a contemporary computer software engineer, uses technology to make contact with Ada Lovelace, who invented the first computer language in Victorian England. The narrative moves between Emmy in the present day and Ada in the past as Emmy figures out how to send a software agent into the past to retrieve information. Although the plot doesn't really make sense scientifically, it resonates emotionally; despite the 150 years separating their lives, Emmy and Ada face some of the same issues as women working with technology.

7Instead of building sets, Leeson developed a technique of blending live action footage shot against a blue screen with digitized photographs of Victorian inns. As she explains in the technical notes of the DVD of the film (and also on the Conceiving Ada website):

8I felt it important to use the technology Ada pioneered. Virtual sets and digital sound . . . provided environments in which she moves freely through time, becomes liberated and, ultimately, moves into visibility.

9The actors and filmmaker collaborated in what amounts to a consensual hallucination:

10On the set, these images were maneuvered through several computers where mattes were added and images were put into perspective or enlarged. They were then laid onto digital videotape while the actors were performing. Actors could reference their location through a monitor that showed them their "virtual" environment. . . . The immediacy of shooting live action while simultaneously manipulating digitized backgrounds in real time was, remarkably, exhilarating.

11By mixing past and present, fact and fiction, personal and professional, digital and analog, live action and animation, Conceiving Ada tells its powerful story in both form and content. Although it is not immediately obvious that the sets are virtual rather than actual, much of the story takes place in front of and inside Emmy's computer equipment. There are many shots of Emmy gazing into the computer, trying to make her programs work, and when she does make contact with Ada, she can see her memories and talk with her on the computer screen. The film frame often encompasses the computer screen, so we too see the graphic interfaces Emmy designs and animates. By mixing some of the techniques of video art with film style and the malleability of the digital image, Leeson extends her trailblazing career in many directions at once in a work that is a mix about mix. A representation of intense digital interactivity, Conceiving Ada's heroines, creators, and spectators use technology, and specifically digital imaging technology, to create agency.

12Like Conceiving Ada, Time Code is also an example of digital cinema that calls attention to the digital techniques used in its production, and enlists its spectators as accomplices in creating the narrative. Time Code is itself the product of improvisation, and uses digital technology to capture "real time." Director Mike Figgis divides the screen into four frames; each quadrant contains a continuous 90-minute take of an unscripted, improvised performance, all shot simultaneously with four Sony Dvcam DSR-130s; the film was blocked out on music paper. Figgis describes his movie as a "black comedy about a 90-minute slice of life in Hollywood," and it takes filmmaking as one of its subjects along with jealousy, infidelity, and a fin-de-siecle philosophical and artistic exhaustion. The audio mix of the theatrical release of the film shifts its emphasis between the quadrants, thus directing the spectator's attention to a certain quadrant. Choosing to focus on a quadrant is a kind of spectator-editing. Looking at the entire frame means seeing a new kind of animation, created by multiple screens, encompassing multiple points of view. (See Time Code clip here.)

13The film folds in on itself self-reflexively. The plot centers around the intersecting lives of four characters involved personally and/or professionally with Red Mullet, Inc., a movie studio (which is the real-life name of Figgis's production company; see, and becomes increasingly Brechtian as the movie builds to the climactic scene in which a hot young independent filmmaker pitches a movie that, like the one we are watching, splits the screen and follows the interactions between four characters. In what could be taken as a manifesto for digital cinema that counters the "chastity" of DOGME 95 with the passionate embrace of technology, the filmmaker announces, "Montage has created a fake reality. . . . Technology has arrived, digital video has arrived, and is demanding new expressions, new sensations. . . . It's time to say again: Art, Technology: a new union." She shows the studio executives storyboards of how four cameras will follow four characters, who are really four aspects of the same character at different points in their lives. But the filmmaker's passionate and theoretical speech is undercut by the context of black comedy that infuses the film: artistic praxis clashes with business practice, and all the plotlines peak as Stellan Skarsgard's Alex bursts into laughter and exclaims, "This is the most pretentious crap I've ever heard. . . . Do you think anybody around this table has a clue about what you're talking about?"

14Figgis assumes his audience does, that Bauhaus, Soviet montage, and Guy Debord are not as foreign to his spectators as they are to the executives; the filmmakers both within and outside the frames show their theoretical orientation.

15Time Code is the first major work of digital cinema. It creates a new kind of animation based on subjectivity and point of view, and calls for an active creation of belief from the spectator-editor who takes in and ultimately creates the narrative of the film. The DVD extends the spectator's agency even further. Among its special features is a documentary about how the film was shot 15 times, all four cameras operating simultaneously around the actors' improvisation. Figgis chose the fifteenth version for the theatrical release, but also includes the first version on the DVD. Because the making of the film, and how it uses digital technology, is so central to the spectator's experience, the director's commentary and interviews with the cast reinforce the spectator as an active creator of belief and meaning in the film.

16Moreover, by including a special audio mixing feature, the DVD gives the medium a new level of interactivity. Using the remote control of a DVD player, the spectator/participant can switch between the audio of the different quadrants. Because the audio is a major aspect of directing the spectator's attention (in addition to visual elements such as movement and stasis), being able to make choices in the audio mix is, to use the music metaphor that the film is based on conceptually, to become the conductor of the film.8 The medium is the mix.

17Like Time Code and Conceiving Ada, the new French film Gamer is also a particularly digital mix. Gamer moves between live action and digital environments as its main character Tony gets the idea for, designs, and then is swindled out of a computer game. The first time the film environment switches to the game environment is when the hero is in a car chase and his car morphs into a game graphic of a car. The shift between a reality created by conventional film style and the unconventional use of game graphics style reveals the main character's subjectivity, for his reality (and other characters' as well) are constructed through their interaction in the playing and making of games.

18Gamer is aesthetically and viscerally ambitious in the range of live action and computer graphic interfaces it moves between. The adrenalized state of game-playing mixes with a fictionalized account of game design. Gamer succeeds in creating an immersive text about two differently immersive mediums, film and computer games. When the film depicts how live action can be made digital, and both shows and denies the indexicality of the digital image, it explores the nature of digital cinema in a way that is complementary to the projects of Conceiving Ada and Time Code.


19In addition to fostering new relations of looking, these digital narratives make forays into nonlinearity. Conceiving Ada and Gamer both have discursive plots that revise the conventions of the linear plot, moving between nested narrative frames in time, space, and subjectivity. Time Code is in one way relentlessly linear, but its synchronic depiction of multiple physical and emotional points of view ruptures the cinematic conventions of time and space constructed by the dominant style of continuity editing. Taken separately, these three films hover around the center point of my diagram, raising the issues of agency and linearity that will continue to be at the center of digital narrative. Taken together the films offer a model of digital transformation that points the way to what is bound to be a medium that increasingly involves its audience in thinking about and then participating in increasingly immersive, nonlinear, and interactive experiences.