Unlike some current discourse on automediality, this essay eschews most of the analysis concerning the adoption or modification of avatars to deliberately enhance, extend or distort the self. Rather than the automedial enabling of alternative, virtual selves modified by playful, confronting or disarming avatars we concentrate instead on emerging efforts to present the self in hyper-realist, interactive modes. In doing so we ask, what is the relationship between traumatic forms of automediation and the affective impact on and response of the audience? We argue that, while on the one hand there are promising avenues for valuable individual and social engagements with traumatic forms of automediation, there is an overwhelming predominance of suffering as a theme in such virtual depictions, comingled with uncritically asserted promises of empathy, which are problematic as the technology assumes greater mainstream uptake.
As Smith and Watson note, embodiment is always a “translation” where the body is “dematerialized” in virtual representation (“Virtually” 78). Past scholarship has analysed the capacity of immersive realms, such as Second Life or online games, to highlight how users can modify their avatars in often spectacular, non-human forms. Critics of this mode of automediality note that users can adopt virtually any persona they like (racial, religious, gendered and sexual, human, animal or hybrid, and of any age), behaving as “identity tourists” while occupying virtual space or inhabiting online communities (Nakamura). Furthermore, recent work by Jaron Lanier, a key figure from the 1980s period of early Virtual Reality (VR) technology, has also explored so-called “homuncular flexibility” which describes the capacity for humans to seemingly adapt automatically to the control mechanisms of an avatar with multiple legs, other non-human appendages, or for two users to work in tandem to control a single avatar (Won et. al.). But this article is concerned less with these single or multi-player online environments and the associated concerns over modifying interactive identities. We are principally interested in other automedial modes where the “auto” of autobiography is automated via Artificial Intelligences (AIs) to convincingly mimic human discourse as narrated life-histories.
We draw from case studies promoted by the 2017 season of ABC television’s flagship science program, Catalyst, which opened with semi-regular host and biological engineer Dr Jordan Nguyen, proclaiming in earnest, almost religious fervour: “I want to do something that has long been a dream. I want to create a copy of a human. An avatar. And it will have a life of its own in virtual reality.” As the camera followed Nguyen’s rapid pacing across real space he extolled: “Virtual reality, virtual human, they push the limits of the imagination and help us explore the impossible […] I want to create a virtual copy of a person. A digital addition to the family, using technology we have now.”
The troubling implications of such rhetoric were stark and the next third of the program did little to allay such techno-scientific misgivings. Directed and produced by David Symonds, with Nguyen credited as co-developer and presenter, the episode “Meet the Avatars” immediately introduced scenarios where “volunteers” entered a pop-up inner city virtual lab, to experience VR for the first time. The volunteers were shown on screen subjected to a range of experimental VR environments designed to elicit fear and/or adverse and disorienting responses such as vertigo, while the presenter and researchers from Sydney University constantly smirked and laughed at their participants’ discomfort. We can only wonder what the ethics process was for both the ABC and university researchers involved in these broadcast experiments. There is little doubt that the participant/s experienced discomfort, if not distress, and that was televised to a national audience. Presenter Nguyen was also shown misleading volunteers on their way to the VR lab, when one asked “You’re not going to chuck us out of a virtual plane are you?” to which Nguyen replied “I don't know what we’re going to do yet,” when it was next shown that they immediately underwent pre-programmed VR exposure scenarios, including a fear of falling exercise from atop a city skyscraper.
The sweat-inducing and heart rate-racing exposures to virtual plank walks high above a cityscape, or seeing subjects haptically viewing spiders crawl across their outstretched virtual hands, all elicited predictable responses, showcased as carnivalesque entertainment for the viewing audience. As we will see, this kind of trivialising of a virtual environment’s capacity for immersion belies the serious use of the technology in a range of treatments for posttraumatic stress disorder (see Rizzo and Koenig; Rothbaum, Rizzo and Difede).
Figure 1: Nguyen and researchers enjoying themselves as their volunteers undergo VR exposure
In their pioneering 2008 work, Automedialität: Subjektkonstitution in Schrift, Bild und neuen Medien, Jörg Dünne and Christian Moser coined the term “automediality” to problematise the production, application and distribution of autobiographic modes across various media and genres—from literary texts to audiovisual media and from traditional expression to inter/transmedia and remediated formats. The concept of automediality was deployed to counter the conventional critical exclusion of analysis of the materiality/technology used for an autobiographical purpose (Gernalzick). Dünne and Moser proffered a concept of automediality that rejects the binary division of (a) self-expression determining the mediated form or (b) (self)subjectivity being solely produced through the mediating technology. Hence, automediality has been traditionally applied to literary constructs such as autobiography and life-writing, but is now expanding into the digital domain and other “paratextual sites” (Maguire).
As Nadja Gernalzick suggests, automediality should “encourage and demand not only a systematics and taxonomy of the constitution of the self in respectively genre-specific ways, but particularly also in medium-specific ways” (227). Emma Maguire has offered a succinct working definition that builds on this requirement to signal the automedial universally, noting it operates as
a way of studying auto/biographical texts (of a variety of forms) that take into account how the effects of media shape the kinds of selves that can be represented, and which understands the self not as a preexisting subject that might be distilled into story form but as an entity that is brought into being through the processes of mediation.
Sidonie Smith and Julia Watson point to automediality as a methodology, and in doing so emphasize how the telling or mediation of a life actually shapes the kind of story that can be told autobiographically. They state “media cannot simply be conceptualized as ‘tools’ for presenting a preexisting, essential self […] Media technologies do not just transparently present the self. They constitute and expand it” (Smith and Watson “Virtually Me” 77).
This distinction is vital for understanding how automediality might be applied to self-expression in virtual domains, including the holographic avatar dreams of Nguyen throughout Catalyst. Although addressing this distinction in relation to online websites, following P. David Marshall’s description of “the proliferation of the public self”, Maguire notes:
The same integration of digital spaces and platforms into daily life that is prompting the development of new tools in autobiography studies […] has also given rise to the field of persona studies, which addresses the ways in which individuals engage in practices of self-presentation in order to form commoditised identities that circulate in affective communities.
For Maguire, these automedial works operate textually “to construct the authorial self or persona”.
An extension to this digital, authorial construction is apparent in the exponential uptake of screen mediated prosumer generated content, whether online or theatrical (Miller). According to Gernalzick, unlike fictional drama films, screen autobiographies more directly enable “experiential temporalities”. Based on Mary Anne Doane’s promotion of the “indexicality” of film/screen representations to connote the real, Gernalzick suggests that despite semiotic theories of the index problematising realism as an index as representation, the film medium is still commonly comprehended as the “imprint of time itself”:
Film and the spectator of film are said to be in a continuous present. Because the viewer is aware, however, that the images experienced in or even as presence have been made in the past, the temporality of the so-called filmic present is always ambiguous” (230).
When expressed as indexical, automedial works, the intrinsic audio-visual capacities of film and video (as media) far surpass the temporal limitations of print and writing (Gernalzick, 228). One extreme example can be found in an emergent trend of “performance crime” murder and torture videos live-streamed or broadcast after the fact using mobile phone cameras and FaceBook (Bender). In essence, the political economy of the automedial ecology is important to understand in the overall context of self expression and the governance of content exhibition, access, distribution and—where relevant—interaction.
So what are the implications for automedial works that employ virtual interfaces and how does this evolving medium inform both the expressive autobiographical mode and audiences subjectivities?
The Catalyst program described above strove to shed new light on the potential for emerging technology to capture and create virtual avatars from living participants who (self-)generate autobiographical narratives interactively. Once past the initial gee-wiz journalistic evangelism of VR, the episode turned towards host Nguyen’s stated goal—using contemporary technology to create an autonomous virtual human clone. Nguyen laments that if he could create only one such avatar, his primary choice would be that of his grandfather who died when Nguyen was two years old—a desire rendered impossible. The awkward humour of the plank walk scenario sequence soon gives way as the enthusiastic Nguyen is surprised by his family’s discomfort with the idea of digitally recreating his grandfather.
Nguyen next visits a Southern California digital media lab to experience the process by which 3D virtual human avatars are created. Inside a domed array of lights and cameras, in less than one second a life-size 3D avatar is recorded via 6,000 LEDs illuminating his face in 20 different combinations, with eight cameras capturing the exposures from multiple angles, all in ultra high definition. Called the Light Stage (Debevec), it is the same technology used to create a life size, virtual holocaust survivor, Pinchas Gutter (Ziv).
We see Nguyen encountering a life-size, high-resolution 2D screen version of Gutter’s avatar. Standing before a microphone, Nguyen asks a series of questions about Gutter’s wartime experiences and life in the concentration camps. The responses are naturalistic and authentic, as are the pauses between questions. The high definition 4K screen is photo-realist but much more convincing in-situ (as an artifact of the Catalyst video camera recording, in some close-ups horizontal lines of transmission appear). According to the project’s curator, David Traum, the real Pinchas Gutter was recorded in 3D as a virtual holograph. He spent 25 hours providing 1,600 responses to a broad range of questions that the curator maintained covered “a lot of what people want to say” (Catalyst).
Figure 2: The Museum of Jewish Heritage in Manhattan presented an installation of New Dimensions in Testimony, featuring Pinchas Gutter and Eva Schloss
It is here that the intersection between VR and auto/biography hybridise in complex and potentially difficult ways. It is where the concept of automediality may offer insight into this rapidly emerging phenomenon of creating interactive, hyperreal versions of our selves using VR. These hyperreal VR personae can be questioned and respond in real-time, where interrogators interact either as casual conversers or determined interrogators.
The impact on visitors is sobering and palpable. As Nguyen relates at the end of his session, “I just want to give him a hug”. The demonstrable capacity for this avatar to engender a high degree of empathy from its automedial testimony is clear, although as we indicate below, it could simply indicate increased levels of emotion.
Regardless, an ongoing concern amongst witnesses, scholars and cultural curators of memorials and museums dedicated to preserving the history of mass violence, and its associated trauma, is that once the lived experience and testimony of survivors passes with that generation the impact of the testimony diminishes (Broderick). New media modes of preserving and promulgating such knowledge in perpetuity are certainly worthy of embracing. As Stephen Smith, the executive director of the USC Shoah Foundation suggests, the technology could extend
to people who have survived cancer or catastrophic hurricanes […] from the experiences of soldiers with post-traumatic stress disorder or survivors of sexual abuse, to those of presidents or great teachers. Imagine if a slave could have told her story to her grandchildren? (Ziv)
Yet questions remain as to the veracity of these recorded personae. The avatars are created according to a specific agenda and the autobiographical content controlled for explicit editorial purposes. It is unclear what and why material has been excluded. If, for example, during the recorded questioning, the virtual holocaust survivor became mute at recollecting a traumatic memory, cried or sobbed uncontrollably—all natural, understandable and authentic responses given the nature of the testimony—should these genuine and spontaneous emotions be included along with various behavioural ticks such as scratching, shifting about in the seat and other naturalistic movements, to engender a more profound realism?
The generation of the photorealist, mimetic avatar—remaining as an interactive persona long after the corporeal, authorial being is gone—reinforces Baudrillard’s concept of simulacra, where a clone exists devoid of its original entity and unable to challenge its automedial discourse. And what if some unscrupulous hacker managed to corrupt and subvert Gutter’s AI so that it responded antithetically to its purpose, by denying the holocaust ever happened? The ethical dilemmas of such a paradigm were explored in the dystopian 2013 film, The Congress, where Robyn Wright plays herself (and her avatar), as an out of work actor who sells off the rights to her digital self. A movie studio exploits her screen persona in perpetuity, enabling audiences to “become” and inhabit her avatar in virtual space while she is limited in the real world from undertaking certain actions due to copyright infringement. The inability of Wright to control her mimetic avatar’s discourse or action means the assumed automedial agency of her virtual self as an immortal, interactive being remains ontologically perplexing.
Figure 3: Robyn Wright undergoing a full body photogrammetry to create her VR avatar in The Congress (2013)
The various virtual exposures/experiences paraded throughout Catalyst’s “Meet the Avatars” paradoxically recorded and broadcast a range of troubling emotional responses to such immersion. Many participant responses suggest great caution and sensitivity be undertaken before plunging headlong into the new gold rush mentality of virtual reality, augmented reality, and AI affordances. Catalyst depicted their program subjects often responding in discomfort and distress, with some visibly overwhelmed by their encounters and left crying. There is some irony that presenter Ngyuen was himself relying on the conventions of 2D linear television journalism throughout, adopting face-to-camera address in (unconscious) automedial style to excitedly promote the assumed socio-cultural boon such automedial VR avatars will generate.
There are numerous ethical considerations surrounding the potential for AIs to expand beyond automedial (self-)expression towards photorealist avatars interacting outside of their pre-recorded content. When such systems evolve it may be neigh impossible to discern on screen whether the person you are conversing with is authentic or an indistinguishable, virtual doppelganger. In the future, a variant on the Turning Test may be needed to challenge and identify such hyperreal simulacra. We may be witnessing the precursor to such a dilemma playing out in the arena of audio-only podcasts, with some public intellectuals such as Sam Harris already discussing the legal and ethical problems from technology that can create audio from typed text that convincingly replicate the actual voice of a person by sampling approximately 30 minutes of their original speech (Harris). Such audio manipulation technology will soon be available to anybody with the motivation and relatively minor level of technological ability in order to assume an identity and masquerade as automediated dialogue. However, for the moment, the ability to convincingly alter a real-time computer generated video image of a person remains at the level of scientific innovation.
Also of significance is the extent to which the audience reactions to such automediated expressions are indeed empathetic or simply part of the broader range of affective responses that also include direct sympathy as well as emotions such as admiration, surprise, pity, disgust and contempt (see Plantinga). There remains much rhetorical hype surrounding VR as the “ultimate empathy machine” (Milk). Yet the current use of the term “empathy” in VR, AI and automedial forms of communication seems to be principally focused on the capacity for the user-viewer to ameliorate negatively perceived emotions and experiences, whether traumatic or phobic.
When considering comments about authenticity here, it is important to be aware of the occasional slippage of technological terminology into the mainstream. For example, the psychological literature does emphasise that patients respond strongly to virtual scenarios, events, and details that appear to be “authentic” (Pertaub, Slater, and Barker). Authentic in this instance implies a resemblance to a corresponding scenario/activity in the real world. This is not simply another word for photorealism, but rather it describes for instance the experimental design of one study in which virtual (AI) audience members in a virtual seminar room designed to treat public speaking anxiety were designed to exhibit “random autonomous behaviours in real-time, such as twitches, blinks, and nods, designed to encourage the illusion of life” (Kwon, Powell and Chalmers 980). The virtual humans in this study are regarded as having greater authenticity than an earlier project on social anxiety (North, North, and Coble) which did not have much visual complexity but did incorporate researcher-triggered audio clips of audience members “laughing, making comments, encouraging the speaker to speak louder or more clearly” (Kwon, Powell, and Chalmers 980). The small movements, randomly cued rather than according to a recognisable pattern, are described by the researchers as creating a sense of authenticity in the VR environment as they seem to correspond to the sorts of random minor movements that actual human audiences in a seminar can be expected to make.
Nonetheless, nobody should regard an interaction with these AIs, or the avatar of Gutter, as in any way an encounter with a real person. Rather, the characteristics above function to create a disarming effect and enable the real person-viewer to willingly suspend their disbelief and enter into a pseudo-relationship with the AI; not as if it is an actual relationship, but as if it is a simulation of an actual relationship (USC). Lucy Suchman and colleagues invoke these ideas in an analysis of a YouTube video of some apparently humiliating human interactions with the MIT created AI-robot Mertz. Their analysis contends that, while it may appear on first glance that the humans’ mocking exchange with Mertz are mean-spirited, there is clearly a playfulness and willingness to engage with a form of AI that is essentially continuous with “long-standing assumptions about communication as information processing, and in the robot’s performance evidence for the limits to the mechanical reproduction of interaction as we know it through computational processes” (Suchman, Roberts, and Hird).
Thus, it will be important for future work in the area of automediated testimony to consider the extent to which audiences are willing to suspend disbelief and treat the recounted traumatic experience with appropriate gravitas. These questions deserve attention, and not the kind of hype displayed by the current iteration of techno-evangelism. Indeed, some of this resurgent hype has come under scrutiny. From the perspective of VR-based tourism, Janna Thompson has recently argued that “it will never be a substitute for encounters with the real thing” (Thompson). Alyssa K. Loh, for instance, also argues that many of the negatively themed virtual experiences—such as those that drop the viewer into a scene of domestic violence or the location of a terrorist bomb attack—function not to put you in the position of the actual victim but in the position of the general category of domestic violence victim, or bomb attack victim, thus “deindividuating trauma” (Loh).
Future work in this area should consider actual audience responses and rely upon mixed-methods research approaches to audience analysis. In an era of alt.truth and Cambridge Analytics personality profiling from social media interaction, automediated communication in the virtual guise of AIs demands further study.
Anon. “New Dimensions in Testimony.” Museum of Jewish Heritage. 15 Dec. 2017. 19 Apr. 2018 <http://mjhnyc.org/exhibitions/new-dimensions-in-testimony/>.
Australian Broadcasting Corporation. “Meet The Avatars.” Catalyst, 15 Aug. 2017.
Baudrillard, Jean. “Simulacra and Simulations.” Jean Baudrillard: Selected Writings. Ed. Mark Poster. Stanford: Stanford UP, 1988. 166-184.
Bender, Stuart Marshall. Legacies of the Degraded Image in Violent Digital Media. Basingstoke: Palgrave Macmillan, 2017.
Broderick, Mick. “Topographies of Trauma, Dark Tourism and World Heritage: Hiroshima’s Genbaku Dome.” Intersections: Gender and Sexuality in Asia and the Pacific. 24 Apr. 2010. 14 Apr. 2018 <http://intersections.anu.edu.au/issue24/broderick.htm>.
Debevec, Paul. “The Light Stages and Their Applications to Photoreal Digital Actors.” SIGGRAPH Asia. 2012.
Doane, Mary Ann. The Emergence of Cinematic Time: Modernity, Contingency, the Archive. Cambridge: Harvard UP, 2002.
Dünne, Jörg, and Christian Moser. “Allgemeine Einleitung: Automedialität”. Automedialität: Subjektkonstitution in Schrift, Bild und neuen Medien. Eds. Jörg Dünne and Christian Moser. München: Wilhelm Fink, 2008. 7-16.
Harris, Sam. “Waking Up with Sam Harris #64 – Ask Me Anything.” YouTube, 16 Feb. 2017. 16 Mar. 2018 <https://www.youtube.com/watch?v=gMTuquaAC4w>.
Kwon, Joung Huem, John Powell, and Alan Chalmers. “How Level of Realism Influences Anxiety in Virtual Reality Environments for a Job Interview.” International Journal of Human-Computer Studies 71.10 (2013): 978-87.
Loh, Alyssa K. "I Feel You." Artforum, Nov. 2017. 10 Apr. 2018 <https://www.artforum.com/print/201709/alyssa-k-loh-on-virtual-reality-and-empathy-71781>.
Marshall, P. David. “Persona Studies: Mapping the Proliferation of the Public Self.” Journalism 15.2 (2014): 153-170.
Mathews, Karen. “Exhibit Allows Virtual ‘Interviews’ with Holocaust Survivors.” Phys.org Science X Network, 15 Dec. 2017. 18 Apr. 2018 <https://phys.org/news/2017-09-virtual-holocaust-survivors.html>.
Maguire, Emma. “Home, About, Shop, Contact: Constructing an Authorial Persona via the Author Website” M/C Journal 17.9 (2014).
Miller, Ken. More than Fifteen Minutes of Fame: The Evolution of Screen Performance. Unpublished PhD Thesis. Murdoch University. 2009.
Milk, Chris. “Ted: How Virtual Reality Can Create the Ultimate Empathy Machine.” TED Conferences, LLC. 16 Mar. 2015. <https://www.ted.com/talks/chris_milk_how_virtual_reality_can_create_the_ultimate_empathy_machine>.
Nakamura, Lisa. “Cyberrace.” Identity Technologies: Constructing the Self Online. Eds. Anna Poletti and Julie Rak. Madison, Wisconsin: U of Wisconsin P, 2014. 42-54.
North, Max M., Sarah M. North, and Joseph R Coble. "Effectiveness of Virtual Environment Desensitization in the Treatment of Agoraphobia." International Journal of Virtual Reality 1.2 (1995): 25-34.
Pertaub, David-Paul, Mel Slater, and Chris Barker. “An Experiment on Public Speaking Anxiety in Response to Three Different Types of Virtual Audience.” Presence: Teleoperators and Virtual Environments 11.1 (2002): 68-78.
Plantinga, Carl. "Emotion and Affect." The Routledge Companion to Philosophy and Film. Eds. Paisley Livingstone and Carl Plantinga. New York: Routledge, 2009. 86-96.
Rizzo, A.A., and Sebastian Koenig. “Is Clinical Virtual Reality Ready for Primetime?” Neuropsychology 31.8 (2017): 877-99.
Rothbaum, Barbara O., Albert “Skip” Rizzo, and JoAnne Difede. "Virtual Reality Exposure Therapy for Combat-Related Posttraumatic Stress Disorder." Annals of the New York Academy of Sciences 1208.1 (2010): 126-32.
Smith, Sidonie, and Julia Watson. Reading Autobiography: A Guide to Interpreting Life Narratives. 2nd ed. Minneapolis: U of Minnesota P, 2010.
———. “Virtually Me: A Toolbox about Online Self-Presentation.” Identity Technologies: Constructing the Self Online. Eds. Anna Poletti and Julie Rak. Madison: U of Wisconsin P, 2014. 70-95.
Suchman, Lucy, Celia Roberts, and Myra J. Hird. "Subject Objects." Feminist Theory 12.2 (2011): 119-45.
Thompson, Janna. "Why Virtual Reality Cannot Match the Real Thing." The Conversation, 14 Mar. 2018. 10 Apr. 2018 <http://theconversation.com/why-virtual-reality-cannot-match-the-real-thing-92035>.
USC. "Skip Rizzo on Medical Virtual Reality: USC Global Conference 2014." YouTube, 28 Oct. 2014. 2 Apr. 2018 <https://www.youtube.com/watch?v=PdFge2XgDa8>.
Won, Andrea Stevenson, Jeremy Bailenson, Jimmy Lee, and Jaron Lanier. "Homuncular Flexibility in Virtual Reality." Journal of Computer-Mediated Communication 20.3 (2015): 241-59.
Ziv, Stan. “How Technology Is Keeping Holocaust Survivor Stories Alive Forever”. Newsweek, 18 Oct. 2017. 19 Apr. 2018 <http://www.newsweek.com/2017/10/27/how-technology-keeping-holocaust-survivor-stories-alive-forever-687946.html>.