Art as Sensory Pharmakon & the Mass Media Overdose
The following presentation is a thought experiment that, in all seriousness, asks you to examine the nature of media. I honestly believe that social, political, and financial challenges, as well as the far more imposing ecological crisis, can be approached and potentially mitigated by our prompt, critical engagement with media. In that respect, much has been made lately of ‘knowledge mobilization.’ According to the Social Sciences and Humanities Research Council of Canada (a post-secondary federal funding agency), the aim of knowledge moblization is the completion of the circuit by which the academy gives back to the community that supports it.
But the very phrase itself risks undermining its own agenda. For mobilization suggests the movement of resources, as in troops or goods; it’s as though universities are sitting on stockpiles of research that need only be made accessible to meet our obligation to the public at large. This is precisely the misguided, unidirectional thinking that I will be arguing against . For if universities are to fully embrace the admirable ideals behind knowledge mobilization, then they must be willing to change what they consider knowledge to be. To that end this talk will focus on the capacity for media to shape knowledge through an uncanny cause and effect process. In order to do that we will need to engage in a form of ‘guerilla phenomenology,’ as we look at art, mass media, and things in between as opportunities to explore sensory experiences.
Art Effect Before Mass Media Cause
Here we have a pointillist painting by Seurat of his mother dated 1883. And here we have a television image, approximately 1954.
This is a work by Paul Klee, expressionist/cubist painter, circa 1914. This is a defractor lens taken from an LCD projector in 2012.
This footage is of Myron Krueger’s Videoplace [start video at 2:14, drawing on screen], which began to take shape in 1971. And here is the Microsoft Kinect camera in 2010 [start video at 0:14 – kinectimals scene].
Art As Pharmakon
These examples are intended to illustrate that in terms of form and sense, art can and often does precede mass media. By this I mean that the pointillist painting, with its black and white dots forming a composite image in the mind of the viewer is similar in kind to the perceptual engagement with the low-fidelity television image with its array of monochrome dots. I am not claiming that Seurat anticipated the television nor am I saying that without pointillism TV wouldn’t exist. Instead I am claiming that the artist and the critically active observer of art engage with sensory experiences decades, sometimes centuries, prior to their deployment as tools of mass media. In this sense, each artwork contains its own cautionary tale in the form of a pattern known to be efficacious when put in contact with the human sensorium. This pattern is like a pharmakon: in small doses it builds an immunity, but in large amounts, it’s poisonous.
Tom is 54 years old. In 1970s he attended one of Myron Krueger’s Videoplace exhibitions and witnessed the potential for a computer to mediate the body. When Tom saw the first commercial for the Xbox Kinect he merely shrugged while others heralded the increased interactivity.
When Apple released their high-definition retina technology for their mobile devices, the iPhone and iPad, users and reviewers alike marvelled at the clarity of the image. Truly this was a step forward in mobile technology.
What I wish to highlight here is that whether interactive human-computer interface, gaming console, tablet, or cell phone, each medium relies on impressions made on the sensorium first and then, after the fact, a conclusion is drawn. Or as McLuhan would have it: “Effects are perceived, whereas causes are conceived” First the light reflects off a mess of black and white dots, striking the retina. Afterwards the cause is determined. You might say: ‘Surely it was a woman before I realized this fact.’ And just like that, a single cause is retroactively inserted. The image goes from points in human form to human in pointillist form. At issue here is that we produce media not according to their effects (what they can teach us) but as causes in themselves (heightened realism). By cause I mean a singular, localized source; the billiard ball that first sets the others in motion, the finger that pulls the trigger to fire the gun, the computer that starts the calculation. In other words, causes are reductions. Did the finger fire the gun? Or was it the firing pin that struck the bullet? Or perhaps it was the chemical reaction that actually launched the projectile? Ultimately, a cause is a series of effects reduced after the fact to a single interpretation.
Put differently, cause and effect is a linear interpretation of a decidedly non-linear occurrence. This is apparent in Merleau-Ponty’s characterization of classical science as “a form of perception which loses sight of its origins and believes itself complete” (Merleau-Ponty) .This applies especially to those empiricists who dedicate their lives to studying effects without any thought to the causality imposed by their observation techniques. Speaking on the subject of film in the late 1980s, Charles Eidsvik wrote that “The basic problem in theorizing about technical change…is that accurate histories of the production community and its perspectives, as well as of the technological options…must precede the attempt to theorize.” While on the topic of games Aphra Kerr asks “How can we talk with authority about the effects of digital games when we are only beginning to understand the game/user relationship and the degree to which it gives more creative freedom and agency to users?” With all due respect to these theorists, we must abandon this naive approach to media studies. For in essence Kerr is stating that we must wait for the results to emerge that fit our method of testing. With this mentality, it’s no wonder we are impotent in the face of the present ecological crisis. We have Geiger counters, carbon sensors, and emission testers; that is say, we have all the tools to tell us it’s too late, and few to prevent the actual event.
Events in the Gap
But the future is not something that happens to us. Our role as victims or beneficiaries of the passage of time is self-ascribed. It happens that this temporality is sustained and perpetuated by our media, which are decidedly linear in there causality. Consider how one reads a comic book. As our eyes move from panel to panel the mind fills in the missing information needed to complete the action. But despite the fact that the additional information is added last, the overall experience of reading the comic means that this third event occurs prior to the second. That is, the cause of the events in the second panel is determined AFTER the panel is experienced but we read the cause as taking place BEFORE.
If we explore this cause in detail, it is at once familiar to us and yet unknown; it soon becomes clear that bounded by the adjacent panels, events in the gap are indeterminate, irreducible but somehow already present. To read this form, ultimately, is to accept indeterminacy without having to pay homage to the undefined. My critique here is not aimed at comic books specifically but at specialized literacies in general, and especially those that follow from linear media. What we need to consider is the possibility of media that are not so successive, that are not so ostensibly progressive, where the only role of the reader/participant/gamer is to perpetuate the dialogue/the script/or the gameplay. But this notion of linearity still bears description. For a more literary example, consider the murder mystery where the disclosure of the killer’s identity rewrites the narrative. When the detective reveals that the butler did it, our minds return to his seemingly innocuous behaviour. Suddenly the butler’s absence from the dining hall now appears grossly suspicious, his phrasing now indicates a measure of foreknowledge about one of the murders, and his behaviour when one of the bodies was found which appeared at first reading to be genuine now smacks of pretense. Given this example perhaps it would be best to revisit the forefather of the detective genre.
In Edgar Allan Poe’s essay “Philosophy of Composition,” the American author offers reveals, in a remarkably upfront manner, his methodology: “I prefer commencing with the consideration of an effect…I say to myself, in the first place, ‘Of the innumerable effects, or impressions, of which the heart, the intellect, or (more generally) the soul is susceptible, what one shall I, on the present occasion, select?’” While Poe’s ensuing example here is ‘The Raven,’ it was his short story series starring detective Auguste Dupin that popularized the writing technique of starting with a clever conclusion (the effect) and working backwards to the beginning where events were hardly so clear (the cause). In one particular tale, called “The Purloined Letter,” Poe describes the failure of the Parisian police in their task of retrieving a stolen letter. We are told at the opening of the tale that the police know who stole the letter and in fact, they even know which house it is in. But after exhaustive and creative endeavours to locate the hiding place, the police come up empty handed. And yet Poe’s detective extraordinaire, Dupin, locates the letter almost immediately. For it was hidden in plain sight, obscured by nothing other than its familiar placement amongst other letters on a bureau.
Familiarity plays a key role here. Just as we are familiar with how to read a comic by inserting the necessary action between panels, the police were trained to be familiar with petty criminals and the routine behaviour of a thief. We can call this familiarity ‘literacy.’ But like the dyslexia that follows speed reading, the more familiar we become, the more our expectations supersede reality. The pharmakon-quality of sensory patterns resurfaces. Our empirical methods, like those of the highly trained police officers in Poe’s story, fail to account for the simple fact that we cannot find what we do not expect to see. It’s true that test results may change our expectations of the future but by then it’s already the past. The question then becomes: Can we disrupt this dependency? And if so, how?
In my own work I see each medium as bringing with it a series of structural metaphors that facilitates communication across a culture. New media = new metaphors. In fact, we can push this even further to say that media modify the conditions of possibility within a culture. For instance, the computer, as metaphor, has radically altered not only how Westerners conceive of the human body but biology in general. It is not uncommon to hear biologists speak of the behaviour of microorganisms as running operating systems; and in fact it is difficult to conceive of the entire field of synthetic biology, the design and construction of biological devices and systems, without the computer as metaphor . Here we have an example taken from a synthetic biology competition.
This example illustrates that to a certain, unknown degree, media have a very real impact on what humans can conceive of and talk about. Recall that effects are perceived, whereas causes are conceived, and we can begin to see that each medium, in fact, instantiates its own form of causality when paired with an accompanying literacy. And to date our literacies mimic the linear causality of the media that produce them. How we read the pointillist painting and how we correlate movement with an on-screen avatar each requires our senses to connect visual, audile, and tactile information based on feedback from the medium. But like a linguistic form of quantum entanglement, as I speak this sentence each additional word modifies the meaning of the entire sentence such that it always already meant what the last word intended it to mean. The listener, as well as the reader, the film observer, and the gamer, is perpetually waiting for change to occur, for effects to reveal causes.
But as we have seen, the cause after the effect indicates a systemic blind spot that is filled in by mental processes. It is worth considering that the present array of social, political, and ecological crises have emerged from behind such blind spots. Thus, to await the effects of a medium, a program, a policy, or a chemical process, does less to guide future endeavours and more to expose the inadequacy of our current perceptual tools. The sheer magnitude on which humanity now operates condemns this feedback approach as a viable model for change. What, then, is the alternative? In response to feedback media, I would suggest that it’s feedforward art. It’s Seurat, and Klee, and Krueger as effects before their cause as mass media. It’s perpetually new forms of media immunizing us from long-term side effects of mass media. It’s about creating a dynamic array of perspectives such that no blind spot goes unexamined. It’s about moving from receptive to active media. It’s a full realization that the future of the future is the present. Let’s take what we’ve covered here and return to the notion of knowledge mobilization:
“The relationship between knowledge mobilization, and outcomes and impacts is far from a simple question of “cause and effect” and, rather, more recursive…A social sciences and humanities voice on knowledge mobilization opens the door to non-linear, dialogical, discursive and multi-directional approaches with the general acknowledgement that all knowledge is “socially constructed” unlike the unidirectional “producer-consumer” implications of concepts such as knowledge and technology transfer.” (Social Sciences and Humanities Research Council of Canada Knowledge Mobilization Strategy)
Certainly there is some interesting language choice here. Yes, we need to move beyond the simple question of cause and effect and yes non-linearity will be key moving forward. But we must also accept that effects precede causes. And we must not mistake multi-directional or multi-vocal approaches as non-linear. The Canterbury Tales may provide stories from a range of classes but the medium itself is still fastidiously linear. Instead, the mobile element of knowledge mobilization should be directed less at the material and more at the intended recipients; in other words, what will mobilize the community? I think we can all agree that it isn’t open access to JSTOR articles.
Then what is it? I think it begins with non-linear media. And in that respect I can only offer three steps towards a definition, and they are small ones at that. What follows are three projects that focus on the computer specifically. This is because another stumbling block towards feedforward is the fact that we populate new media with old metaphors, such that when McLuhan’s observation that “Computers are still serving mainly as agents to sustain precomputer effects” remains as true today as when he uttered it over thirty years ago. And nowhere is this clearer than in the realm of videogames. Our gaming machines are populated with narrative effects borrowed from literature, film, radio, newspapers, and the stage. The unique effects that follow digital, ludic narratives are rarely realized and the following are hardly the best examples. But here are three attempts to realize that potential nonetheless. In terms of guerilla phenomenology, each game here explores an aspect of perception, starting with time, space, and then causality.
- Based on Poe’s Purloined Letter and the idea that the desktop or bureau is no longer so quotidian
- Events were populated in real-time. Some were written in real-time as well (i.e. cell phone conversation)
- The story is about a serial killer who, it is revealed, claims his victims according to one of the most ancient patterns: the Sator Square. With murders that began in the 80s, the killer resurfaces in the present day, continuing the pattern of matching profession and initials in order to determine his victims. As a player we act as voyeur, watching the desk of FBI Agent Mike Kim as he attempts to solve the mystery. At the end of the game it is revealed that in fact there was no pattern to begin with. The murders in the 80s were found via pattern analysis on the part of the killer, as he played upon the FBI’s own obsession with pattern-analysis and profiling.
- The objective here was that by telling the narrative in real-time and in such a confined manner that I was telling a story that was inseparable from the medium.
- In terms of non-linearity, the persistent nature of the narrative actually disrupts the role of the player/reader as a necessary agent of action. And the fact that it was, in part, a performance, means that it can never be played again.
- This one is a little more straightforward to explain. Essentially, the history of videogames, like literature, has a series of styles that appear to succeed one another in terms of the experience of realism. Games offer a unique opportunity to debunk this myth of realism and mimesis
- What the game involves is a series of visual and audile modes that increase in ‘fidelity’ throughout the game. Each successive level will appear to invalidate the preceding one as the so-called realism increases. But as all gamers know, an 8-bit game can be as engaging as the most technically advanced titles. Thus, players will meet a series of challenges, some of which will be easier to complete on lower fidelities.
- Apophis is a near-Earth asteroid that will approach Earth in 2029. Should it pass through a gravitational keyhole 800m wide, it will be on a collision course for the Earth in 2036. The current strategy to avoiding a catastrophic event is to detect such collisions 10 to 20 years in advance. The idea being that if you can strike the asteroid with a relatively small missile, the very minute course correction will compound over several decades, resulting in a miss.
- Of course, this would involve governments cooperating and coordinating on a global scale about an event that is likely but not guaranteed to occur in the near future. That should sound rather familiar…
- The game itself takes place over 30 days and involves playing a range of characters over those 30 days in a non-chronological fashion. The idea being that actions on day 3 may or may not impact events on day 29. What I hope to achieve is a separation of cause and effect, removing the simple pleasure we get from gaming, thereby increasing the pressure on the decision as the result will not be immediately apparent.