(Semiotic Review of Books 18:3, 2009)
Natural Born Cyborgs Among cutting edge media theorists with fascinating resumes few compare with French intellectual and journalist Régis Debray who in the late sixties spent time as a Marxist revolutionary fighting alongside Che Guevera and later worked as an adviser to French President François Mitterand on Foreign Affairs, but Brian Rotman comes close. Mathematician, playwright, cultural theorist, Rotman holds a Ph.D. in combinatorial mathematics from the University of London, is the author of many plays for stage and radio, and since 1998 has been Professor in the Department of Comparative Studies at Ohio State University. Like N. Katherine Hayles, who parlayed scientific training into a prestigious career as a cultural and literary critic, Rotman applies extensive technical and scientific knowledge to problems in the humanities. His evident lack of guerrilla warfare experience notwithstanding, he is an unusually intrepid thinker and his latest book stirs with revolutionary import.
Rotman’s abilities bring to mind other unclassifiable, ambidextrous thinkers like Donna Harraway, Bruno Latour, Isabelle Stengers, and Michel Serres whose work uses knowledge from the sciences and the humanities to probe new ontological and epistemological realities. After two decades spent teaching pure mathematics in the UK, Rotman went to the US on a series of humanities fellowships where his attention drifted from mathematics to semiotics. The result was two remarkably focused and original books each examining an aspect of the mathematical uncanny from the point of view of a single signifier: Signifying Nothing: The Semiotics of Zero, which takes a Saussurean approach to the zero sign, and Ad Infinitum… The Ghost in Turing’s Machine, which adapts the semiotics of Charles Sanders Peirce to the concept of infinity. In Becoming Beside Ourselves, Rotman completes his trilogy with a study that focuses on yet another solitary signifier (an alphabetic character this time rather than a mathematical sign): ‘I’.
Despite their very narrow focus, there is nothing reductive about these books. Quite the opposite. Like a pinhole camera that captures a startlingly vivid and wide image, Rotman’s gaze takes in vast intellectual landscapes through a very narrow aperture. From the point of view of semiotics, the zero sign appears as “a major signifying event” both in the writing of numbers and in terms of parallel movements in other sign systems such as the development of vanishing point in representational art and the concept of imaginary money in economics. The ability to signify nothing opens up a new apparatus both conceptual and spatial for imagining and positioning absence and its effects. Hard to believe a single signifier—and of all signifiers the ‘0’ sign—could change the world, but there you have it.
In his study of the infinity sign, Rotman uses semiotics to produce surprising results starting from a simple principle. The simple principle is that “counting is an activity involving signs”(6); the surprising result is the disclosure of a ghost in the workings of mathematics: a disembodied, transcendental agent that a system of abstract counting calls upon to count endlessly on our behalf any time the sign “…” or “∞” appears. The identification of a fantasy narrative involving a strange ghost working deep within the operations of math is rather spooky, but this ghost in the machine of mathematics is merely the pretext for searching out its doppelganger, the corporeal body that actually counts. Therein lies the possibility of a new understanding of counting and an occasion “to start rewriting the connections between God, Number, the body” (11). No more should we see mathematics as just a handmaiden to science; it also enfolds a robust corporeal existence. The fascinating prospect for Rotman is not to ratchet up the power of numbers to facilitate abstract operations, but rather to rehabilitate the body as the literal, material condition of all counting.
In Becoming Beside Ourselves, Rotman’s claim is characteristically simple and far-reaching: “any act of self-enunciation is medium-specific,” such that, “to utter ‘I’ and to write ‘I’, despite their everyday conflation within Western textual discourse, are radically different signifying acts” (xxxiii). The book as a whole shows how this basic distinction—and media-specific analysis generally—can produce a decisive transformation in our understanding of culture.
To utter ‘I’, of course, is to participate into an oral/aural economy where prosody (the volume, pitch, tone, pace, and rhythm of speech) infuses words with embodied meanings. Prosody is the presence and action of the body in communication, and although we are very practiced at overlooking it, this presence is extraordinarily meaningful and evocative. The embodied ‘I’ signifies within a material matrix of overwhelming complexity and nuance shot through with emotion, affect, ethos, and intonational force.
The genius (and, if you’re a poet, the great liability) of the alphabet is its almost miraculous efficiency, or what amounts to the same thing, its indifference to prosody. In recording what the speaker says but not how it is said, the alphabet disengages communication from the body and streamlines all the rich and dynamic redundancy and prolixity of speech down to 26 letters and a handful of punctuation and diacritical marks. Disconnecting language from the body revolutionizes communication by rendering information and ideas portable. It also gives rise to a completely new cognitive apparatus. Ground zero for this new apparatus is the radically transformed status of the speaker in writing.
In speech, the speaker is a material, sensory presence within a social semiotic defined by heteroglossia. In writing, the speaker becomes disincarnate, silent, invisible—a cipher and a shifter. Corresponding to the new status of the ‘I’ in writing is a new type of agency: transcendent, abstract, totalizing, indexical, “an unembodied being outside the confines of time and space”. The names we give to this agency—God, Mind, Infinity—form the metaphysical horizons of Western religious, philosophical, and mathematical thought. Each one of these ghosts, Rotman claims, is “a phenomenon inseparable from alphabetic writing” (7). God, therefore, is “a mediological achievement.”
This style of thinking does not so much cut across disciplines as it does pull the rug out from under them. It’s pioneers—Harold Innis, Marshall McLuhan, and Friedrich Kittler—don’t have much truck with scholarship in the traditional sense, which is, after all, like God, a mediological achievement. Instead they posit a special kind of agency to media and technology. Far from being mere tools, writing, print, sound recording, cinema, radio, television, digital media, and now the internet condition our existence in profound ways.
The earliest advocates of the claim used historical method and lots of panache to build and support their arguments. Since then, evidence has emerged from scientific methods as well. “We now know,” writes Marianne Wolf, “that groups of neurons create new connections and pathways among themselves every time we acquire a new skill…. Reading can be learned only because of the brain’s plastic design, and when reading takes place, that individual brain is forever changed, both physiologically and intellectually” (5). The same holds true for all media.
In this way, media function like cognitive displacements/upgrades. The new mechanisms they supply for notating, inscribing, and capturing traces of the real change our minds in specific ways by sublimating gesture and speech and expanding the ways we are able to think. In the most general sense, Western sign systems—the alphabet and mathematical signs—give us the means to construct the world as an assemblage of static, abstract, totalizable concepts and systems. With this comes the ability to bracket our immanence and put our experience of the world under arrest so that we can study it. Concepts, things, ideas, logic, the academy, democracy—the great molarities of Western antiquity—are all consequences of the alphabet, just as capitalism, humanism, Protestantism, the individual, the nation state, colonialism, industrialism, parliamentary democracy, the Renaissance, the Enlightenment, the Scientific Revolution, and the novel are all consequences of the printing press. For two and half millennia, the alphabet has been the operating system of Western culture, and “for at least the last half millennium,” Rotman points out, “the very concept of a person has adhered to that of a ‘lettered self’, an individual psyche inextricable from the apparatus of alphabetic writing describing, articulating, communicating, presenting, and framing it” (2).
But it is not until many hundreds of years after the great social and cultural revolutions of the literate era that we come to recognize their mediological profile. The ubiquity and dominance of writing and print render them environmental, masking their agency. A significant part of the writing/print era is therefore its legacy of media blindness. The one thing it failed to conceptualize was its own agency. “Media” doesn’t enter the language as a word for communications technologies until 1923, and it is not until the age of radio that Milman Parry discovers the oral basis of the Homeric epics. Before the twentieth century the only stable method for media-specific analysis was the oft maligned and beleaguered practices of rhetoric and poetics.
Recently, media studies and science studies have taken on the important work of media- and technology-specific analysis to show how communication and scientific technologies condition knowledge and structure the methodologies informing academic disciplines. For example, N. Katherine Hayles uses media specific analysis to see into the epistemological unconscious of literary criticism. “Lulled into somnolence by five hundred years of print, literary studies have been slow to wake up to the importance of media-specific analysis. Literary criticism and theory are shot through with unrecognized assumptions specific to print. Only now, as the new medium of electronic textuality vibrantly asserts its presence, are these assumptions clearly coming into view” (67). Bruno Latour uses media and technology specific analysis as an occasion to audit normative methods of knowledge creation in the sciences. Mark Hansen critiques literary theory from much the same point of view.
As unacknowledged legislators of knowledge (because they structure cognition, epistemology, and method) and experience (because they “structure our lifeworlds” (Hansen 4) and generate new forms of embodiment), media and technology come to represent, in a way that language did for poststructuralism, “the totality of our problematic horizon” (Derrida’s phrase). By universalizing textuality as a supermedium though, poststructuralism elides the specific material agencies of technology and media. By substituting an analysis of mediality for textuality, media studies offers a corrective, prompting a revealing shift in focus and methodology: from textuality to mediality/materiality, from discourse to experience, from interpretation to media specific analysis.
The theoretical retrofits and retooled methodologies required to take account of media and technology are significant—significant enough to destabilize disciplines. As Hansen points out, “the displacement of language as a universal medium and the correlated recourse to experience confronts the contemporary poststructurally smart cultural critic with what amounts to a significant theoretical quandary” (1). Literary studies and cultural studies must now confront the idea that their signature methods of close reading and critique might not get to the bottom of culture. Semiotics is likewise forced to reckon with the fact that the materialities of communication condition signification and problematize the sign as a structure of understanding.
Rotman’s contribution to this strain of thinking is to combine the grand narrative of media studies with a vivid and precise analytic. His book is notable both for its colossal scope and its very particular emphasis on the status of the speaker in language. The big picture is loaded with portent: If, as Friedrich Kittler says, media determine our situation, and if the two and a half millennia era of alphabetic graphism is drawing to a close, what kind of virtuality and embodiment will correspond to networked media and in what ways will (is) this apparatus restructur(ing) our experience of the world? The vector he provides for tracking and understanding the implications—the diverse instantiations of the ‘I’ across media—dramatizes the stakes of media technologies for human subjects throughout history.
The big picture and the focus on the speaker in language are tightly linked. “A succession of media—speech, alphabetic writing, digital writing—transform their environments through a wave of virtuality specific to them,” Rotman explains (7). “In the first, virtuality is located in the symbolic function per se, inherent in a speaker’s capacity to refer to non-existent and disembodied agencies; in the second, virtuality is located in writing’s ability to signify across space and time in the absence of a real or embodied speaker; the third, still breaking, wave is manifest in the contemporary phenomenon of virtualizing X, where X ranges over the characteristic abstractions and processes of the alphabetic, pre-digital age. Associated with each of these virtual waves is a ghost effect” (7-8). The ghost effect of the spoken ‘I’ is the belief in a spirit; for the alphabetic ‘I’ it is the transcendental agencies known as God, Mind, and Infinity; the ghost effect corresponding to the networked ‘I’ has yet to appear, but the networked ‘I’ itself is coming into view. It is “immersive and gesturo-haptic,” experiencing itself through touch and vision; it is “porous,” flowing over its boundaries and being traversed in turn by other networked ‘I’s; it is “plural and distributed,” constituted by many heterogeneous actions and perceptions at once, governed by parallel protocols rather than linear sequences of input.
Rotman is especially taken with the virtuality associated with the alphabet—what he sometimes refers to as the “metaphysics of the alphabet”: its capacity as a medium, already mentioned, to “perform a reflexive, self-citational move … and thereby give rise, under appropriate conditions, to a disembodied, supernatural agency” (14). But the alphabet’s ability to enable a powerful metaphysics populated by great disembodied agencies like God, Mind, and Infinity is contingent on its ability to eliminate the body, and thereby prosody, from the communicational matrix. The alphabet’s elimination of prosody is easy to overlook and not many scholars outside the fields of sociolinguistics, discourse analysis, and ethnopoetics have bothered to think about it, but for Rotman it is of momentous importance. Indeed, he says, “It would be difficult to exaggerate the consequences of prosody’s omission for the development of western literacy” (26).
Why is the omission of prosody so crucial? It is both the condition of possibility of fully abstract, virtual systems in which signs are able to index the entirety of verbal communication (i.e. all meaningful speech sounds have a corresponding representation in letters) and an enormous cultural reduction. From the point of view of neuroscience, the ability to notate speech sounds in a small number of signs is a cognitive breakthrough, a form of shorthand with enormous payoff in terms of cognitive efficiency. But the incredible efficiency of the alphabet is also a measure of how much information it omits. Speech is a “data flow” of such overwhelming complexity that speech recognition technology has been hailed as the holy grail of computing. As Mladen Dolar writes, “words fail us when we are faced with the infinite shades of the voice, which infinitely exceed meaning” (13). Reading is such an engrossing enterprise because there is so much for the reader to do, so much information that the reader’s imagination must provide. As Rotman puts it, “The history of reading is the history of redressing what writing fails to represent” (27).
That the meaning letters cannot formulate is physically constituted by the body makes it more, not less, important.
What writing omits from speech is the body: the feelings, moods, emotions, attitudes, intuitions, embodied demands, declarations, expressions, and desires located in the voice, rather than consciously formulated (writable) thought. What it omits is the entire field of affect conveyed and induced by human vocality, through the voice’s impulsions inflections, and rhythms, its aural texture and emotional dynamics. A vocal field bordered on one side by song and on the other by the non-speech of sighs, moans, cries, grunts, screams, laughs, and so on, all that, in Roland Barthes’ phrase, surrounds “language lined with flesh.” (27)
Writing effects a split between body and mind, intonation and abstract thought, sound and meaning, setting up its own neurological apparatus to handle purely cognitive tasks. “Neurologically, the requirements of literacy create in the brain what we might call a ‘literacy module’, a neural complex within the neocortex dedicated to writing and reading purely textual entities,… words decoupled from the moods, feelings, desires, and regulatory activity routinely evinced (and induced) by spoken utterance” (30-1). The resultant hierarchy—thought over feeling, cognition over affect, mind over body—define the literate episteme and the scholarly and scientific methods that emerge from it.
Now new media furnish an apparatus in which the body’s expressivity is captured and reconstituted. Here the prosody that the alphabet forsakes returns. This is not in itself a new idea—first generation media theory observes a renaissance in orality as a consequence of new media generally—but Rotman develops it in a new and original way by noticing that it is not just the prosody of speech that returns with new media but also the gesturo-haptic poetics of the hands, head, face, and voice.
Gesture is an aspect of the body’s expressivity that we know even less about than speech but which infuses bodies and moving images with enormous significance. Unlike the alphabet, which segments the continuous stream of speech sounds into discrete quanta of sound (phonemes) and notates them using letters, new media don’t need to detour through symbols. They capture “the entire communicational, instrumental, and affective traffic of the mobile body” and render it reproducible at high resolution (4). When new media capture gestures in the manner of their occurrence rather than simply notate them via symbols a cultural threshold is crossed. In two conceptual jumps Rotman imagines a culture that leapfrogs symbolization all together.
First jump, beyond the written mark: why interpret ‘writing’ as notation, as the projection of body activity (here, speech) onto a pre-set list of inscribed marks and a syntax? Why not an a-symbolic mediation—a direct sampling or capture rather than a coded representation? Second jump, beyond the oral-vocal apparatus: why the restrictions of the movements of the organs of speech, to the physiology and neurology of breathing and its articulation into consonants and vowels? Why not the movements of any and all of the body’s organs and parts, oral, aural, or otherwise, traditionally signifying and a-signifying alike. (41)
Whereas alphabetic characters notate and store speech sounds, motion capture stores and reproduces kinesis. The question arises: “Could motion capture be about to induce a transformation as radical and far reaching for the body’s gestural activities, for its skin and organs of grasping and reaching, as writing accomplished for the organs of speech?” (47) It’s difficult to say. The success of the alphabet was a function of its extreme reductiveness—its ability to distill the manifold facts of speech into a system that a child could learn and use. For motion capture to have the kind of impact that writing did the resultant interface would have to be something both children and adults could use for a broad range of purposes: creative, administrative, juridical, pedagogical. Especially pedagogical. The epic apparatus of orality and the academic apparatus of literacy purposed speech and writing with an educational mission that was crucial to maintaining their status as cultural gateways. Orality and literacy were monopolies.
Could motion capture be a monopoly? That’s an interface question. In a notation system, the code that makes the object and the code that displays it—the creation interface and the user interface—are one and the same, whereas in digital technologies the two codes are different. To learn to read is to know how to write, but viewing a website doesn’t make you a web designer. Will the motion capture interface have the requisite degree of ubiquity and simplicity? Will there ever be just one? How long will it continue to be screen-based? Perhaps there is another template for understanding how the medium will integrate itself into human affairs that has yet to announce itself. Whatever the case, a motion capture interface that was ubiquitous and simple would bring creative, expressive, and playful movement into the practices of teaching, learning, administration, and communication generally. People would dance at business meetings, and children would teach adults how to communicate better.
Still, the prospect of leapfrogging symbolization all together is a little dizzying, perhaps even a little absurd (or is that just the alphabet talking?). What would it mean to by-pass the cultural bottlenecks of writing and speech and embrace a corporeal axiomatic in which the body formed a complete cultural canvas? It’s difficult to imagine. But then comes the realization that, in some ways, we’ve already started to make the transition. The genius of Charlie Chaplin is an early and very persuasive example. Even written language assumes adaptive functions (emoticons, new punctuations marks) that attempt to index the body. The prospect of a revolution from notation to capture is doubtful. Convergence seems more likely.
One thing that does lag behind though is our understanding of the new apparatus and our ability to study it. As Hayles points out, many of the methods we use to understand what is happening in networked environments unwittingly project the old literate hierarchy onto new forms of communication. Writing’s long cultural dominion over gesture and speech means that we have organized all our knowledge around words and symbols and the orders of discourse, representation, and science constituted by them. The principle methods of humanities scholarship—semiotics, hermeneutics, close reading, and critique—are methods for reading culture qua symbol, culture minus embodiment. What methods will we use to read culture in its material form, as embodiment? Is reading even the right word?
In working his way through his topic, Rotman hews to semiotics even as he demonstrates its limitations. Semiotics seems to have been the intellectual enticement that brought Rotman from the UK to the US, and from mathematics to the humanities. As a semiotician, he burned through Saussure very quickly before moving on to Peirce. Becoming Beside Ourselves pushes semiotics to the limit and dramatizes a significant methodological conundrum. Can the rubric of signs formulate embodied meanings? While some gestures (ASL signs for instance) index words and function like conventional signs, many other function as emblems that are not translatable into words, but which still convey meaning. “Evidently, emblem gestures say nothing…. In fact they function at their most characteristic when differentiated and opposed to speech. Unlike words… emblem gestures signify… by virtue of their occurrence as events” (19).
In order to go beyond words to a consideration of gestures Rotman makes an important, though undisclosed, methodological move that takes us from a semiotics of monadic, arbitrary signs (Saussure’s semiology) to a more supple analytic that recognizes more than one order of signs (Piercean semiotics). Unlike Saussure, who built his linguistics around the notion of arbitrary signs that signify solely on the basis of convention, Pierce recognized the validity of signs that signify based on similarity (icons or emblems) and signs that signify on the basis of contiguity (indices). Gestures are emblems that signify on the basis of similarity. Their meaning is inseparable from their being. They mean what they do.
Pierce certainly helps things along, but ultimately semiotics fails as a method for studying gesture. “Gesture is outside the domain of the sign insofar as signs are coded and call for a hermeneutics, an interpretive apparatus separable from, and in place prior to, the act of signification. Rather, the mode of action of gesture is enactive, exterior to anything prior to its own performance: it works through bodily executed events, creating meaning and mathematical significance ‘before one knows it’” (36).
Rotman’s interest in the pre-semiotic, corporeal dimension of technology therefore functions implicitly as both an extension and a critique of semiotics. Without stopping to ponder the matter, Rotman delivers us to a methodological roadblock. By tracking the legacy of alphabeticism from the dual point of view of what it enables (the powerful virtualities and abstractions that populate the metaphysical worldview) and what it disables (the prosody of speaking, gesturing bodies) Rotman extends the question of meaning and significance far beyond the parameters of the sign into the uncharted wilds of a-signifying emblem gestures. Media-specific analysis will need methodological development in order to formulate extra-semiotic aspects of embodiment for as the focus of humanities research migrates from signifying economies to the material economies of media and technology, significant methodological challenges emerge.
In the end, the stakes are much higher than mere method, and Rotman charges forward to the implications.
To speak of the end of the alphabet is to suggest the possibility of a shift in Western deism, a reconfiguration of God and the God-effect as momentous as the alphabet’s inauguration of that Being. If this is so, then the stakes for an end of the alphabet would be high indeed, and, to return to Leroi-Gourhan’s fantasy of post-alphabeticism we started from, we have to wonder if such a thing is feasible; in an end to alphabetic writing or, less totally, a shrinkage in its universality, importance, controlling functionality, and hegemonic status, is thinkable from within that very writing here in the West? (54)
Other, more modest, questions emerge as well. If alphabetic literacy institutes what Rotman calls a “lettered self,” what kind of being and identity will correspond to the digital era of networked, mobile communications? Several aspects of the new media apparatus suggest themselves as models: its language (computer code and markup languages), its products and effects (intermediality, virtual reality), its structure and architecture (the internet), its ethos (collaboration and play), and so on, but Rotman chooses as his model the engine that powers everything: the processor, specifically the recent breakthrough of parallel processing.
Until recently, personal computing has been based on machines with single processors the speed of which would double every two years according to Moore’s law. Now PC makers have switched to “duo-core” parallel processors, less powerful but faster set-ups that break tasks down into pieces and compute the parts simultaneously. Rotman generalizes the serial/parallel duo as paradigmatic for the transition from the lettered self to the networked ‘I’. “Parallelism foregrounds co-presence, simultaneity, and co-occurrence and is exemplified in collaborating, displaying, and networking, while serialism foregrounds linear order and sequence and occurs in counting, listing, lining up, and telling” (83).
The lettered self and its protocols are naturalized to such a degree that we forget of how foreign it is for our brains to read. “We were never born to read,” Marianne Wolf writes. “Human beings invented writing only a few thousand years ago. And with this invention, we rearranged the very organization of our brain, which in turn expanded the ways we were able to think, which altered the intellectual evolution of the species” (3). It is a quirk of human history that to advance as a species we had to invent a form of serial processing (writing) that greatly undersells the brain’s parallel processing abilities.
If the human brain is built for parallel processing and now computers are too, the resulting compatibility could be momentous. “This porting of parallelism into thought and selfhood encounters a parallelism already present, long before any engagement with machine computation, consisting of many layers of simultaneous activity of the body from the cellular level to the organization of the central nervous system. A parallelist psyche, then, will be as much an intensification of these existing parallelisms as it is a computational planting into a self that knows nothing of such things” (92). In a rather witty synthesis, Rotman’s model of the technoid subject invokes the latest developments in computing and the age-old functionality of brain’s organic structure. We are “natural born cyborgs,” he reminds us, an amalgam of biology, culture, and technology. What we understand as mind, meaning, consciousness, culture, subjectivity and so on is put into form not only by language and the orders of discourse and representation that attach to it, but also by physiology and technology.
It seems unlikely that humans will forsake the alphabet any time soon. Eventually the keyboard will be trumped by voice recognition as an input device, but even then computers will still store and present text alongside images and sounds. Besides, notation isn’t inferior to capture in every way. You can’t really skim a sound file, for example. The story of a post-alphabetic future is really the story of what is happening now, the formation of a new cultural apparatus in which the alphabet is just one aspect, and not the most important one, within a new media ecology. But the revolutionary profile of new media should not overshadow their status as sites for convergence either. At least as exciting as the ability of networked environments to emancipate the lettered self from the reductive, disembodied, low res matrix of print literacy is the opportunity to marry systems of abstract notation and the cognitive efficiencies they achieve to the creative thinking we perform with our bodies.
Rotman’s use of parallel processing as a heuristic for understanding the agency of new “computational affordances” and the effects they have on subjectivity and culture is very valuable. It aptly formulates and gives a material basis for the ability of individuals to create and distribute multiple forms of agency, intelligence, and presence and to participate in networks of collective cognition. It performs significant theoretical work too, implicitly disabling the humanist, literate fiction of an internal self and an external other.
The serial protocols of literacy aren’t the only things disabled by new media. The ability to produce, combine, and manipulate digital images also discombobulates Renaissance perspectivalism and with it the humanist construction of a singular, situated point of view. “Now such objects [multi-image assemblages], imaged images, have become a default contemporary visual paradigm which, by presenting many images simultaneously within a single optical act, calls for a visual self engaged in a mode of parallel rather than serial seeing. The result is a form of visual polyphony with sampled images as voices…” (98). In networked environments de-framed and mobile images and identities interanimate with heterogeneous actions and perceptions in an expanded field predicated on the simultaneities of parallel processing.
Depending on your point of view, the implications of Rotman’s study are either repulsive or intoxicating. For me, it’s the latter. Motion capture technology combined with a user-friendly and simple interface will revolutionize education, making it possible to see the average child’s need and gift for movement as a vital part of a creative learning apparatus. Achievements of this kind so far have been astounding. For example, the composer, inventor, and educator Tod Machover of MIT has invented new musical instruments that capture and translate the untrained motions of their users into musical sounds, which can then be transcribed into musical notation and played by professional musicians. Another of his projects is a computer interface called Hyperscore that teaches children and adults with no musical training how to compose music by simply drawing lines on a screen. In both cases, motion capture technologies harness the gesturo-haptic poetics of the moving body and link it to an existing notation interface. What will they think of next? And will it be the university of the corporation that takes the lead?
It might be a race because commercial applications of new technologies will also have a large impact. The Nintendo Wii uses motion capture sensors to animate game functions that bring the whole body into play. As the number of sensors multiplies they will come to be housed in a suit. Educational apps and interfaces will, one hopes, follow. The boundary between education and entertainment will continue to blur, and the answer to the all-important question Will children be able to use it? will be obvious. Children will not only be able to use it, they will be the vanguard users. In place of an educational apparatus that hazes children with the violence of literacy, we might, if we’re lucky, end up with an apparatus that channels and legitimizes the creative genius of embodiment and combines capture with notation in a way that changes our understanding of learning and intelligence. One thing seems certain: the ability to more fully activate the expressive and communicational potential of bodies will greatly alter human culture. In Rotman’s view, the result would be nothing less than “an alteration in the conditions of possibility of being human” (49).
Scott Pound is Associate Professor of English at Lakehead University. His research examines the intersection of poetics and new media in the twentieth century. He is currently at work on a project called “The Poetics of Intermediality,” which studies cultural impacts of new media through the lens of twentieth-century vanguard poetics.
Dolar, Mladen (2006) A Voice and Nothing More. Cambridge: MIT.
Hansen, Mark (2000) Embodying Technesis: Technology Beyond Writing. Michigan: University of Michigan Press.
Hayles, N. Katherine (2004) “Print is Flat, Code is Deep: The Importance of Media-Specific Analysis” Poetics Today 25.1: 67-90.
Rotman, Brian (1993) Ad Infinitum … the Ghost in Turing’s Machine Taking God out of Mathematics and Putting the Body Back in. Stanford: Stanford University Press (1993)
————- (1989) Signifying Nothing: The Semiotics of Zero. Stanford: Stanford University Press.
Wolf, Maryanne (2007) Proust and the Squid: The Story and Science of the Reading Brain. New York: Harper/Perennial.