EDUCATING FOR INTERACTION
By Dan Collins
A first version of this paper was given at the Performative Sites
conference at Penn State in October 2000.
A published version appeared in New Art Examiner in February
2001.
What is interaction? How can we begin to make sense of the avalanche
of educational toys, computer programs, and artworks that claim
to be "interactive?" What would a new pedagogy structured around
the rules of interactivity look like?
------
Some theorists have argued that all "art is
interactive," effectively shutting the door on the discrete contributions
of new digital/computer-based interactivity. Conversely, other
theorists such as Simon Penny have insisted that traditional artworks
such as painting and sculpture are simply "instances of representation"
and as such should not be defined as truly "interactive systems."
For Penny, interactive artworks are "virtual machines which...produce
instances of representation based on real time inputs." (Penny
1996) Still other theorists distinguish between the relationship
of system interactivity (the enabling hardware and software) and the nature of the interaction (the actual exchange--be it aesthetic, educational, political,
etc.) (Hillman et al, 1994). For our purposes here, a high level
of interaction equals "mutual reciprocity"--a state of dialectical
exchange between two or more entities. Ideally, interactive systems--whether
a high tech computer game or a Socratic dialogue--can be tools
for learning providing intelligent feedback that refines and amplifies
user input.
While the demand for "interactivity" is a relatively recent phenomenon
in the arts, the culture at large has long been obsessed with
the idea of machines that learn. The evidence is mounting. From
media spectacles such as Big Blue's defeat of World Chess Champion
Garry Kasporov in May of 1997 to quieter revolutions in teaching
autistic children, computers that master the behaviors of their
users are beginning to find a place in the culture. There is more
than a hint of narcissism in our desire to be personally reflected
in the machines we make. We don't want simply "dumb" tools, we
want "intelligent" machines that respond and learn by interacting
with their owners. Even our cooking appliances and car radios
are "programmable" to reflect individual tastes.
Few art schools provide courses for producing let alone interpreting
or critiquing "interactive artworks." Though the borderline between
the fine arts and other cultural practices (such as science, technology,
and entertainment) is becoming increasingly blurred, it is clear
that the development of "interactive art" is largely dependent
on "non-art" traditions. From a technical and theoretical perspective,
such strange bedfellows as computer gaming, combat simulation,
and medical diagnostics have more in common with much recent digital
and interactive art practice than main stream art history or criticism.
Theorizing this territory is less a matter of mining, say, the
Art Index, and more a matter of conducting systematic research
into areas such as communications theory, human computer interaction,
educational technology, and cognitive science. With this in mind,
it may be helpful to briefly review how other disciplines are
looking at the issues surrounding interaction.
THEORIES OF INTERACTION
"Interaction" is a useful construct in helping to understand
the complex relationships occurring in a computerized learning
environment. Educational technologist Ellen Wagner defines interaction
as "… reciprocal events that require at least two objects and
two actions. Interactions occur when these objects and events
mutually influence one another." (Wagner, 1994)
Wagner points to historical examples of communication theory to
illustrate the move from "one-way" systems of communication to
"multi-directional" systems. C.E. Shannon's mathematical theory
of communication (Shannon, 1948), for example, was a highly linear
engineering model of information transfer involving the one-way
transmission of information from a source to a destination using
a transmitter, a signal, and a receiver. Later theorists built
upon Shannon's model to include the concepts of interactivity
and feedback. It is only recently that truly interactive systems
that support both synchronous and a-synchronous exchanges among
multiple users have been available. Common examples of synchronous
exchange include live satellite uplinks, telephones, and chat
rooms on the Net. E-mail is the prototypical example of an "asynchronous"
exchange system.
Other trends supporting the development of interactive systems
come from research in artificial intelligence and cognitive science.
If "mutual influence" and reciprocity are criteria for true interactivity,
then the system needs to be capable of delivering more than pre-existing
data on demand. Interactive systems need to be able to generate
"custom" responses to input and queries. In short, the system
needs to be smart enough to produce output that is not already
part of the system. Interactivity must be more than following
predetermined prompts to preprogrammed conclusions like in a video
game
While most natural and living systems are "productive" in the
sense of creating new "information," human-made machines that
can respond with anything more than simple binary "yes/no" responses
are a relatively recent phenomenon. To paraphrase media artist
Jim Campbell, most machines are simply "reactive," not interactive.
"Intelligent" machines, being developed with the aid of "neural
networks" and "artificial intelligence," can interact by learning
new behaviors and changing their responses based upon user input
and environmental cues. Over time, certain properties begin to
"emerge" such as self-replication or patterns of self-organization
and control. These so-called "emergent properties" represent the
antithesis of the idea that the world is simply a collection of
facts waiting for adequate representation. The ideal system is
a generative engine that is simultaneously a producer and a product.
"INTERACTIVE" ARTWORKS
Creating an experience for a participant in an interactive artwork
must take into account that interactions are, by definition, not
"one-way" propositions. Interaction depends on feedback loops
that include not just the messages that preceded them, but also
the manner in which previous messages were reactive. When a fully
interactive level is reached, communication roles are interchangeable,
and information flows across and through intersecting fields
of experience that are mutually reciprocal. The level of success
at attaining mutual reciprocity could offer a standard by which to critique
interactive artwork.
Many artists have developed unique attempts at true interaction,
addressing problems of visual display, user control processes,
navigation actions, and system responses. Different works have varying levels
of audience participation, different ratios of local to remote
interaction, and either the presence or absence of emergent behaviors. Moreover, different artistic attempts at interactivity
suggest different approaches to interaction could be used for
diverse kinds of learners in a variety of educational settings.
Understanding experiments with interaction in an art context may
help us to better understand interaction in pedagogical settings.
Carol Flax: Journeys: 1900/2000
Detail showing a viewer turning a page of the book and triggering
a video projection.
Photo credit: Patricia Clark.
Arizona artist Carol Flax has created an "interactive book" entitled,
Journeys: 1900/2000 at the Institute for Studies in the Arts at Arizona State University,
where I am the Interim Director. At the heart of the project is
a reproduction of a 19th century travel album that trades in fragments
of memory, pieces of voyages, and bits of history. It uses single
images from various existing albums, reproducing and recontextualizing
them to create a completely original "voyage." Movement sensing
technology (computerized tracking devices) sense the presence
of a viewer. "Bend sensors" embedded in the pages cause an electrical
signal to be sent to a computer when a page is turned. Video and
audio clips are in turn triggered that support, amplify, or contest
the veracity of the photographic prints of idealized ancient settings
through the simple juxtaposition of contemporary imagery and sound
with historical photographs. For example, a black and white image
of a Middle Eastern market--a classic example of the "exotic"
and the "picturesque"--is overlaid with a video closeup in color
of an orange being passed from the hand of one person to another
in an Arizona back yard. Text is used throughout both descriptively
and ironically to throw into question the truth value of what
we are seeing. The message is deliberately multi-leveled and ambiguous,
but one thing is clear: we are creating a journey in which we
are complicit, not simply voyeurs.
The work enables a unique method of navigating the content and
scores high in providing a seamless encounter between the user
and the subject matter. While the artwork is described as being
an interaction between a single user and a variable content, the
entire set of options remains fixed in the computer's database.
The work does not claim any degree of "reciprocity" between the
object and the user. It does not "learn" the reader's habits.
Therefore, it is not, strictly speaking, interactive. This does
not diminish the project's ability inspire repeated visits and
reward the user with unexpected discoveries. Given its non-linear
organization and randomized sequences of multiple video clips,
each user’s experience of the book is actively engaging and unique.
As a model for a different "textbook" perhaps, the project points
toward a new class of books that are constructed
with the individual user in mind and that respond with some intelligence
to reader's choices. The fact that Flax insists on preserving
the essential kinesthetic aspects of reading--the feel of the
paper, the turning of the pages--implies that certain direct forms
of knowing just cannot be improved upon. However, the use of moving
images, scrolling text, and audio clips that spill beyond the
boundaries of the book have more in common with immersive experiences
such as VR than reading. This is neither a conventional book nor
an over-scaled e-book, but rather a hybrid of traditional forms
and "reactive" hi tech processes.
Christa Sommerer and Laurent Mignonneau: Interactive Plant Growing
(1993)
Austrian-born Christa Sommerer and French-born Laurent Mignonneau teamed up in 1992, and now work at the ATR Media Integration and Communications Research Laboratories in Kyoto, Japan. In nearly a decade of collaborative work, Sommerer and Mignonneau have built a number of unique virtual ecosystems, many with custom viewer/machine interfaces. Their projects allow audiences to create new plants or creatures and influence their behavior by drawing on touch screens, sending e-mail, moving through an installation space, or by touching real plants wired to a computer.
Artist's rendering of the installation showing the five pedestals with plants and the video screen.
Interactive Plant Growing is an example of one such project. The
installation connects actual living plants, which can be touched
or approached by human viewers, to virtual plants that are grown
in real-time in the computer. In a darkened installation space,
five different living plants are placed on 5 wooden columns in
front of a high-resolution video projection screen. The plants
themselves are the interface. They are in turn connected to a
computer that sends video signals from its processor to a high
resolution video data projection system. Because the plants are
essentially antennae hard wired into the system, they are capable
of responding to differences in the electrical potential of a
viewer's body. Touching the plants or moving your hands around
them alters the signals sent through the system. Viewers can influence
and control the virtual growth of more than two dozen computer-based
plants.
Screen shot of the video projection during one interactive session.
Viewer participation is crucial to the life of the piece. Through
their individual and collective involvement with the plants, visitors
decide how the interactions unfold and how their interactions
are translated to the screen. Viewers can control the size of
the virtual plants, rotate them, modify their appearance, change
their colors, and control new positions for the same type of plant.
Interactions between a viewer's body and the living plants determine
how the virtual three-dimensional plants will develop. Five or
more people can interact at the same time with the five real plants
in the installation space. All events depend exclusively on the
interactions between viewers and plants.
The artificial growing of computer-based plants is, according
to the artists, an expression of their desire to better understand
the transformations and morphogenesis of certain organisms (Sommerer
et al, 1998).
What are the implications of such works for education? How can
we learn from this artistic experimentation to use technological
systems to be better teachers? Educators have long recognized
the importance of two-way or multi-directional communication.
Nevertheless, many educators perpetuate the mindset of the one-way
"broadcast"--a concept that harks back to broadcast media such
as radio and echoes the structure of the standard lecture where
teacher as "source" transmits information to passive "receivers."
The notion of a "one-to-many" model that reinforces a traditional
hierarchical top-down approach to teaching is at odds with truly
democratic exchange. In Interactive Plant Growing, Sommerer and
Mignonneau invert this one to many model by providing a system
for multiple users to collaborate on the creation of a digital
wall projection in real time. The system in effect enables a real
time collaboration that takes many diverse inputs and directs
them to a common goal. And this is exactly what good teaching
is. This conceptualization of critical pedagogy has been developed in many different situations, but here is combined with technology that mirrors its structure.
Sommerer and Mignonneau: Verbarium (1999)
In a more recent project the artists have created an interactive "text-to-form" editor available on the Internet. At their Verbarium web site, on-line users are invited to type text messages into a small pop up window. Each of these messages functions as a genetic code for creating a visual three-dimensional form. An algorithm translates the genetic encoding of text characters (i.e., letters) into design functions. The system provides a steady flow of new images that are not pre-defined but develop in real-time through the interaction of the user with the system. Each different message creates a different organic form. Depending on the composition of the text, the forms can either be simple or complex. Taken together, all text images are used to build a collective and complex three-dimensional image. This image is a virtual herbarium, comprised of plant forms based on the text messages of the participants. On-line users help to not only create and develop this virtual herbarium, but also have the option of clicking on any part of the collective image to de-code earlier messages sent by other users.
Screen shot of the Verbarium web page showing the collaborative
image created by visitors to the site.
The text to form algorithm translated "purple people eater" into
the image at the upper left.
This image was subsequently collaged into the collective "virtual
herbarium."
In both the localized computer installations and web-based projects
realized by Sommerer and Mignonneau, the interaction between multiple
participants operating through a common interface represents a
reversal of the topology of information dissemination. The pieces
are enabled and realized through the collaboration of many participants
remotely connected by a computer network. In an educational setting,
this heightened sense of interaction needs to be understood as
crucial. Students and instructors alike become capable of both
sending and receiving messages. Everyone is a transmitter and
a receiver, a publisher and a consumer. In the new information
ecology, traditional roles may become reversed--or abandoned.
Audience members become active agents in the creation of new artwork.
Teachers spend more time facilitating and "receiving" information
than lecturing. Students exchange information with their peers
and become adept at disseminating knowledge.
Ken Rinaldo: Autopoiesis (2000)
Overview of all fifteen robotic arms of the Autopoiesis installation.
Photo credit: Yehia Eweis.
A work by American artist Ken Rinaldo
was recently exhibited in Finland as part of "Outoaly, the Alien
Intelligence Exhibition 2000," curated by media theorist Erkki
Huhtamo. Rinaldo, who has a background in both computer science
and art, is pursuing projects influenced by current theories on
living systems and artificial life. He is seeking what he calls
an "integration of organic and electro-mechanical elements" that
point to a "co-evolution between living and evolving technological
material."
Rinaldo's contribution to the Finnish exhibition was an installation
entitled Autopoiesis, which translates literally as "self making."
The work is a computer-based installation consisting of fifteen
robotic sound sculptures that interact with the public and modify
their behaviors over time. These behaviors change based on feedback
from infrared sensors which determine the presence of the participant/viewers
in the exhibition, and the communication between each separate
sculpture.
The series of robotic sculptures--mechanical arms that are suspended
from an overhead grid--"talk" with each other (exchange audio
messages) through a computer network and audible telephone tones.
The interactivity engages the participants who in turn effect
the system's evolution and emergence. This interaction, according
to the artist, creates a system evolution as well as an overall
group sculptural aesthetic. The project presents an interactive
environment which is immersive, detailed, and able to evolve in
real time by utilizing feedback and interaction from audience
members.
What are the pedagogical implications for systems such as Autopoiesis
that exhibit "emergent properties?" Participant/learners interacting
with such systems are challenged to understand that cognition
is less a matter of absorbing ready made "truths" and more a matter
of finding meaning through iterative cycles of inquiry and interaction.
Ironically, this may be what good teaching has always done. So
would we be justified in building a "machine for learning" that
does essentially the same thing that good teachers do? One argument
is that by designing such systems we are forced to look critically
at the current manner in which information is generated, shared,
and evaluated. Further, important questions are surfaced such
as "who can participate"; "who has access to the information;"
and "what kinds of interactions are enabled?" The traditional
"machine for learning" (the classroom) with its single privileged
source of authority (the teacher) is hardly a perfect model. Most
of the time, it is not a system that rewards boundary breaking,
the broad sharing of information, or the generation of new ideas.
It IS a system that, in general, reinforces the status quo. Intelligent
machines such as Rinaldo's Autopoiesis can help us to draw connections
between multiple forms of inquiry, enable new kinds of interactions
between disparate users, and increase a sense of personal agency
and self-worth. While intelligent machines will surely be no smarter
than their programmers, pedagogical models can be more easily
shared and replicated. Curricula (programs for interactions) can
be modified or expanded to meet the special demands of particular
disciplines or contexts. Most importantly, users are free to interact
through the system in ways that are suited to particular learning
styles, personal choices, or physical needs.
IMPLICATIONS FOR ART AND EDUCATION
Interactive artworks of the future will enable interactions that
are at once personal and universal. These interactions will be
characterized by a subtle reciprocity between the body and the
natural environment, and an expanded potential for self-knowledg
and learning. Truly interactive experiences are already empowering
individuals (consider the "disabled" community or autistic learners,
for example).
Returning to various theories of interaction (particularly those
of Ellen Wagner), several recommendations for artists emerge that
begin to trace a trajectory for the education of the interactive
artist. They include training on and empowerment with various technologies; understanding media-structured feedback loops (1) and methods
for enhancing "mutual recriprocity"; rethinking where meaning is constituted (cognitive theory is
now suggesting that "meaning" is seen as something that happens
between rather than inside individuals); and redefinition of the roles of educators and learners. Rapid
evolution in the art profession as a whole is creating changes
in the definitions and roles played by art teachers and prospective
artists.
There is no question that the uses of technology outlined here
need to be held against the darker realities of life in a hi-tech
society. The insidious nature of surveillance and control, the
assault on personal space and privacy, the commodification of
aesthetic experience, and the ever-widening "digital divide" between
the technological haves and have nots are constant reminders that
technology is a double edged sword.
But there is at least an equal chance that a clearer understanding
of the concept of interaction--specifically interaction enabled
by technology--will yield a broader palette of choices from which
human beings can come together to create meaning. In watching
these processes unfold, educators will surely find new models
for learning.
_____________
Notes
(1) "The feedback loop is perhaps the simplest representation
of the relationships between elements in a system, and these relationships
are the way in which the system changes. One element or agent
(the 'regulator' or control) sends information into the system,
other agents act based upon their reception/perception of this
information, and the results of these actions go back to the first
agent. It then modifies its subsequent information output based
on this response, to promote more of this action (positive feedback),
or less or different action (negative feedback). System components
(agents or subsystems) are usually both regulators and regulated,
and feedback loops are often multiple and intersecting (Clayton,
1996, Batra, 1990)." (Morgan, 1999)
References
Flax, Carol. (2000).
URL: http://www.arts.arizona.edu/cflax/
Hillman, D., Willis, D.J. & Gunawardena, C.N. (1994). Learner-interface
interaction in distance education: An extension of contemporary
models and strategies for practioners. The American Journal of Distance Education, 8(2).
Huhtamo, Erkki (1993). Seeking deeper contact: interactive art
as metacommentary.
URL: http://www.ccms.mq.edu.au/mus302/seeking_deeper_contact.html
Moore, M. (1989). Editorial: Three types of interaction. The American Journal of Distance Education, 3(2), 1-6.
Morgan, Katherine Elizabeth (1999). A systems analysis of education
for sustainability and technology to support participation and
collaboration. Unpublished Master's Thesis at the University of
British Columbia. http://www.interchange.ubc.ca/katherim/thesis/index.html
Penny, Simon (1996) Embodied agents, reflexive engineering and
culture as a domain. p. 15. (talk given at the Museum of Modern
Art in New York City, May 20, 1996)
URL: http://adaweb.walkerart.org/context/events/moma/bbs5/transcript/penney01.html
Penny, Simon. (2000).
URL: http://www-art.cfa.cmu.edu/Penny/works/traces/Tracescode.html
Rinaldo, Ken (2000).
URL: http://ylem.org/artists/krinaldo/emergent1.html
Shannon, C.E. (1948). A mathematical theory of communication,
Bell System Technical Journal, vol. 27, pp. 379-423 and 623-656, July and October. URL: http://cm.bell-labs.com/cm/ms/what/shannonday/paper.html
Sommerer, Christa and Laurent Mignonneau (1998). Art as a living
system. Leonardo, Vol. 32, No. 3., pp. 165-173.
Sommerer, Christa and Laurent Mignonneau (2000).
URL: http://www.mic.atr.co.jp/~christa
Wagner, M. (1994). In support of a functional definition of interaction.
The American Journal of Distance Education, 8(2).
Wilson, Stephen. (1993). The aesthetics and practice of designing
interactive computer events.
URL: http://userwww.sfsu.edu/~swilson/papers/interactive2.html
-----------
Dan Collins
Institute for Studies in the Arts
Arizona State University
Tempe, AZ 85287-3302
USA
telephone: 480-965-0972
dan.collins@asu.edu