Faculty and Research

The School of Arts, Media and Engineering conducts research on experiential media and systems that impact large-scale societal problems that are invested in improving the human condition.

Home / Content / Research initiatives

Research initiatives

Arts, Media and Engineering Experiential Complex Systems

Experiential Systems

We develop environments within which groups of people can experience simulations of complex dynamical systems of weather, cities and other complex biosocial, physical or symbolic worlds.  Collaborating with experts from relevant scientific domains, we simulate these systems with enough richness and accuracy to not only capture essential dynamics but also to realize them for real-time steering.  A key distinction is that people can feel their way through the simulations by steering the dynamics with gesture and movement or sound, with multi-sensory engagement, using live responsive computational media – moving image, spatialized sound, light fields, physical props, and narrative media.  (See also the Improvisational Environments initiative.)

For more information about this initiative, please contact:
Chris Roberts
Brandon Mechtley
AffiliatesMelissa Bukovsky, Linda Mearns (NCAR); Sha Xin Wei

School of Arts, Media and Engineering Rhythmanalysis: Cities, Organizations, Movement, Media

Rhythmanalysis: Cities, Organizations, Movement. Media

This project studies how these biosocial complex systems evolve, cohere, disintegrate, and most importantly generate novel pattern via a special attention to rhythm. For our purposes, rhythm can be described as the variation of material = energy + matter + media through different biosocial, physical and symbolic states. Understood this way, rhythm is not sonic, it does not have to be regular periodic, indeed it does not have to be unidimensional or any particular dimension at all.

For more information about this initiative, please contact:
Todd Ingalls
Sha Xin Wei
Affiliates: Julian Stein, John MacCallum, Teoma Naccarato
Partners: Adrian Freed (Berkeley); Cumhur Erkut, Dan Overholt (Aalborg)

Learning Evironments

Learning Environments

Learning Environments is a research group focusing on new approaches to education that harness emerging technologies, enable collaborative learning, and prepare students for an increasingly dynamic world. The group has created several projects exploring physical and online learning systems. 

For more information about this initiative, please contact:
Loren Olson
David Tinapple

 

 

 

 

Sensing Systems

Sensing Systems and Algorithms (website coming soon)

The Sensing Systems and Algorithms (SSA) Initiative seeks to model, understand, and build algorithms, software, and hardware for audiovisual sensing. Through solving challenges in the capture, processing, and representation of sensor data, SSA seeks to build platforms for a multitude of media processing applications, including virtual/augmented/mixed-reality and interactive installations. 

For information about this initiative, please contact:
Robert LiKamWa
Pavan Turaga

 

   

Arts, Media and Engineering Sound and Touch

Sound and Touch

This research explores why making use of haptic interfaces—which offer physical feedback and resistance to the performer—may be viewed as an important approach in addressing the shortcomings of some the standard systems used to mediate the performer’s and audience’s engagement with various sorts of digital musical information. 

For more information about the initiative, please contact:
Lauren Hayes
Affliates: Jessica Rajko, Xin Luo

 

Arts, Media and Engineering Enactive Interfaces

Enactive Interfaces

Live music making can be understood as an enactive process, whereby musical experiences are created through human action. This suggests that musical worlds coevolve with their agents through repeated sensorimotor interactions with the environment (where the music is being created), and at the same time cannot be separated from their sociocultural contexts. 

For more information about this initiative, please contact:
Lauren Hayes

Arts, Media and Engineering Improvisational Environments

Improvisational Environments

The main scientific goal of this experimental research on experience is to gain insight into (1) how people can improvise gestures and activity that are meaningful to themselves and each other in environments where the media respond to activity as richly as physical matter, and (2) how people enactively shape events continuously through embodied interaction with computationally augmented, experientially rich media environments. Art production is a means but not the end of this research.  By art, we mean the poetic—and poetically precise—conditioning of experience. This builds on the responsive environments that have been pioneered by the Topological Media Lab and AME since 2001. (NSF CISE/CRI, NEA motione, NSF IGERT , Media Choreography, Ozone)

For more information about this initiative, please contact:
Julian Stein
Sha Xin Wei
Affiliates: Todd Ingalls, Loren Olson, Chris Ziegler

Interactive Media and Sensorimotor Studies

Interactive Media and Sensorimotor Studies

Our team has developed an adaptive, mixed reality rehabilitation (AMMR) system to train reaching and grasping movements of stroke survivors. Mixed reality therapy, combining training in virtual and physical environments, was developed to connect virtual learning to physical reality, and thus better facilitate the transfer of strategies from therapy to activities of daily living. The AMRR system integrates rehabilitation and motor learning theories with high-resolution motion capture and sensing technologies, smart physical objects, and interactive computer graphics and sound. 

For more information about this initiative, please contact:
Todd Ingalls
Pavan Turaga
Claude Ghez

Arts, Media and Engineering Speculative Design

Speculative Cultures (Website coming soon)

The Speculative Cultures transdisciplinary research axis asks a very basic question: How can we generate a culture of speculation, imagination and experimentation in the 21st century?  This axis proposes we can forge a culture of research and creation from the unobserved, the unimagined, and the unthought. To do this, we draw on techniques from cultural theory and philosophy (Nocek), citizen-based science (Kuznetsov), literature and imagination research (Finn), somatics (Coleman), and experimental arts and sciences (Sha).  We adapt and invent methodologies, techniques and discourses suitable for exploring what does not yet exist but may one day. A number of labs, centers, and art/design initiatives are associated with the Speculative Cultures research axis:  Center for Science and the Imagination (CSI);  Laboratory for Critical Technics (LCT);  Social and Digital Systems Group (SANDS);  Synthesis Center

For more information about this initiative, please contact:
Adam Nocek
Ed Finn
Stacey Kuznetsov
Grisha Coleman
Sha Xin Wei

Site Responsive Sonic Art

Site-Responsive Sonic Art

This research is about performances of live electronic music which take place in unusual locations. I use un-usual to refer to buildings and spaces that have not been designated as concert venues, as well as outdoor urban and rural settings. These sites range from retired industrial structures, visually stimulating landscapes, architecture with unique acoustic properties, natural environments, or places of cultural or historical significance.

For more information about this initiative, please contact:
Lauren Hayes
Affliate: Julian Stein

Arts, Media and Engineering Embodied Learning

Embodied Learning

This research explores the role of technology in creating environments for engendering participatory learning which is inclusive. This includes embodied approaches to learning which encourage participation from children of all genders, regardless of their socioeconomic status, or ability.  

For more information about the initiative, please contact:
Lauren Hayes

Arts, Media and Engineering Design Fiction

Design Fiction (Website Coming Soon)

How can we make the future tangible? Design fiction combines making and storytelling to explore the technical, social, and ethical consequences of technological change. Through these visceral experiences of what might be, we can make imagined worlds real and investigate the future as a collaborative, inclusive design space.

For more information about this initiative, please contact:
Adam Nocek
Ed Finn
Stacey Kuznetsov

Arts. Media and Engineering Improvising Cyborgs

Improvising Cyborgs

The practice of improvisation is used as a site to explore the seams and boundaries of technology, social interaction, and collaborative musicking. 

For more information about this initiative, please contact:
Lauren Hayes