A trans-disciplinary research team, led by Alexander Klippel, professor of geography and Gosnell Senior Faculty Scholar, received one of 10 seed grants to pilot programs that support Penn State’s 2016–2020 Strategic Plan. The proposal is titled, “Digital Innovation through Immersive Technologies: Establishing New Paradigms for Environmental Decision Support.”
The project primarily supports the thematic priority, Driving Digital Innovation, and also addresses Advancing the Arts and Humanities, Stewarding our Planet’s Resources, and Transforming Education. The goal is to demonstrate the potential that immersive technologies offer for all academic disciplines, but with a focus on environmental communication.
Klippel has been working with immersive technologies for several years, launching projects within ChoroPhronesis and collaborating with faculty across Penn State and other institutions. “There is a network of faculty here at Penn State who are enthusiastic about immersive technologies. And that fits nicely with the strategic plan.”
“Just take a moment to realize that we can provide access to any place on earth at any point in time from the bottom of the ocean to boreal forest in central Asia, from the height of Mayan civilization to the future of Centre County under various climate change scenarios,” said Klippel. “We‘ll be able to deliver fully immersive, embodied experiences that interactively allow for accessing all information available about a place, and understanding it in context. Through expanding physical reality to open people‘s minds, we can facilitate adopting critical, previously impossible perspectives.”
Klippel said that his lab has been using virtual reality for a long time for behavioral experiments because it allows for controlling critical environmental characteristics. “And in 2014 it became clear that virtual reality would become a medium for mass communications.”
S. Shyam Sundar, distinguished professor co-director of Media Effects Research Laboratory, in the Donald P. Bellisario College of Communications, has also been conducting research along these lines. “I see this modality of communication growing by leaps and bounds in the next decade. It will soon dominate the way we experience media. Just like television ushered in a new way of experiencing the real world compared to simply reading about it via newspapers or hearing about it on the radio, VR will change the game.”
The term “xR” is now being used to represent the continuum of the concepts and technologies, from augmented reality to mixed reality to virtuality, Klippel said. “xR is not new,” he said. “What is new is the fact that every technology company (e.g., Google, Apple, Samsung) is investing heavily in it and creating a new industry that will transform every aspect of human life.”
“Immersive virtual technology has developed in lightning speed in the past five years, partly due to advances in understanding the important spatial visual features in the environment and the industry’s ability to implement this understanding into their technology,” said Ping Li, professor of psychology, linguistics, & information sciences & technology, co-director, Center for Brain, Behavior, and Cognition, and associate director, Institute for CyberScience.
Klippel describes how xR can be used for teaching. “If you want to teach your students about the structure inside a volcano or the ecology on the bottom of the ocean or the gravity on the surface of Mars, you can assign a reading or show a video; but now, you can also take them on a virtual field trip and give them an immersive, embodied experience that is tremendously more powerful and empirically proven to foster understanding and learning,” Klippel said. “It is a medium that allows almost a full sensory immersion. It also allows for exploring at completely different scales yet still with all the benefits of an embodied experience. Think of the films, “Fantastic Voyage” (1966) or “Ant-man” (2015). An xR lesson can allow the student to shrink to the size of a molecule or become a giant who lifts the top off a mountain to see the geologic structure inside.”
Sundar’s research on has shown the implications of this. “Immersive technologies are changing the nature of media by introducing a new modality of communication, namely a virtual space to which users can be transported and allowed to explore on their own. Unlike moving a keyboard button or joystick, users are able to explore the space by turning their heads and waving their arms. Our research has found this to be an effective way of telling stories about distant lands and peoples, by giving users a greater of sense of ‘presence’ and a feeling of ‘being there’ in the portrayed environment. We found that VR also increases empathy for characters in the stories.”
Immersive technologies are not only improving education and research, they are also becoming a critical part of any kind of training from fork lifters to submarines, Klippel said. They allow for reducing stage fright and treating PTSD. “Ultimately they will be able to permeate our daily lives, allowing for better decisionmaking from selecting furniture, hairstyles, but also for planning for sustainable communities and understanding climate change at the very local level.”
Klaus Keller, professor of geosciences, see this application in his research as well. “Immersive technology has already made important differences in my research. For example, I would use it with high dimensional data sets that characterize trade-offs between different objectives in the design of climate risk management strategies. Understanding and communicating these trade-offs is very challenging when using the standard two-dimensional projections. Using immersive technology, in collaboration with researchers from Dr. Klippel’s research group, enabled important new insights that have informed new research directions and the design of decision support systems.”
Some of the pilot project goals include creating a prototype immersive virtual reality theatre; developing a repository of knowledge on immersive technologies; investigating and disseminating research-based best practices for using xR in research, education, and service.
“I think this project will help the University see the potential of this technology and will indeed change every aspect of academic life from teaching to service to outreach,” Klippel said. “As a seed grant, it is meant to bring people together with the goal to write bigger and bigger grants, and that is already happening. There are a lot of funding opportunities.
What is the future of immersive technologies or “xR”? Keller and Li said they anticipate combining it with big data to transform teaching, learning, research, and decisionmaking.
More information about all the funded proposals and their authors is available at http://strategicplan.psu.edu/funded-initiatives.