A U.S. Office of Naval Research-sponsored lab showcases mixed, virtual and augmented reality technologies
for warfighters, researchers, government, industry and academia to collaborate. (U.S. Navy/John F. Williams)
Augmenting reality versus fixing reality
Now that we can integrate digital information into a user’s environment in real-time — the very definition of augmented reality, or AR — the possibilities seem endless. While the use of AR technology to overlay content onto the real world enriches a user’s perception of reality, it doesn’t replace it.
As our daily lives become ever more digital, we’re experiencing an even greater convergence of the real and virtual worlds with the advance of AR technology.
Augmented reality offers an interactive experience that uses computer-generated perceptual information to enhance the real world. Software, apps, and hardware such as AR glasses overlay digital content onto real-life environments and objects.
The 2023 GESDA Science Breakthrough Radar® tells us that connecting the virtual and physical worlds will require a new global computing infrastructure. Responsive 3D content tied to spatial information requires far more data and processing than the text, images and videos that make up today’s Internet. Over the next quarter century, breakthroughs in the processing power of AR devices means content no longer has to be served from the cloud, reducing infrastructure requirements again.
The Radar also cautions that AR will lead to a blurring of the boundaries between the physical and digital worlds, which could have profound impacts on human psychology and social relations. The reality-distorting effects of social media are a harbinger of what could come, and phenomena like disinformation, echo chambers and body dysmorphia caused by image filters could all be turbocharged by AR technology.
The personalization of AR experiences could ultimately lead to a breakdown in the shared reality that is essential for the smooth functioning of societies. Extensive psychology, neuroscience and social science research will be required to understand how AR will impact humanity and preempt potential pitfalls, the Radar forecasts, and VR simulations of future AR scenarios could provide crucial insights.
Augmented reality, in its present form, could provide a temporary respite from fixing reality. But what happens if it’s no longer possible to differentiate between the artificial and the real? What if AR becomes so effective, that it’s possible to take a walk outside without knowing what’s a projection and what’s part of the environment?
AR technology can bring us the world we imagine. It also can track today’s real world and help us experience it in new ways. In Switzerland, for example, AR is already regularly used to overlay reality with digital geodata. SWISS Airlines launched an augmented reality app for one of its CS300 aircraft to mark a traditional winegrowers’ festival that occurs only once in a generation. More than half the Swiss population have already experienced AR without realizing it, according to the Swiss Augmented Reality Barometer, a study carried out by the University of Lucerne.
Visual and hearing aids are a form of augmented reality that are firmly part of our lives today, and implant technologies could become the norm in future. In 2022, Silicon Valley company Mojo Vision announced the first major third-party consumer application to be tested on the prototype of an AR contact lens with 30 times the pixel density of an iPhone. The contact lens, developed with Amazon’s support, can be used to shop more easily.
AR technology is poised to transform society and people’s lives, particularly in health care and mobility, but more agreement is needed to set guardrails for their ethical uses. The Council of Europe recently launched a strategic action plan on human rights and technologies in biomedicine.
In March, London-based Cromwell Hospital reported becoming the first hospital in the United Kingdom and Europe to use an Apple Vision Pro mixed-reality headset as a logistics and organizational tool. The headset, which runs on AI-based software from augmented reality surgical software firm eXeX, helped surgeons successfully perform two microsurgical spine procedures.
“The software is seamless and has improved efficiency within the Complex Spine team,” said Syed Aftab, a consultant orthopaedic spinal surgeon.
Preserving cultural heritage and urban planning
In April, the U.S.-based AR Alliance announced that Facebook’s parent company Meta had joined as a founding member to help companies and organizations develop augmented reality hardware. “The promise of AR and its potential market are so vast that there is ample room for all our member companies to succeed together,” said Bharath Rajagopalan, an electrical engineer who serves as the alliance’s board chair.
The field of augmented reality application began in the 1990s, evolving with technological advances and developing information and communication technologies. AR adds information to a real environment, improving users’ experience with computer-generated graphics, sounds or touch feedback. AR is now used in everything from health care to education to transportation, but it has particular resonance for preserving cultural heritage.
Virtual and augmented reality technology can help people express themselves in ways beyond what’s possible now in the physical world. AR technology has also been used at UNESCO’s World Heritage Sites to emphasize their cultural, historical, scientific and other aspects of significance.
Sarah Kenderdine, a professor of digital museology at the Swiss Federal Institute of Technology in Lausanne (EPFL), has built breakthrough technologies in curation and cultural heritage. At EPFL, she built the Laboratory for Experimental Museology (eM+), exploring the convergence of imaging technologies, immersive visualization, digital aesthetics and cultural and scientific big data.
“We have been designing and building large-scale immersive virtual reality and augmented reality systems for about 20 years now,” Kenderdine told a GESDA summit panel on augmented and extended reality technologies. “They’re panoramic, hemispheric, augmented, panoptic and they offer us strategies for multi-sensory engagement and give us powerful ways to reformulate narrative.”
Japan’s Hachioji City combined AR technology with 3D city models to help citizens discuss opportunities for land redevelopment. The approach boosted public engagement, drawing people ranging from 20 to 60 years old, during the planned renovation of a public waste facility.
Participants used AR to visualize renovation ideas onto a map during a planning workshop that “received strong positive feedback and attracted a younger demographic than traditional planning events,” the Organization for Economic Cooperation and Development reported in February. “The promises of this new technology have inspired other cities in Japan to experiment with their public planning processes.”
The evolution of visualization methodology from the 18th century (Journal of Big Data)
‘Augmented reality will replace the mobile phone as our primary interface’
The first interactive AR technology came out of a parachuting accident over Dayton, Ohio, in the summer of 1992. Louis Rosenberg, an American mechanical engineer, was testing the Virtual Fixtures platform, the first interactive augmented-reality system that he developed at Wright-Patterson Air Force Base. He developed it to test how well “perceptual overlays” might improve human performance in manual tasks.
“The project began in 1991, when I pitched the effort as part of my doctoral research at Stanford University. By the time I finished—three years and multiple prototypes later—the system I had assembled filled half a room and used nearly a million dollars’ worth of hardware,” he recounted in IEEE Spectrum, published by the Institute of Electrical and Electronics Engineers. “And I had collected enough data from human testing to definitively show that augmenting a real workspace with virtual objects could significantly enhance user performance in precision tasks.”
Rosenberg also had worked in a NASA lab where researchers explored how to enable realistic, immersive simulated worlds. The term ‘virtual reality’ was coined in 1987 by Jaron Lanier, a computer scientist who sold the first VR headsets and wired gloves. Lanier’s work, in turn, was based on research by Ivan Sutherland, a computer scientist who pioneered computer graphics.
“One day at Wright-Patterson, where skydiving engineers tested experimental parachutes, a dummy rig fell to the ground when a parachute failed to open. Rosenberg examined the smashed sensors and cameras from the failed test with the dummy rig. “By some miracle, I was able to piece together two working units from the six that had plummeted to the ground,” he said. “And so, the first human testing of an interactive augmented-reality system was made possible by cameras that had literally fallen out of the sky and smashed into the earth.”
Today, Rosenberg says he believes that VR and AR technology, now referred to as the metaverse, will become an important part of most people’s lives by the end of the 2020s. “In fact, based on the recent surge of investment by major corporations into improving the technology,” he said, “I predict that by the early 2030s augmented reality will replace the mobile phone as our primary interface to digital content.”
The GESDA Radar forecasts the era of the open metaverse will begin about a quarter century from now.
The 10 most active countries in the field of AR in cultural heritage (Applied Sciences)
Where the science and diplomacy can take us
The 2023 GESDA Science Breakthrough Radar®, distilling the insights of 848 scientists from 73 countries, tells us people’s digital experiences have become a central part of their daily lives, thanks to the advent of personal computers, smartphones and social media, and that the real and virtual worlds are poised to become even more tightly enmeshed as augmented reality technology advances.
The findings in the 2023 Science Breakthrough Radar®
Based on the Radar, here’s where we stand in several important areas:
The technology is already finding applications in education, workplace training, and industry. In combination with data streams from the expanding Internet of Things, AR will create an intuitive new layer of information for workers — allowing them to easily and continuously check the status of industrial processes, for example, or providing navigation tools when operating in unfamiliar surroundings.
1.4.1 Augmented reality hardware
AR hardware developments have been spearheaded by advances in microelectronics, sensor technologies and vision equipment, which have also reduced the cost. With 5G availability on mobile phones, AR can be implemented in limited ways on portable devices without the need of expensive eyeglasses and Head-Mounted Displays (HMDs).
Anticipation in a nutshell
5-year horizon: Hardware begins to mature
10-year horizon: AR displays can be worn all day
25-year horizon: AR and VR begin to feel like reality
Early incarnations of AR will be concerned with two fundamental activities: learning about the world and interacting with other people. This will include tasks like overlaying instructions or translations on the real world, or participating in 3D video calls. Near real-time interaction is possible depending on the amount of processing required, and AR has moved from game-playing and applications requiring lower accuracy to precise engineering and medical fields.
Anticipation in a nutshell
5-year horizon: AR overlays for simple information
10-year horizon: Consumer-grade glasses and adaptive user interfaces
25-year horizon: Individually-tailored reality becomes the norm
Connecting the virtual and the physical worlds will require a new global computing infrastructure. Responsive 3D content tied to spatial information requires far more data and processing than the text, images and videos that make up today’s Internet. AR hardware, however, will not be capable of providing this alone. A massive expansion in cloud services will be necessary to render high-fidelity graphics at scale. Faster wireless technology, including 5G, and edge stations that can bring processing closer to the user, will be crucial for ensuring AR content is delivered with low latency.
Anticipation in a nutshell
5-year horizon: Early commercial AR applications emerge
10-year horizon: High-fidelity AR experiences transmitted wirelessly
25-year horizon: Era of the open metaverse begins
AR will lead to a blurring of the boundaries between the physical and digital worlds, which could have profound impacts on human psychology and social relations. The reality-distorting effects of social media are a harbinger of what could come, and phenomena like disinformation, echo chambers and body dysmorphia caused by image filters could all be turbocharged by AR technology. The personalization of AR experiences could ultimately lead to a breakdown in the shared reality that is essential for the smooth functioning of societies. Extensive psychology, neuroscience and social science research will be required to understand how AR will impact humanity and preempt potential pitfalls. VR simulations of future AR scenarios could provide crucial insights.
Anticipation in a nutshell
5-year horizon: Research maps out social implications of AR
10-year horizon: Government regulation begins
25-year horizon: Pervasive AR threatens shared reality
We can now observe and examine learning practices using digital technologies. By gathering and analyzing anonymized data using computer-based vision technology, student-held devices and other tools, researchers are beginning to make sense of the best practices in teaching, and to understand what enables effective learning.
5-year horizon: Frameworks for sensing are established
10-year horizon: Sensing technology goes mainstream
25-year horizon: AI and wearables change the learning experience
5.2.4 Neuroscientific aspects of learning
Although investigations of neuroscience as applied to learning have yet to deliver significant tangible breakthroughs in educational philosophy or practice, there are reasons to continue efforts to understand how the brain functions when learning. A better grasp of the operations behind working memory, executive function, attention, cognitive flexibility, theory of mind, and inhibition, for example, would open up avenues for improving the efficiency and outcomes of education. The role of social factors is also an important subject of investigation here: has evolution equipped us to learn differently in groups as opposed to when we are alone?
5-year horizon: Progress in basic neuro-learning research
10-year horizon: Brain tech comes of age
25-year horizon: Augmented reality accelerates education
Geneva Science and Diplomacy Anticipator
Fondation Campus Biotech
Chemin des Mines 9
1202 Geneva
+41 58 201 02 61
© 2020 – Geneva Science and Diplomacy Anticipator design by colegram privacy policy
Geneva Science and Diplomacy Anticipator
c/o Fondation Campus Biotech
Chemin des Mines 9
1202 Geneva
+41 58 201 02 61