XR technologies will make it possible for humans to interact with AI agents embodied in both physical and virtual environments. This could enable powerful new forms of AI-human collaboration but will require major advances in both the fundamental capabilities of agents and their ability to interact smoothly with the user.

Future Horizons:

×××

5-yearhorizon

AI agents become a common XR feature

AI-powered virtual agents that can communicate in natural language and demonstrate spatial understanding become a common feature of XR experiences. AI is able to dynamically enhance the appearance of the user’s 3D representation, or avatar, in virtual spaces to enhance communication and improve social experiences.

10-yearhorizon

XR users enjoy an always-on virtual assistant

XR devices featuring embedded AI agents with access to the full suite of sensors become commonplace. This provides users with an always-on virtual assistant that can help with everyday tasks in the real world and augment their perception with useful information and helpful virtual elements. The AI can even assist with real-time control of the user’s avatar in virtual spaces, making movements more graceful or modifying their speech to make it more fluent.

25-yearhorizon

Users’ everyday reality is powered by XR

The use of AI-powered XR devices becomes so ubiquitous that the technology is no longer seen as a virtual assistant but instead as a natural part of the user. AI mediates people’s perception around the clock, making it a fundamental component of their everyday reality. These tools increasingly exert control over the way people behave in the real world as well as in virtual spaces.

The rapidly improving reasoning and decision-making capabilities of large language models now allow them to take control of simulated bodies in virtual environments to solve a variety of tasks.⁴³,⁴⁴ The ability of these embodied “agents” to communicate in natural language also enables them to collaborate with each other, as well as with humans, to tackle challenges.⁴⁵ This is opening up the prospect of populating virtual environments with interactive AI characters for both entertainment and to act as digital co-workers.⁴⁶ Integrating agents into XR devices and providing them with access to their suite of sensors could also enable always-on digital assistants that see the world through the user’s eyes.⁴⁷

Considerable challenges remain. Even today’s most powerful models exhibit only limited embodied intelligence,⁴⁸ and our understanding of how AI and humans interact remains rudimentary.⁴⁹ Evidence suggests that creating useful digital assistants is challenging, and poorly designed AI can often hinder rather than help.⁵⁰,⁵¹

Nonetheless, the possibility raises a host of practical and ethical questions. Should AI agents be embodied in human form, and how will their appearance and personality reinforce social biases? Will users control their own agents, or will they be services provided by companies? Should agents be able to augment how others perceive the user in virtual spaces, adjusting their appearance, movements and even speech? And if agents can digitally alter the user’s XR environment to boost immersion or enhance productivity, how do we ensure this doesn’t lead to deception or manipulation?⁵²

Human-AI interaction - Anticipation Scores

The Anticipation Potential of a research field is determined by the capacity for impactful action in the present, considering possible future transformative breakthroughs in a field over a 25-year outlook. A field with a high Anticipation Potential, therefore, combines the potential range of future transformative possibilities engendered by a research area with a wide field of opportunities for action in the present. We asked researchers in the field to anticipate:

The uncertainty related to future science breakthroughs in the field
The transformative effect anticipated breakthroughs may have on research and society
The scope for action in the present in relation to anticipated breakthroughs.

This chart represents a summary of their responses to each of these elements, which when combined, provide the Anticipation Potential for the topic. See methodology for more information.