It encompasses several overlapping technologies. These include virtual-reality (VR) headsets that transport wearers to simulated worlds, “augmented reality” (AR) systems that overlay digital information onto a user’s view of the real world via a mobile app or smart glasses, and “mixed reality” (MR) devices that transpose interactive virtual elements into the user’s field of vision.¹ Consumer headsets are already bringing VR entertainment to the masses, with more advanced offerings from providers such as Meta, Apple and Google also providing some MR functionality.²,³ Although prior investments were made by Microsoft and Apple, AR systems have seen limited uptake by the military and industry,⁴,⁵ while simpler AR smart glasses that look like regular eyewear are proving popular with consumers.⁶,⁷

Recent demos of advanced smart glasses by Meta and Google, which squeeze cutting-edge XR functionality into a sleek form factor, show the technology’s potential.⁸,⁹ However, these remain research prototypes, highlighting the field’s long-standing challenges with commercial viability. It seems the commercial players are looking to get feedback from users and also create ecosystems to develop applications for their devices, similar to what we have witnessed for smartphones. Creating capable, comfortable and reasonably priced XR hardware, along with the software and infrastructure to support widespread use, will require huge investment.¹⁰ So far companies have struggled to find mass-scale consumer use-cases that could justify the sums required.

However, advances in AI have given the field a boost. Breakthroughs in spatial computing, gesture recognition and language understanding, all of which are powered by AI, promise to boost the capabilities of XR devices. Simultaneously, AI developers are hunting for devices that can take full advantage of the technology’s rapidly improving visual and language skills — a new field called visual-language is developing very fast. Smart glasses not only allow the hands to be free but also view the world through the wearer’s eyes, opening up the prospect of an always-on AI assistant. Such an assistant will be able to seamlessly answer in any language users’ questions about the physical world the user is interacting with, while providing a personalised digital overlay of information in all modalities and 3D, dynamic, visual augmentations.

The blurring of the boundaries between the physical and digital worlds would entail and hold unknown and potentially enormous implications for society. However, achieving this vision will still require significant hardware advances to balance the competing pressures of performance, cost, and wearability.¹¹,¹² Significant advances also have to be made that make it easy to both author or create and use virtual content for XR devices.¹³ Meanwhile, developers will need to identify more targeted early use-cases to drive the investment needed to make fully fledged XR a consumer technology. Gaming, immersive and hands-on education,¹⁴ manufacturing,¹⁵ athletic coaching and spectatorship,¹⁶ and medical training¹⁷ appear to be promising options, each with its own unique contexts and challenges for uptake.