Designing Augmented and Mixed Reality experiences from UX
The augmented reality (AR) is not a new technology, but it is a discipline that is still under construction from an experience design point of view. Unlike other more mature digital environments, such as web or mobile, design for AR and mixed reality still lacks solid patterns, widely adopted principles and clear frameworks from UX.
AR amplifies the real world by adding a digital layer of information, context and interaction. Its potential is enormous: education, retail, entertainment, healthcare, industry, architecture. However, designing for AR isn't just about placing 3D objects in space; it requires rethinking how people perceive, understand and interact with information in a physical environment.
In my experience, many users are still unfamiliar with augmented reality and often confuse it with virtual reality. During almost five years working on UX and UI experiences for AR platforms and projects -very closely with the Goopy Augmented Reality ecosystem- I've learned that the biggest challenge is not technological, but rather cognitive and experiential.
Today, with the evolution of ARKit y ARCore, the arrival of mixed reality on devices such as Meta Quest 3, and the launch of Apple Vision Pro, the role of the UX designer becomes even more relevant. We are entering a stage where design ceases to live exclusively on flat screens and begins to coexist with the physical world.
This article is not intended as a technical guide, but rather as a practical and strategic introduction to the concepts, formats and tools that a UX designer should know to start working in AR and MR.
Thinking about AR in terms of formats, not software
One of the first mindset shifts when designing for AR is to understand that formats matter as much as tools.
In 2024, traditional formats such as OBJ or FBX are no longer ideal as the primary standard for AR experiences. Today, the ecosystem is leaning towards formats optimized for performance, portability and real time.
- glTF / GLB have become the standard for efficient 3D models, especially in web, AR and immersive experiences.
- USDZ is key within the Apple ecosystem, especially for iOS and visionOS experiences.
- Video with Chroma Key (MOV / MP4) allows the integration of holograms, people and animations that provide narrative and emotional context.
- PNG with transparency remains an essential resource for overlays, floating UI and lightweight graphic elements.
From UX, this translates into a key question:
What level of visual fidelity does this experience really need to meet its objective?
Not everything has to be hyper-realistic 3D; often, simple communicates better.
AR frameworks: understanding the context, not the code
A UX designer does not need to master the code of AR frameworks, but he/she does need to understand what each platform can and cannot do.
- ARKit defines much of the AR experience on Apple devices.
- ARCore sets the standards in Android.
- Unity remains the bridge between design, interaction and cross-platform development.
Knowing these platforms allows for better design decisions, anticipating technical limitations and designing experiences that are more coherent with the environment where they will live.
Tools and resources that expand UX thinking
Designing for AR doesn't happen in a single tool. It is an ecosystem.
- Sketchfab allows you to explore and understand how 3D models behave in real time.
- Vectary democratizes the creation of 3D scenes without the need for code.
- Adobe After Effects brings motion, narrative and life to the AR elements.
- Figma (visionOS resources) marks the beginning of spatial interface patterns.
- WebGL and Three.js open the door to AR experiences directly in the browser.
From the UX point of view, these tools are not the end, but the means to prototyping, validating and storytelling in three-dimensional spaces.
Strategic conclusion: UX beyond the screen
Designing for augmented and mixed reality is not simply an evolution of traditional digital design; it's a paradigm shift.
In AR, the user does not enter the experience:
experience enters the user's world.
This forces UX designers to rethink fundamental concepts:
- The interface is no longer always visible
- The physical context matters as much as the digital context
- The user's attention is limited and shared with the real environment.
- Visual overload can break the experience in seconds
The real value of the UX in AR is not in the technology, but in the know when to intervene and when to disappear. The best augmented experiences don't feel invasive; they feel natural, useful and almost invisible.
We are at an early stage. There are no definitive rules, but there is a great opportunity for UX designers to actively participate in the construction of these new interaction languages.
This is just the beginning.
And in UX Yeah, the conversation is just getting started.
