Apple's move into an entirely new category was launched yesterday at their Worldwide Developers Conference with the announcement of their Apple Vision Pro mixed reality headset.
The $US3,499 device is designed to offer virtual and augmented reality experiences, or what Apple calls "spacial computing".
Today, at Apple's headquarters in Cupertino, I was given the opportunity to experience the Apple Vision Pro for the first time in a controlled demonstration.
READ MORE: Australian accused of drunken rampage in Indonesia released from jail
Apple constructed an entirely new temporary building on their Apple Park campus for the demonstrations to select media, sitting open and overlooking the campus football field next to their basketball courts and fitness centre.
It was, in true Apple style, excessive and impressive all at the same time.
Getting the right fit for Vision Pro
My experience started like it would for you if you purchase the Apple Vision Pro - a fitting and lens check.
Using an app on an iPhone, I scanned my face and head in the same way you might know setting up FaceID on an iPhone works if you've ever done that.
Then, as someone who wears glasses, a consultant took a measurement of my lenses to ensure they could fit the correct Zeiss optics lenses to the Vision Pro.
For buyers, this experience will most likely happen within an Apple retail store in the same way it did for me.
However if you purchase online, you'll use the Apple Store app to provide the fitting details, and your vision prescription will be required to get Zeiss to ship the correct lenses to you - these come at an additional cost to the headset.
Understanding the Apple Vision Pro device
The device in the hand carries some weight, it is after all a computer you wear on your face.
There are just two buttons, a physical button on the left top side, and a digital crown on the top right-hand side.
The button on the left is for capturing spacial photos and videos, while the digital crown is primarily your home button and a dial to dial up and down the level of immersion in some experiences.
READ MORE: 'Look through, not at': Apple unveils new kind of virtual reality headset
Apple Vision Pro initial setup
Once you put on the device, you need to get a comfortable fit, adjusting the headband around the back of your head with a dial to tighten, and the optional head strap across the top of your head as well.
I did re-adjust after 10 minutes of my demo, because I felt like I was looking up to see anything, and a movement of the device up on my head to sit higher on my cheeks made all the difference.
On-screen you get the iconic Apple "hello" graphic, followed by two important setup functions.
You configure the eye-tracking by looking at dots that appear in the space on front of you, while keeping your head straight.
Then you place your hands in front of the device as if pushing against a wall, to configure your hands with the device.
READ MORE: iOS17 brings new features to iPhone including retro call screening
How do you control the Apple Vision Pro
This is where the first impressive difference to most all other "headset" devices kicks in.
You need to think of your eyes as the mouse, and your fingers as the mouse buttons.
The first screen you see is the home screen, looking at apps makes them enlarge, though, I would argue not enough and their highlighted state could be more obvious.
You can look at any icon for an app, or even smaller menu items and it will highlight in the same way as if you had hovered a mouse over them - but there is no graphical pointer on screen, it's just assumed based on where you look.
To "click" you tap your index finger to your thumb and release - so a pinch will not click, it's a tap and clear release that operates as a click of the icon or action.
Scrolling however is perhaps the most intuitive and well-designed action of all, you pinch your fingers together then motion your hand left or right, or up and down to scroll in those directions.
I was only told this once and picked it up instantly - a very intuitive experience.
READ MORE: New data shows ambulance ramping not improving
The user Interface
Perhaps the best demonstration of Apple's work on this device is the overall operating system and user interface.
The home screen apps appear as round icons, but windows launch in a semi-transparent but very familiar design.
Imagine the Mac apps running a grey or silver bar at top, that's semi-transparent and integrates with the room you're in so it doesn't feel like it's just a block in front of you.
Importantly, any window can be moved. At the bottom of any window is a small dot and a line.
The dot when you look at it becomes a cross which you can then click to close the app.
The line is something you can look at then pinch and control.
This is a fantastic experience, moving an app closer to you, further away, put it on a wall or move it to the side.
Like having multiple monitors, you can move apps around you so you look to them when you need them.
READ MORE: Rate hikes, Ukraine war behind sharp slowdown in global economy
Seeing the room around you
When you first put on the Apple Vision Pro after setup, you see the room you are in. But straight up, it's not the same as just sitting there.
Your field of view is narrowed, things to your left and right that you might turn your eyes to see you are needing to turn your head to see.
But of course, that's the way it has to be. Just takes a moment to get used to, and it does take away from the real "reality" of the experience, but - the vision is colour, great resolution, and there is no latency to speak of.
Tapping the table in front of me felt natural seeing the movement and sound together, even listening to someone talk next to me was real-time in terms of what I saw vs what I heard.
In simple terms, it's kinda like putting a filter on the world around you. It's perhaps not as bright as the real world, nor is it as clear - there's a step down in clarity like going from a HD TV to an SD one, but it's certainly extremely high resolution.
Everything you do happens around you in that room, and thanks to the sensors on the device, you can place things onto walls because it knows the boundaries of your space.
READ MORE: Harry claims tabloids destroyed childhood but can't recall specifics
Switching to an immersive virtual experience
To remove yourself from the room you are in, you can take yourself to what Apple calls "Environments". From the home menu you can expand a tool bar on the left to choose environments.
You can choose from a range of "places", which then appear in front of you, but fade away to the room around you on the edges. Until you turn the digital crown, twist it one way that environment reduces and brings more of the room around you in, go the other way you're getting rid of your room and going full 360 on the environment.
For me, the mixed reality concept is far more impressive because we've seen plenty of other headsets that remove you from the world, but the option to choose a level of immersion is impressive.
Things I experienced on Apple Vision Pro
I was able to scroll through photos which frankly look stunning in this virtual big-screen in front of me.
I found it wonderful placing experiences and apps on the back wall of the room, as if to have them on a giant screen in the room.
Panoramic photos shot on iPhone can be experienced in a 180-degree wrap-around too.
Then, spacial photos and videos taken on the Vision Pro appear in front of you in full 3D, a photo feels deep and rich, while a video floats in the space in front of you and when a child blows out birthday candles it was like I felt the breeze.
Wonderful stuff. My only concern with spacial photos is using the headset to take them. I'm not sure I want to be the dad that throws on a headset goggles to take a photo and remove myself from the moment.
I imagine a time when future Apple iPhones allow you to capture these spacial videos and audio.
My one-minute meditation session was relaxing, and I can imagine this being a great use case for many people.
READ MORE: Small-brained ancient human cousins may have buried their dead
An Apple staff member made a FaceTime call to me, and that was a great experience, seeing what was a virtual but realistic representation of her, with the sensors on the device offering realistic impressions of eye and mouth movements.
We were able to share a presentation, putting that up on the wall, while the FaceTime "View" of the other caller was moved to the side as I chose.
I watched the latest Avatar in 3D, and viewed it in both the room I was in, but also immersed in a cinema environment blocking out the wold around me.
Apple also demonstrated a new format of video - "Apple Immersive Video", basically a 180-degree high-resolution experience of any situation you might choose.
The demo had everything from shots on a lake to a woman rock climbing but it was at its most impressive in Sport.
A back of the hoop view of basketball, and a dugout view of baseball had me on the edge of my seat.
Finally, I went into an interactive spatial experience where a butterfly flew through the room and landed on my finger.
Key takeouts from my time with Apple Vision Pro
This is a headset-based reality experience like no other on the market. While many offer immersion in virtual reality and some offer an augmented reality experience, Apple's approach to the augmented is class-leading and offers a true insight into what is possible with this device.
Simple to understand and use navigation using just your regular hands in a normal seated position makes it feel less "silly" and that's key to the whole thing.
But - I don't know what I looked like, and I think the answer would be silly, or nerdy, or just weird.
That's going to be the biggest constraint to grow I think - because the system, sensors, cameras and software are on point and ready to setup a strong future for Apple - if people believe in their vision for the future.
Trevor Long travelled to California as a guest of Apple
