Vison Pro pioneers a new class of computing devices that blend the digital world into the real world, enabling augmented reality (AR). This device is compatible with various software of iOS and iPadOS, can work, entertain, shoot spatial videos, and can interact with only hands, eyes, and voice. Apple calls this new computing paradigm “spatial computing.”
New Paradigm: Hand-Eye Voice Interaction + Eyesight
Vision Pro is a mixed-reality device that integrates AR and VR. Users can play games, watch movies, work, and experience the functions of VR in a fully immersive way. They can also use the sensors on the surface of the headset to view people and objects in the outside world. Project into the virtual world to realize the AR function.
Each generation of Apple devices has its unique interaction innovation. For example, the Mac has a mouse, the iPod has a Click Wheel, and the iPhone has a multi-touch. The interactive innovation of the Vision Pro lies in the development of a controller that eliminates the need for contact with controllers and additional hardware. mode of interaction.
There is no need for a handle for interaction, only the cooperation of eyes and hands, and the movement of the eyes to select the module you want to interact with, such as an app you want to open, and the operation can be completed with small movements of the hands.
Most iOS and iPadOS applications can be used in Vision Pro, such as Safari browser, Keynote, etc. Apple also said it will create a new special version of the App Store just for Vision Pro, which will be the primary way to download new AR and VR software.
In addition, in order not to isolate users from the people around them, Apple has also developed a feature called Eyesight, which is another basic interaction design of Vision Pro. When someone is nearby, the Vision Pro’s black shell can become transparent, allowing those around you to see the user’s eyes and make those around you aware of what the user is paying attention to.
For example, when viewing an app, people around you will know that you can see them. But at full immersion, the mask becomes a blur and people around you know you can’t see them. This includes letting people see you taking photos and videos when taking photos and videos with Vision Pro.
It is worth mentioning that Vision Pro is Apple’s first personal device equipped with a 3D camera, which can take panoramic photos and panoramic videos and play them on the device. This device can also be used in office scenarios, such as opening multiple monitors to work in a virtual space or having remote meetings with colleagues. You can also watch movies at home or on the plane and enjoy a private theater.
It is worth mentioning that there is a top knob on the top of the headset, similar to the side knobs of the Apple Watch, which can be used to adjust the size of the virtual space. In this way, even if you are in a small space, you can feel like you are in a vast world.
What Kind of Headset Costs $3,499?
For the first-generation spatial computing platform, Apple developed many “whimsical” functional designs. To realize these designs, Apple spent a lot of money on hardware equipment, which also kept the manufacturing cost and selling price of Vision Pro high.
In terms of overall appearance, Vision Pro looks like a pair of ski goggles. The headset has a whole glass front panel and an aluminum alloy frame, which contains sensors, cameras, two 23-megapixel displays, and a cooling fan. Comes with an external battery and a new magnetic charger.
The mask and band are cloth-lined and modular, and Apple says they can bend to fit a variety of face and head shapes. The threaded headband can be exchanged for different sizes and styles. Due to the ergonomic design that fits the face, Vision Pro cannot be used directly with glasses. To solve this problem, Apple and Zeiss collaborated to develop custom lenses that can be magnetically attached to the lens. There is a knob “Digital Crown” on the top for users to control the depth in VR or AR, similar to using the crown on AirPods Max to control the volume. The external battery provides 2 hours of battery life and is connected via a “flexible braided cable” that can be placed in your pocket when in use.
In terms of display, Apple says its display will be clearer than ever. Vision Pro is equipped with 12 cameras and is equipped with a Micro OLED screen with 23 million pixels. The pixels assigned to each eye exceed that of a 4K TV, enabling viewing on a 100-inch screen, supporting 3D video viewing, and providing a cinematic video viewing experience.
Of course, there is also standard spatial audio. Vision Pro is equipped with the most advanced spatial audio system. There are a set of dual-drive units on each side of the ear. The sound surrounds the three-dimensional sound. It also analyzes the situation in the space to match the best audio effect.
Dual-chip design: Vision Pro adopts a dual-chip design, including a Mac-level M2 chip and a real-time sensor processing chip R1. The R1 chip is mainly responsible for sensor signal transmission and processing. Officials say that the transmitted data can be reached in 12 milliseconds, which is faster than the blink of an eye and almost eliminates delay.
VisionOs
Vision Pro’s new vision operating system will feel familiar yet unfamiliar, built on decades of engineering innovation in macOS, iOS, and iPadOS.
Vision Pro is compatible with Microsoft apps, and iOS and iPadOS apps are available in both visionOS and Apple Vision Pro. But the difference is that these icons have come to space.
In the demo screen, the interface design inside the device looks like it is actually in the room where you are. Users can switch between multiple different interfaces in the space, place any screen in front of them, and adjust it freely within the viewing distance.
The Vision Pro does not come with a controller but is operated via voice, eye tracking, and gestures. Users can browse app icons by looking at them, tap to select, swipe to scroll, or issue voice commands. They can also enter text using a virtual keyboard.
Additionally, users can connect their Mac via Bluetooth and expand its display in a virtual space. In other words, in visions, there are no restrictions on the boundaries of the monitor. The space in front of you is the monitor. After mirroring the Mac with an external keyboard and mouse, you can even cut videos directly into the space.
Users can also project 3D objects into real space, including pulling objects from message threads and placing them in the real world.
The solution to the problem of user images during video is to prepare a 3D face in advance, called the Persona function. FaceTime on Vision Pro can display life-size characters during video calls, input the user’s 3D face data in advance, and produce a 3D modeling rendering of themselves.
visionOS is the first operating system designed from the ground up for spatial computing, and visionOS adds many new features to support the low-latency requirements of spatial computing.
Such as a new real-time execution engine that can guarantee performance-critical workloads. Such as the introduction of dynamic foveated rendering, which delivers maximum image quality exactly to every frame your eyes are looking at, and the first multi-application 3D engine, which allows different applications to run simultaneously in the same simulation.
Content Ecology
In addition to good hardware and a good operating system, to get more consumers to pay for “spatial computing”, Apple also needs to build a good enough content ecosystem.
At the meeting, Disney CEO Bob Iger made an appearance and said that the Disney+ service will be available on Vision Pro on the launch day. He also said that this is only part of the cooperation between the two parties, and more content will be shared in the next few months.
Apple has made Vision Pro available to some developers before, and some amazing content has been developed so far.
An interactive 3D heart created by Complete HeartX. JigSpace allows designers to visually review their drafts, such as F1 racing cars. Business operators can use PTC’s Stages to review production lines before they are officially put into production. The DJay app can be used to mix music. SkyGuide can lead users into a private planetarium to gaze at the stars. Microsoft Office software and video conferencing software such as Zoom are also connected and can be used directly. Apple also announced a partnership with Unity to allow developers to bring 3D content to the Vision Pro platform.
Vision Pro will have a separate App Store where users can explore and download apps built specifically for Vision.
Privacy & Security: Optic ID
Finally, Apple also talked about privacy and security issues for its next-generation computing platform. In Vision Pro, user log in and authentication will no longer use password login in the PC era or fingerprint and face recognition in the smartphone era but use Optic ID.
Optic ID uses iris recognition for verification. Human eye iris recognition is unique and is a more secure verification method than face and fingerprint.
Optic works with Apple Pay or apps downloaded from the Apple Store, and data is encrypted and never leaves the device, only accessible through a secure verification process within the device.
In addition, Apple has also incorporated a variety of privacy protection measures into Vision Pro. For example, the gaze point data of human eyes and the video data of the space where they are located involve personal privacy. This data will be isolated and processed separately in the background of the device.
As the most important hardware product of Tim Cook’s Apple era, Vision Pro sets a new benchmark in hardware and engineering. Apple’s originality is also reflected in the software and interaction levels.
But whether this device becomes the platform that defines the next decade may ultimately depend on whether the content and ecosystem can truly be established.