Will the Ultimate Quest 3 and Vision Pro accessories become a book?

Opening an ordinary-looking book and seeing magical three-dimensional creatures jumping out of it seems like what AR glasses were supposed to do years ago. In some cases, they already do: Way back, Sony has a Harry Potter licensed AR book that works with the PlayStation’s camera, and Nintendo has been working on its own marker-based cards and books yourself by using the 3DS’s camera and 3D display to create an AR experience. , also. A new experience, based on Jim Henson’s Storyteller, presents itself as an idea for how physical accessories like books could reemerge for mixed reality headsets in the future.

Felix and Paul Studios have already created AR experiences, but this is a new venture for the company focused on cinematic VR. of the studio Continuous series of 360-degree movies filmed on the International Space Station does an incredible job of capturing reality for viewing inside a VR headset. But in the case of AR-based Storyteller experiences, it’s the opposite challenge. Instead of taking me somewhere else, Jim Henson’s The Storyteller: The Seven Crows places a small story world inside a physical prop that resembles a book. The visuals are captivating, like a miniature video game cinema come to life, and Neil Gaiman’s voice narration certainly doesn’t hurt.

The experience felt like I was looking down at the book in an inner world, where I could tilt and rotate to get a better look beyond the book portal. Sometimes, characters also appear and look at me through the window. Tilting the book sideways can cause water to spill out from the ocean scene. A new AR scene begins every time one of the book’s board-thick pages is turned. Also, when I got to what I thought was the end of the book, I was instructed to turn it over and start turning the pages in the other direction. The story continued as I turned the pages. A book like this could be timeless.

Tools beyond the controller

The future of VR and AR headsets looks like it won’t include controllers. Hand tracking will become more standard thanks to devices like the upcoming Meta Mission 3promises better depth tracking sensors and Apple’s professional vision coming in 2024. If a controller-free mixed reality future is on the horizon, relying on our hands to reach out and touch things that aren’t there, a variety of accessories are likely and mixed-reality-ready objects can offer the feeling of AR as it expands to a wider range of tactile tools.

The keyboard and trackpad are the obvious next step. Apple’s Vision Pro and Meta’s Quest headsets support these, and several companies have come to mind The keyboard supports AR. Perhaps other accessories will appear later. Books or objects can be small spaces designed to be ready for AR to be projected onto them, which could be the next product.

It’s reminiscent of many previous marker-encoded toys and cards that worked with AR-ready phones and tablets or even headsets. But technological developments may be coming at the right time.

The Seven Ravens experience and its book illusion work quite well and cleverly use its book design to cover up a major limitation of current AR glasses. Headsets like the Magic Leap 2 have a limited field of view, which in this case conforms to the outline of a custom prop book for the experience. The hard-coded pages are turned over one by one, each page bringing a new chapter to life and appearing to be embedded in the book frame (this also helps increase the transparency of 3D objects in AR, as the book pages are dark black).

The Storyteller Experience based on Magic Leap was created five years ago and was intended to launch in 2020. It was delayed because of the pandemic and because Magic Leap changed course, moving away from episodic trials focus from creativity to relaunching with business goals. The Storyteller experience rose from the ashes to premiere at the Venice Film Festival, but Felix & Paul Studios co-founder Paul Raphael now sees the experience as a stepping stone to creating a book product soon actually for headsets, maybe even for mixed reality devices like Apple’s Vision Pro or the Meta Quest 3.

Raphael sees recent transition-based mixed reality VR headsets that can do AR as the perfect match for ideas like this, partly because people actually own these devicesas opposed to the AR glasses space, which is currently on the fringes.

“I think transitions are great for this. In some ways it’s better; in some ways it’s not quite as good. You’re aware of the environment, what you have at hand, But the focus is on what’s in the book,” Raphael said. “Even if what’s shown isn’t exactly like the photo, you’re still getting close to the Vision Pro. And perhaps even the Quest 3 would do a very good job. I absolutely consider it that type of device.” is responsible for this.” experience.”

The narrator isn’t a real product yet, but the idea for the book could be. “Once AR becomes popular and glasses like the ones you’re wearing can do AR, everything will be augmented,” Raphael told me after I tried it out. “When we started designing this book, we were really thinking about audience-based storytelling in general. This book is the first, the one where we can get the most out of it, But we have concepts for all kinds of augmented objects that we love.” to make it a reality.”