How-to: Spectator View, a new tool to help others see what you see in HoloLens

Hello everyone! I wanted to take a quick moment to announce an exciting new tool we have developed to help bring to life the apps and experiences you have created for Microsoft HoloLens.

Since our launch last year, one of the pieces of feedback we have heard from many folks is, “How can I create a way for people to see what someone sees when they are wearing a HoloLens from a third-person perspective? I want people to not only see what I see when wearing a HoloLens, I want people to feel like they are part of the experience!”

Today, I am excited to share with you that we have answered these questions with the release of documentation that will allow you to build a spectator view camera for HoloLens.

So, what is a spectator view camera?

When we created HoloLens, we knew that we needed to create a way to connect the experience of someone wearing a HoloLens with that of an audience (whether that audience is 1 person or 100 people). Since a HoloLens can only be worn by one person, how do you open the view so everyone can see what that person sees? To solve this problem, we developed Mixed Reality Capture (MRC) to aid in the visualization of holograms that the wearer is seeing from a first-person perspective.

For many reasons MRC is a great way to let others see what you see when you are wearing a HoloLens. It is built right into the device so it’s always there and ready to use. It’s a very simple way to capture what is seen from the device so that your audience can follow along when you are giving a demo or enjoying a HoloLens app you want others to see.

Though MRC is easy to use, there are some constraints. Because MRC is built directly into the device it is subject to what’s available in the device, for example the specs of the photo/video camera, available processing resources to do capture, compositing, and more. It’s also a camera attached to the wearer’s head so it’s subject to the normally imperceptible head movements that can sometimes make the resulting video not as stable as some situations might require. If you have used MRC before, you may notice that sometimes even the slightest head movement by the wearer can make it hard for the audience to follow along.

We have developed the spectator view camera to specifically address those constraints.

A spectator view camera will allow your audience to do more than just see what you see when wearing a HoloLens. Yes, it allows others, who aren’t wearing HoloLens, to see the holograms you would see if you were wearing the device, but it also allows you to see what the people wearing HoloLens are doing and how they are interacting with their mixed reality experience. Additionally, because a spectator view camera is not captured with a camera attached to a human’s head, it won’t be subject to the normally imperceptible head movements so the resulting video footage will be much more stable and easier for the audience to follow along.

How would I use a spectator view camera?

Instead of me explaining it, here is a peek into what the world will look like once your spectator view camera setup is complete.

As you’ll see in the video, a spectator view camera can be used in three specific scenarios:

Capturing photos of a mixed reality scene.
Shooting video of a holographic environment in action.
Live streaming holographic content to an audience.

Great, so how do I get my hands on this?

To create your own spectator view camera, which is based off of the same technology we use to bring HoloLens demos to life for our audiences when we do presentations and keynotes, head on over to GitHub and check out the documentation needed to help you get started. But before you do that, here is some important background to understand.

As you can see from the video, you will need some development experience to build your own spectator view camera, and with all the parts in hand, you’d be able to erect one in less than a day. We designed the above video to help walk you through the steps to bring your spectator view camera to life. Think of video as a companion piece to the full documentation and GitHub step-by-step guidance we have provided. At a high level, here’s what you’ll have to do:

Acquire the proper hardware and a DSLR camera with HDMI output.
Mount one HoloLens to the camera with the mounting bracket.
Once mounted, you will be able to connect your spectator view camera to your PC. The HoloLens will communicate to your PC via Wi-Fi – most gaming and workstation PCs will be powerful enough to handle the job.
Your app will need to be a shared experience, and can run on both a HoloLens and a desktop – leveraging the Universal Windows Platform. The online Holograms 240 class will help you learn how to create a shared experience.
Complete some calibrations to ensure your HoloLens and DSLR camera are aligned.
You are now ready to share your holographic creations!

We recently shared the spectator view camera technology with a few of our partners and it has been great to hear from them first-hand how spectator view is opening new ways to show off their innovations. Here is what a couple of them had to say.

“Showing 3D objects in your physical world is a magical experience that is hard to translate to a 2D monitor. Spectator view with a stabilization rig makes it possible to really show the 3D nature of mixed reality experienced by a HoloLens user.” – Michael Hoffman, Co-founder, Object Theory

“Valorem’s Immersive Experiences Team has been hard at work developing a HoloLens hardware and software solution offering, based on the spectator view technology, which Microsoft is sharing publicly today. We’re excited to announce that our solution offers a turnkey way for HoloLens device owners to create third-person mixed-reality photo and video captures of their HoloLens applications, without having to build the camera rig themselves. We’re thrilled to be the first agency partner to provide such a solution offering, and to help drive adoption of the Microsoft HoloLens platform.” – Mark Brown, SVP Research & Incubation, Valorem

On behalf of everyone on our team we are thrilled to bring spectator view to you so that you can bring what YOU have created to more people. We can’t wait to see what you create with this new perspective!

-Brandon

Source: How-to: Spectator View, a new tool to help others see what you see in HoloLens