Can your holograms cast shadows on the real world? Let's find out.
In a normal app running on a PC, phone, or even in virtual reality, enabling shadows is as simple as checking a few boxes. But in mixed reality, things get a bit more complicated because the shadow needs to fall on the real world as well as your holographic creations. So, what happens if you "just turn on shadows" and deploy to HoloLens? This. I promise you this was captured from an actual device, not the emulator, and is representative of what you see.
Obviously, that's not what we want. We want shadows on the real world but nothing else. So, we need a material that renders only the shadow cast upon it. Unity doesn't provide a shader that'll do that for you by default, so we need to create our own. Here's what that looks like in Unity. Shadows have been made red for emphasis.
Normally, casting a shadow on another rendered surface would render the material and textures, only darker, using the other lights in the scene and ambient lighting. But the real world doesn't have a material in Unity. We can only add light, not take it away. Two problems that'd normally be taken care of for us to produce a properly lit (or darker) shadow. In this example, we'll simply set the shadow color directly as a property on the Holographic Shadow shader. What color should our holographic shadow be? It can't be black, because that'd be transparent, but it should be close-- 20, 20, 20 (RGB) is a good place to start. When deployed to HoloLens we now see shadows on the table, but nothing else.
For consistency, the shader applied to our holograms also sets shadow color directly so that shadows falling on holograms and the real world are the same color. This is significant for mixed reality as the normal influencers for areas in shadow, such as ambient light color, also impact the color of the holograms themselves. With them separated, we can control the ambient light level of objects in the scene independent of shadow color and intensity.
For more convincing shadows we could build a light scanning phase into the room scanning process. Aligning the light source in-game with the dominant real world light source would cause shadows to fall in the same direction. The above example is close, but not perfect. If the shadows fell in the opposite direction the illusion would break immediately. Sampling (somehow) the correct ambient light level in the room would also make shadows more convincing by matching color and softness.
Clearly, holographic shadows are possible. Whether holographic shadows will add value to your project is up to you. With the right level of fine-tuning shadows help connect your holograms to the real world.
It's worth nothing there is a performance impact associated with shadows. In the case of HoloBlocks, enabling shadows increased the draw could from 5 to over 30, although it was still able to maintain a steady 60 frames per second. Your mileage may vary.
We're preparing a standalone package for the Unity Asset Store, follow @DeckTwelve on Twitter and be the first to know when it arrives!