I spent some time trying to improve my water shader recently, and found there were two problems I had yet to solve in a satisfying way. This post covers the first of them.

Something which makes water look “just wrong” is the lack of correct reflections. A cubemap reflection goes a long way, but it cannot hide the fact that certain surfaces are not being reflected, such as the terrain. This can make the world appear to float when viewed from certain angles:

Terrain appears to float due to the lack of reflections

There has been extensive research done on the subject of real-time reflections, with no “catch-all” method available.

Planar Reflections

Early 3D games had no problems rendering “proper” planar reflections due to their simple nature but as the complexity of our scenes grow, so does the cost of these type of reflections. You can’t “just render the scene twice” if rendering the scene once already costs 90% of your resources. Planar reflections also have their problems, mainly that they can be hard to manage, and are limited to reflecting off a single surface with the same surface normal. If you want multiple reflections, you have to do multiple renders.

Planar reflections in Deus Ex (2000)

Screen-Space Reflections

The shift towards deferred rendering, which itself isn’t very compatible with planar reflections, has given popularity to a method of rendering reflections called screen-space reflections (often abbreviated as SSR). This method uses the pixel normals and depth to calculate “screen-space” reflections, by ray-marching the screen buffer after the scene has rendered. It is very compatible with deferred rendering (since the g-buffer provides the necessary data) and although it requires a lot of samples/filtering/tweaking to look good and still counts as a high-end effect, at least it doesn’t require rendering the scene twice. It also handles reflections in any direction and is independent of scene complexity, much like screen-space ambient occlusion.

Screen-space reflections can look very nice in still shots with an optimal view of the scene (Stachowiak & Uludag, 2015, Stochastic Screen-Space Reflections)

The bad with SSR (and this is a big bad, if you ask me) is that because it is screen-space, most rays will fail to hit what they “should” hit, and many will not hit anything at all. Surfaces which cannot be seen (off-screen, behind other objects etc.) cannot be reflected and so you end up with a kinda-sometimes-works-reflection solution. It is also prohibitively expensive for VR, and the inconsistent reflections end up very distracting.

Hey, where did my reflections go!? The Witcher 3 (2015)

Something Else

Looking at both of these solutions, none of them seem very appealing, so maybe we can come up with something new. Let’s look at our requirements and what we can sacrifice:

Requirements

• Realtime
• Can’t render the scene again
• Must be able to reflect objects off-screen/not limited to screen-space
• Must be fast
• Must be reasonably accurate
• Preferably done directly in water shader

Sacrifices

• Reflecting the terrain is good enough
• The reflection color can be very simple
• Water is often wavy = reflection can be, too

Alright, so that seems like quite the task. Turns out though, it’s actually rather simple.

Ray-marching the terrain height map

Screen-space reflections use a technique called ray-marching. This is a form of ray tracing where you take discrete samples of some data as you travel along a ray to determine if you hit something. The reason SSR reflections disappear is because of lack of data, not because there is something wrong with ray-marching. What we need is some form of data which is available anywhere, at any time. If we upload our terrain height map as a shader parameter, we have exactly this. We can now do ray-marching, and for every step we check the current ray position against the terrain height value. If it is lower, we know we intersected the terrain. For our final intersection point it’s a good idea to pick a point between the last ray march position and the one that was below the terrain. This will give us a slightly more accurate reprensentation of the terrain. This can be improved upon further by stepping back and forth a bit to find the “true” intersection point, however for reflections I’ve found that just interpolating between the two works fine.

Ray-marching the terrain height map

Results

Now let’s take a look at that shot from the beginning with and without ray-marched height map reflections:

Without reflections

With reflections

Much better! It is now obvious where the terrain sits in relation to the ocean.

A really cool side effect of this method is that it works independently of position and orientation of the water plane:

Water plane at high altitude. Hidden mountain lake, perhaps?

Slanted water plane. I would probably pack my bags if I lived here.

I haven’t done rendering timing, but the effect works with just 16 ray-march steps and so has no noticeable performance impact on my machine. I imagine you could do down-sampled rendering to make it blazingly fast even with a high sample count.

What should the reflection color be?

I’m returning a dark terrain-ish color multiplied with the ambient sky value. If you can evaluate your terrain shader anywhere, you can return that instead:

Reflecting the terrain shader.

The reason I don’t is just to save some performance and texture lookups.

What about ray-march step count/length?

Depends on your scene. I’m doing 16 steps with variable length based on the viewing angle:

float stepSize = lerp(750, 300, max(dot(-viewDir, worldNormal), 0));

I imagine this could be improved a lot. Since it works the same way as screen-space reflections, I would look to such implementations for info on this.

What about glossy reflections?

Tricky. Probably best to render the reflection to a separate buffer and do post-processing on it. Same problem applies when rendering planar reflections so I imagine there are resources available online on this.

Code pls

The code will depend on your setup but here it is in basic form:

const static int stepCount 	= 16;		// Maximum number of ray-march steps

float3 ro = worldPos;				// Ray origin
float3 rd = reflect(viewDir, worldNormal);	// Ray direction

float4 hit = 0;
float3 hitPos = worldPos;

float stepSize = lerp(750, 300, max(dot(-viewDir, worldNormal), 0));
float3 prev = ro;
UNITY_LOOP
for (int u = 0; u < stepCount; u++)
{
	// Current ray position
	float3 t = ro + rd * (u+1) * stepSize;

	UNITY_BRANCH
	if (OutsideTerrainBounds(t))
		break;

	float height = GetTerrainHeight(t);

	UNITY_BRANCH
	if (height > t.y)
	{
		// Interpolate between current and last position
		hitPos = (t * prev) * 0.5;
		hit.rgb = GetReflectionColor(hitPos);
		hit.a = 1;
		break;
	}

	prev = t;
}

// Mix with cubemap reflection
float3 reflection = lerp(cubeMapRefl, hit.rgb, hit.a);

GetTerrainHeight is a function which returns the terrain height at that world position (this is where the terrain height map comes in).

GetReflectionColor just returns unity_AmbientSky * 0.35 in my case.

What about reflecting objects other than terrain?

Tricky, but not impossible. I’ve done some testing using Signed Distance Fields, and I think the results are promising. My test scene uses an array of parameters which describe proxy SDF cubes. I already had this set up since I use it for generating terrain shadows and AO on the GPU, so it was just a matter of firing an additional SDF ray.

Reflecting an aircraft carrier using SDF proxies (3 cubes).

Reflecting a bridge using SDF proxies (visualized in the bottom image).

This runs at real-time but is too slow for a practical application (much less a VR game). I imagine you could get pretty decent performance if you use precomputed SDF textures instead. I will probably get around to trying this eventually.

In the next post, I’ll show you how we can use the terrain height map to improve our water shader even further by generating shorelines and a projected ocean floor. Stay tuned!