Normal Maps and Cube Maps

What are they and what do they mean?

Whats the Point of All This?

Q: Whatre we doing?

Whats the Point of All This?

Q: Whatre we doing? A: Making a game that looks good

Whats the Point of All This?

Q: Whatre we doing? A: Making a game that looks good Q: Whatre we really concerned with regards to looking good?

Whats the Point of All This?

Q: Whatre we doing? A: Making a game that looks good Q: Whatre we really concerned with regards to looking good? A: Essentially, whatever the players perceive in their minds of lights interaction with the world such that it is believable

What is Global Illumination?

All sorts of light bouncing around in the scene

What is Global Illumination?

diffuse

specular

What is Global Illumination?

and light finally reaches the viewer

How do People See Things?

We see light via a grid

Then What is a Picture?

An array of contiguous grid points formsa map

Is That It?
Are we limited to just a picture for a map? No, theres much more to this!

Is That It?
A map can be thought of as just some arbitrary set of samples! However, in graphics, we use 2D maps to represent some mathematical quantities of surface characteristics (e.g. textures of surfaces, surface normals, surface displacement, surface of the skydome, or even the surfaces of a cube)

Whats the Point of All This?

Turns out that the most important feature in the average image comes from direct lighting of surfaces I.e. the path where the light bounces off of one surface and enters the eye directly We realized this, so we do geometry lighting, and used texture maps to make the lighting results look even better

Its All in the Details

What if we want the best possible results for surface lighting? For best looking results: keep everything that we started with (e.g. all the polygons, normals, UVs, detailsheck, why not even the original spline surfaces?) But:

Recall

Its All in the Details

Too much memory requirements Too much processing requirements I.e. too expensive for us to do and maintain a reasonable framerate Worst of all, its not even necessary for good-looking, believable output (from todays standards)

Alternatives?
Q: Whatre we really concerned about in terms of local surface details?

A: How light reflects off surfaces

Alternatives?

Some tessellation

Alternatives?

Focus in on a spot

Alternatives?

Surface is flat on a per-triangle basis

Alternatives?
If one thinks of light as a ball, then:
Were mainly concerned with how light bounces off of surface details In other words, how the individual, flat elements of the surface appear to the viewer

Alternatives?

Specular Reflection

This is just mirror reflection

Surface Facing
Mathematically, the surface facing is described by the normal its just a direction in space that is perpendicular to the facing of the surface Its really just the direction that the surface is facing

Per-Vertex Lighting
How do we work with normals? We use normals on a per-vertex level to light triangles We then interpolate the vertex lighting results to generate smooth lighting for the fragments This is natural because the geometry approximates a smooth surface

Interpolated Vertex Normals

However, a problem arises: high frequency lighting isnt being captured we only get correct values for the end points, not the in-between fragments Instead, we interpolate the per-vertex normals, not the lighting color Do lighting on the interpolated per-vertex normals

One Step Further

Well, now that were interpolating normals across fragments, we may as well take this one step further We want normals that we can be modified on a per-fragment level

One Step Further

So, how do we do this without using a ton of extra vertices to fill in the gaps? We use a set of values that represent normals and map those values onto a surface a map of normals, or, a normal map Theyre applied (most of the time) just like regular texture maps

One Step Further

Look up a normal map just like a texture

One Step Further

When we render a fragment of a triangle, we use the UV coordinates to look up the normal map That lookup result is what we use to calculate the normal to be at that point To the artists, this means adjusting a normal is equivalent to reorienting the surface facing of that particular fragment

Adjusting the Normal

Example

Example

Example

Example

Example

Similarities?
Many may have noticed that normal maps resemble some other techniques used in modeling surface details Yes, normal maps are intimately related to bump maps, displacement maps, parallax maps, etc Theyre all just ways to describe surface geometry!

Disclaimer
In practice, to avoid seams and weird transitions, normal maps usually assume that the underlying geometry is continuous in its normals
As in, there is no sudden transition in the normals Discontinuities signify hard edges, where the geometry is meant to be not smooth

Disclaimer
Two ways to model a cube:
1) Faces with normals that are constant in the direction that they face 2) Faces with normals that are continuous across all adjacent surfaces

Usually, only the 1st case is used for a cube

Disclaimer
Normal maps are perturbations to the underlying geometrys interpolated normals
As in, the normal map results are used to offset the interpolated per-vertex normals MLee: Its important to note that nondegenerate geometry shouldnt have degenerate UVs, or we cant generate a correct UV basis

More Goodness
All this time, the examples only focused on direct lighting from local light sources What does a fragment really see?

More Goodness
Photons dont just come from a single light source

More Goodness
They come from every direction on the hemisphere of the fragment were rendering But the problem is, its way too expensive (if at all possible) to model all the interreflections entering through that hemishpere centered on that fragment

The Simple Solution

Is there a way to approximate environment lighting? Well, what have we been using all this time in terms of environmental lighting?
Lighting = Specular + Diffuse + Ambient

The Simple Solution

What is ambient lighting?
It is just a very simple form of environment lighting Were basically saying that a light source contributes a certain amount of inter-reflective lighting to the scene Of course, it can also be used for other things such as setting the mood of a level

Environment Lighting
To come up with a better looking environment lighting model, in addition to local lighting, we will break down the problem of capturing light coming from the environment into two parts:
Light from objects that are close by Light from objects that are far away (*No real-time inter-reflections, sorry)

Why This Way?

Humans perceive the world through perspective projection we experience foreshortening and parallax Objects further away feel like they move less in our field of view than objects that are closer (e.g. just like the backgrounds of old-school 2D side scrollers)

Why This Way?

Very little movement for the clouds! I think

So What?
We can exploit this lack of difference/movement between vantage points We will just assume that distant objects are all so far away that they never move relative to each other, even when the viewer moves about

Why This Way?

Proof: Thats 20 feet away between shots

And the Result?

Light coming from far away looks the same no matter how we move
I.e. the specular reflection of incoming light off of far away objects look the same even if we strafed around (as long as we maintain our facing)

So instead of being lit by just the ambient term of a point light source, now were lit by some environment light source thats dependent on the reflection direction

Cube Environment Maps

Think of a specular cube environment map as a sky dome in the game We think of the surface of our near-by object as a mirror to calculate the specular direction of the reflection Then use the direction of the reflection to look up the cube environment map (where the reflection intersects the cube map)

Cube Environment Maps

The result is used as the specular lighting for the fragment

More Cube Environment Maps

This technique isnt limited to specular lighting We can use the same technique for diffuse/irradiance environment maps
One way is to pre-sum the hemisphere that any point on the surface can see, and store that result into where the normal points at Therefore, look up by normal, not by reflection direction

More Cube Environment Maps

direct lighting indirect lighting

Summary
We want to model as much of a believable environment as possible Use normal maps for surface detail approximation Use environment maps for environment lighting approximation

Credits
The countless people whose work of art (e.g. pictures) I used

Questions?