Setting up Ambient Occlusion

Scheduled maintenance pending. The forum will be closed at 4 o'clock a.m.
  • Ambient Occlusion (AO) is a technique in which shadows that are a result of the indirect environment lighting are precalculated and stored in a separate texture or in the mesh data. These are precalculated because of the high calculation load involved, which at this time does not allow for real-time calculation. However, as the environment light is assumed to be close to constant this doesn't present problems.

    1 Overview

    We can differentiate two fundamentally different techniques:

    • Per-Vertex-AO: In the Content-Tool every vertex of the mesh has a value associated that describes how brightly lit it is by the ambient light.
    • Texture-AO: The level of brightness is "drawn" as shades of grey in a texture. This can be done either by hand or using a suitable graphics program. The texture is then used in the Content-Tool.

    Both methods have certain advantages, disadvantages and peculiarities:

    1.1 Per-Vertex-AO

    The first method is particularly well suited for filigree or small objects with pronounced shapes, like switches, buttons, handles, etc. and / or self-contained surfaces, such as cylinders and balls, like handrails. These usually have a high vertex density whereas a clean mapping of a texture is either difficult or impossible. The texture resolutions that would be required to achieve proper AO would have very high storage demands and requirements. The Per-Vertex-AO uses the secondary texture coordinates to store the brightness information. Because of this it is not possible to combine mesh-animations and Per-Vertex-AO in one mesh. And last but not least is it not possible to include "baked" shadows of artifical lighting.

    1.2 Texture-AO

    The second method is more suited for rather flat and rough or angular objects, which consist mainly of relatively flat surfaces meeting other surfaces at an angle. Examples of this are the interior and exterior walls, floor and roof. Here mapping a texture is mostly without problems. However, the vertex density is so low that the transitions between the vertices strongly distort the impression and, some details may even be completely ignored. Balancing this only works with a texture that has the necessary resolution.


    For Texture-AO there are again two approaches:

    • Using the "normal" texture coordinates
    • Using the alternative/secondary texture coordinates

    Here again the choice depends on what we want to achieve:


    If the different areas on a texture are only used once in a mesh (like with large and individual photo textures for old buildings or instruments) or if the parts of the mesh containing this area are generally lit in a very similar way (like support beams or walls in a wagon hall) then you can use the "normal" texture coordinates. In that case the technique is simpler to use: The AO-texture is a grayscale texture (but in 24 bits file format) that shows the ambient brightness at the same location of the normal texture. It is also the only possibility to have AO-information on an object with a mesh animation.


    If a texture is used repeatedly or in many different parts of the mesh that are also lit differently (like a floor being mapped with a tiled texture that is repeated) the second variant is better. In this method the secondary texture coordinates are used. Because of this you can use a completely different texture mapping. With this new mapping you can make sure that each area of the texture is only mapped to one part of the mesh. A texture with a low resolution can be used where this is possible, like for the previously mentioned floor. An automated and optimized texture resolution for AO is for example available in Blender. Because, like with Per-Vertex-AO, the secondary texture coordinates are used and so this method cannot be combined with mesh animations


    In both cases the AO-texture can be created manually or using a suitable 3D program, like Blender. Next, like any other texture, it is copied into the working directory and can be selected in the Material properties.


    Last but not least, texture AO allow you to include "baked" artifical light shadows into the AO texture.

    2 How to do it?

    2.1 Per-Vertex-AO

    For this select the mesh in the Content-Tool and in the section Selected Mesh click on "Generate Per-Vertex-AO". Cancelling can be done using the "x" button next to the progress bar.


    Calculating AO with a high quality is comparatively time consuming because of the quality needed. Using a lower quality would result in an unrealistic effect and is really only suitable for tests. Before release you can then take the time to do the high quality calculation.


    If the selected material has the type "Standard", it will be changed to "Complex". Important: Because the Content-Tool (automatically) updates the material for the Per-Vertex-AO, all other meshes that use this material can have serious drawing errors. Always apply Per-Vertex-AO on either all or none meshes using a material.

    2.2 Texture-AO using normal coordinates

    Setting up a material to use a AO-texture using normal coordinates is very simple: After the texture is created or generated it is stored in the working directory. The material type is then set to at least "Complex". Then "Intensity of AO texture" is set to 1 and both "Activate AO-texture" and "AO uses original mapping" are turned on. Lastly the texture is selected for "Ambient Occlusion (AO) Texture".

    2.3 Texture-AO using secondary texture coordinates

    To create the secondary texture coordinates a copy of the object is made in the 3D program. This is named as "@2m_" followed by the name of the original object. So if the object is name "House" the copy is named "@2m_House".


    The copy has to be included in EVERY export of the original object and if the original object is changed a new copy must be created as before. In principle very small deviations in the vertex data (position, normals etc) is ok, but it is very highly recommended to create a new copy.


    When we import the original object and the @2m_ copy in the Content-Tool the texture coordinates of the copy are added to the original object as secondary texture coordinates. After importing only the original object should show up, the @2m_ object should not appear in the list on the right!


    The remainder of the setup is like with the "Texture-AO using normal coordinates", however "AO uses original mapping" is not set.

    2.4 Intensity

    The value for "Intensity of AO" can be used in all three cases to "reduce" the AO effect. "Reduce" in this cases brightens the object overall.

    2.5 Artifical Light (Shadows)

    The AO textures are used mainly for the indirect/ambient light in dark corners etc., whereas the direct sun light is limited by the dynamic calculated shadows.


    Die AO-Texturen sorgen grundsätzlich dafür, dass das indirekte Licht in dunklen Ecken usw. entsprechend abgeschwächt wird. Beim direkten Sonnenlicht wird über die dynamischen Schatten dafür gesorgt, dass es nur an die richtigen Stellen gelangt. Was dagegen bisher offen bleibt, ist die Frage, wo die künstlichen Lichtquellen im Innenraum hinreichen. Im Normalfall, wenn es sich um schwarz/weiße AO-Texturen handelt, wird hier diese angewendet.


    Genau genommen ist das aber nicht ganz korrekt, weil ja durchaus die explizite Anordnung der Lichtquelle(n) im Innenraum bestimmte Bereiche gezielt beleuchtet oder eben nicht.


    Wer hier präzisere Lichteffekte wünscht, der kann – sofern er bereits eine normale AO-Textur verwendet – deren grünen Kanal hierfür nutzen. Denn genau genommen verwendet die "normale" AO-Berechnung stets nur den roten Kanal.


    In der Praxis sieht das dann so aus:

    • In Blender werden entsprechende Lichtquellen platziert
    • So, wie auch die AO-Textur in Blender erzeugt werden kann, ist es vergleichbar möglich, eine Schatten-Textur auf Basis der platzierten Lichtquellen generieren zu lassen. Diese lässt man auf eine separate Textur backen, die dieselbe Größe wie die AO-Textur haben muss.
    • Es liegen einem nun zwei Graustufen-Texturen vor. Beide werden nacheinander wie folgt bearbeitet: Die AO-Textur wird "rot eingefärbt", d.h. der blaue und der grüne Kanal werden komplett geschwärzt oder wahlweise werden die Tonkurven von grün und blau komplett nach unten gezogen. Dasselbe wird mit der Schatten-Textur gemacht, allerdings wird diese "grün eingefärbt" (roten und blauen Kanal reduzieren/schwärzen).
    • Die Schatten-Textur wird nun über die AO-Textur gelegt und mit dieser "additiv" vereinigt. Das Resultat ist eine Bitmap, die "so ziemlich gelb" ist; zur Erläuterung: Bereiche, die schwarz oder gelb sind, sind in beiden Originaltexturen schwarz oder weiß und somit abgeschattet/beleuchtet, rote Bereiche degegen sind jene, wo zwar das ambiente Licht hinreicht, die Innenbeleuchtung jedoch nicht - bei grünen Bereichen ist es genau andersherum.
    • Die Konfiguration dieser "Hybrid-Textur" erfolgt so, als wäre es eine völlig normale AO-Textur.