Skip to content

Latest commit

 

History

History
168 lines (111 loc) · 8.66 KB

glsl-reduce-walkthrough.md

File metadata and controls

168 lines (111 loc) · 8.66 KB
Raw

glsl-reduce

glsl-reduce is a tool for automatically reducing a GLSL shader with respect to a programmable condition.

In this walkthrough, we will use a mock shader compiler bug to demonstrate how to use glsl-reduce in action.

We will be using the latest release zip graphicsfuzz-1.0.zip. You can download this from the releases page or build it from source.

Add the following directories to your path:

  • graphicsfuzz-1.0/python/drivers
  • One of:
    • graphicsfuzz-1.0/bin/Linux
    • graphicsfuzz-1.0/bin/Mac
    • graphicsfuzz-1.0/bin/Windows

The graphicsfuzz-1.0/ directory is the unzipped graphicsfuzz release. If building from source, this directory can be found at graphicsfuzz/target/graphicsfuzz-1.0/.

You will also need to install the latest version of the Java 8 Development Kit, either:

# To check that Java 8 is installed and in use:
java -version
# Output: openjdk version "1.8.0_181"

glsl-reduce in action

We'll start by illustrating a mock shader compiler bug. We present commands assuming a Linux/Mac environment, but they should be easy to adapt to a Windows setting.

# Copy the sample shaders into the current directory:
cp -r graphicsfuzz-1.0/examples/glsl-reduce-walkthrough .

# Run the fake shader compiler on the fragment shader file:
python glsl-reduce-walkthrough/fake_compiler.py glsl-reduce-walkthrough/colorgrid_modulo.frag

# Output:
# Fatal error: too much indexing.

Our fake compiler fails to compile the valid shader because it cannot handle shaders with a lot of indexing. Thus, we have found a compiler bug.

Take a look at fake_compiler.py if you want to see why the fake compiler generates this error message for the shader. This is not important for understanding how to use the reducer.

Let's now use glsl-reduce to get a much smaller shader that causes the compiler to fail with this error:

# Observe that the initial shader is reasonably large
cat glsl-reduce-walkthrough/colorgrid_modulo.frag

# Output:
# <contents of the original shader>

# Run glsl-reduce, putting the tool's output in the 'reduction_results' directory (created if it does not exist)
glsl-reduce glsl-reduce-walkthrough/colorgrid_modulo.json ./glsl-reduce-walkthrough/interestingness_test --output reduction_results

# Output:
# <lots of messages about the reducer's progress>

# Confirm that the reduced fragment shader file still reproduces the issue
python glsl-reduce-walkthrough/fake_compiler.py reduction_results/colorgrid_modulo_reduced_final.frag

# Output:
# Fatal error: too much indexing.

# Observe that the reduced shader is much smaller
cat reduction_results/colorgrid_modulo_reduced_final.frag

# Output:
# <contents of the reduced shader>

In reduction_results you will also find many partially-reduced shader jobs named colorgrid_modulo_reduced_X_Y.json, where Y is success or fail depending on whether the shader job was interesting, and X is a counter associated with each shader job, indicating at what point during the reduction process it was considered. These shader jobs are sometimes useful for understanding the steps that were made during a reduction, but can otherwise be ignored.

In more detail

glsl-reduce requires two arguments:

  • a shader job: a path to a .json file representing the shader or shaders of interest
  • an interestingness test: a path to an executable script encoding the properties a shader job must satisfy to be deemed interesting.

It is advisable to also use the --output option to specify a directory for generated results; this will be the current directory by default.

The .json file serves two purposes: (1) it tells glsl-reduce which shaders to operate on -- if it is called foo.json then at least one of foo.frag, foo.vert and foo.comp must be present, and any that are present will be reduced; (2) it allows metadata about these shaders to be stored. At present, glsl-reduce does not allow any metadata when used in stand-alone mode, so the JSON file is required to simply contain {}. (The file is used for meaningful metadata when glsl-reduce is used in conjunction with glsl-fuzz.)

The interestingness test (name inspired by C-Reduce) is a script that codifies what it means for a shader to be interesting. It takes the name of a shader job as an argument, and should return 0 if and only if the shaders associated with the shader job are interesting. What "interesting" means is specific to the context in which you are using glsl-reduce.

In the above example we deemed a shader job foo.json to be interesting if the associated fragment shader foo.frag caused the fake compiler to fail with the "too much indexing" message. Have a look at glsl-reduce-walkthrough/interestingness_test.py (which is invoked by glsl-reduce-walkthrough/interestingness_test for Linux/Mac, and the corresponding .bat file for Windows): it checks precisely this.

In a different setting we might be interested in reducing with respect to a completely different property, e.g. we might check whether rendering using a given shader causes a device's temperature to exceed some threshold, whether a shader causes a driver to consume more than a certain amount of memory, or whether a shader sends a shader compiler into a seemingly infinite loop. Writing the interestingness test is entirely in the hands of the user of glsl-reduce. Notice that in each of these examples, the property we care about (temperature, memory or execution time) is non-binary; it needs to be turned into a binary property by using an appropriate limit on the resource in question inside the interestingness test.

Bug slippage: the need for a strong interestingness test

Let's try our reduction again but with a different interestingness test. Have a look at glsl-reduce-walkthrough/weak_interestingness_test.py. This test just checks whether compilation failed; it does not check the failure string.

Let's use it to perform a reduction:

# Run glsl-reduce
glsl-reduce glsl-reduce-walkthrough/colorgrid_modulo.json ./glsl-reduce-walkthrough/weak_interestingness_test --output slipped_reduction_results

# Output:
# <lots of messages about the reducer's progress>

# Whoah, it's an even smaller result!
cat slipped_reduction_results/colorgrid_modulo_reduced_final.frag

# Output:
# <a shader with an empty main>

# Does it still give us the fatal error?
python glsl-reduce-walkthrough/fake_compiler.py reduction_results/colorgrid_modulo_reduced_final.frag

# Output:
# Internal error: something went wrong inlining 'floor'.

The reduced shader does cause a bug to trigger in the fake compiler, but it is a different bug to the one we saw before: we get a different error message! (Look in glsl-reduce-walkthrough/fake_compiler.py to see why this message is generated; this is of course just a mock bug made up for this illustrative walkthrough.)

This is known as bug slippage (see Section 3.5 of this paper): in reducing a shader that triggers one bug we have ended up triggering a second bug and switching attention to that second bug. What we end up with is a minimized shader that triggers the second bug but not the original bug.

Slippage can be useful as a method for discovering additional bugs (see this blog post on the topic), and the series of partially-reduced shader jobs produced during reduction sometimes provides a rich source of by-product bugs. However, slippage is annoying if we really care about reducing with respect to the original bug. To avoid slippage, you need to write a sufficiently strong interestingness test.

Summary from walkthrough

The walkthrough has illustrated:

  • How to run glsl-reduce on an example
  • What the arguments to glsl-reduce mean, including the concept of an interestingness test
  • The notion of bug slippage, whereby a weak interestingness test can cause reduction to switch from an original bug to some other bug

We emphasize again that the interestingness test is entirely programmable. In this tutorial it has simply involved invoking a (fake) compiler and checking the compiler's output. In practice it could do something much more complex, such as launching an app that will consume the shader of interest and then checking for some specific app behavior.