Building an 8x8 bayer-dither shader in 80 lines of GLSL

Bayer dithering is the kind of trick that looks expensive and isn't. The whole effect is a single comparison against a threshold matrix you can write down on a napkin. I wrote one to dither holiday photographs through NES sub-palettes for the Sunday dither series, and the entire fragment shader fits comfortably on a sticky note.

What ordered dithering actually does

When you quantise a smooth gradient to a tiny palette — say, the four colours of the Castlevania II overworld sub-palette — you get visible banding. Ordered dithering hides the bands by introducing a deterministic, repeating pattern of "should I round this pixel up or down?" The pattern looks like noise from a distance but is in fact a fixed matrix tiled across the screen.

The classic 8×8 Bayer matrix runs from 0 to 63. Normalise it by dividing by 64 and subtracting 0.5, and you get an 8×8 grid of bias values between roughly -0.5 and +0.5. Add that bias to each pixel's value before you quantise, and the quantiser's rounding errors get smeared across the grid in a perceptually pleasant way.

That is the entire trick. There is no temporal component, no error propagation, no neighbourhood lookup. Each pixel is a single texture sample, an addition, and a palette match.

The shader

Here is the full fragment shader I run in Godot 4.3. It's GLSL ES, so it'll port to Shadertoy or a stock WebGL pipeline with no changes to the body.

shader_type canvas_item;

uniform sampler2D source : source_color;
uniform vec3 palette[4];      // RGB triplets, 0..1
uniform float dither_amount : hint_range(0.0, 1.0) = 1.0;

// 8x8 Bayer matrix, normalised to roughly (-0.5, +0.5)
const float bayer8[64] = {
     0.0/64.0, 32.0/64.0,  8.0/64.0, 40.0/64.0,  2.0/64.0, 34.0/64.0, 10.0/64.0, 42.0/64.0,
    48.0/64.0, 16.0/64.0, 56.0/64.0, 24.0/64.0, 50.0/64.0, 18.0/64.0, 58.0/64.0, 26.0/64.0,
    12.0/64.0, 44.0/64.0,  4.0/64.0, 36.0/64.0, 14.0/64.0, 46.0/64.0,  6.0/64.0, 38.0/64.0,
    60.0/64.0, 28.0/64.0, 52.0/64.0, 20.0/64.0, 62.0/64.0, 30.0/64.0, 54.0/64.0, 22.0/64.0,
     3.0/64.0, 35.0/64.0, 11.0/64.0, 43.0/64.0,  1.0/64.0, 33.0/64.0,  9.0/64.0, 41.0/64.0,
    51.0/64.0, 19.0/64.0, 59.0/64.0, 27.0/64.0, 49.0/64.0, 17.0/64.0, 57.0/64.0, 25.0/64.0,
    15.0/64.0, 47.0/64.0,  7.0/64.0, 39.0/64.0, 13.0/64.0, 45.0/64.0,  5.0/64.0, 37.0/64.0,
    63.0/64.0, 31.0/64.0, 55.0/64.0, 23.0/64.0, 61.0/64.0, 29.0/64.0, 53.0/64.0, 21.0/64.0
};

vec3 nearest_palette_colour(vec3 c) {
    vec3  best  = palette[0];
    float bestd = distance(c, palette[0]);
    for (int i = 1; i < 4; i++) {
        float d = distance(c, palette[i]);
        if (d < bestd) { bestd = d; best = palette[i]; }
    }
    return best;
}

void fragment() {
    vec3 src = texture(source, UV).rgb;

    // Look up the bias for this pixel's screen position
    ivec2 p = ivec2(mod(FRAGCOORD.xy, 8.0));
    float threshold = bayer8[p.y * 8 + p.x] - 0.5;

    // Bias and quantise
    vec3 biased = clamp(src + threshold * dither_amount * (1.0 / 4.0), 0.0, 1.0);
    COLOR = vec4(nearest_palette_colour(biased), 1.0);
}

That is the entire effect. Eighty lines with comments, half of which are the matrix itself.

Things worth knowing before you ship it

The 1.0 / 4.0 divisor matters. That number is "one step of palette resolution". With a four-colour palette and reasonably even spacing, one step is about a quarter of the value range, so you want your bias to lie in roughly that range. With an eight-colour palette, divide by 8. If you skip this and just add the raw normalised matrix value, you'll get a vibrating, painful pattern instead of an even one.

Use sRGB-aware textures or your photos will look muddy. Godot's source_color hint does the linear-to-sRGB conversion for you. If you're in raw WebGL, sample in linear space, do the comparison in linear space, then write back out. Mixing the two spaces produces the dirty, "your shader is broken" look that everyone has accidentally shipped at least once.

Distance in RGB is not perceptual distance. For a quick post-processing effect, distance(c, palette[i]) is good enough. For palette extraction or anything that has to survive print, switch to OKLab or even simple CIE Lab. Björn Ottosson's writeup is the clearest source I've found.

Eight by eight is a sweet spot for screenshots. Photographs at 1080p look nice. Pixel art at 64×64, scaled up by integer factors, looks like a 1990s photocopier — sometimes that is what you want, but more often you'll prefer the smaller 4×4 matrix for sprites.

Using it from Aseprite

Aseprite ships a colour mode for indexed palettes and a built-in Ordered Dither command, but the built-in version is fixed at 4×4 Bayer. If you want the 8×8 version, my workflow is:

1. Convert the sprite to RGB.
2. Export it as a PNG.
3. Run it through the shader in a tiny Godot scene I keep around for this.
4. Re-import the result back into Aseprite as an indexed image, pointed at the same palette file.

The whole loop is about thirty seconds once you have it set up. The script file lives in the github repo for the series, if you want to skip the typing.

What it looks like

The Sunday series has twelve weeks of these on the index page, each photographed through a different real NES sub-palette grabbed from the NES palette reference at Nesdev. The Castlevania II overworld palette is my favourite — muted purples and ochres that flatter almost any landscape. The Kid Icarus underworld palette is harder to make work and that is why I keep coming back to it.

If you want to try the shader on your own photographs, the GLSL above is everything. Pick four colours, paste it in, and watch your beach holiday turn into 1986.