okami.mom/ultraviolet
git clone https://git.nkp.pet/okami.mom/ultraviolet
ab6f6cb
main
Renderer conformance tests
Fuzz tests that drive the terminal renderer against real terminal emulators and compare the resulting screen against ground truth.
Why this is a separate module
The renderer keeps a model of what the terminal is currently showing and writes only the difference between that model and the next frame. When the model drifts out of step with reality the output is still valid ANSI and still looks plausible; it just paints the wrong thing. Asserting on escape sequences cannot catch that, because there is no single correct sequence to compare against.
Catching it needs a real terminal emulator to read the screen back from, and the most accurate one available is libghostty, ghostty's own VT engine. That is a cgo package requiring Go 1.26 and a C library, and neither cost belongs in the library that ships to users.
Keeping it in its own module means the root module never sees it: no dependency, no cgo, no Go version bump. Nothing here reaches anyone who imports ultraviolet.
Running
Everything needs libghostty-vt. Build it once with zig and CMake, from this
directory:
# Build the commit go.mod depends on. go-libghostty's CMakeLists pins the # ghostty revision it fetches, and a newer one exports a different set of # symbols than these bindings call, so building its HEAD fails at link time. version =$( awk '$1 == "go.mitchellh.com/libghostty" { print $2 }' go.mod) rev =${ version %+incompatible} [[$ rev =~ -([0-9a-f]{12})$ ]]&& rev =${ BASH_REMATCH [1]} git clone https://github.com/mitchellh/go-libghostty /tmp/go-libghosttygit -C /tmp/go-libghostty checkout" $ rev " make -C /tmp/go-libghostty buildexport PKG_CONFIG_PATH ="/tmp/go-libghostty/build/_deps/ghostty-src/zig-out/share/pkgconfig"
CI does the same thing from .github/actions/libghostty-vt, reading the pin out
of the same line of go.mod. Splitting on the last dash is not enough: only a
pseudo-version ends in a commit, so a tag has to be used as it stands.
Then, still from this directory:
# Run the fuzz targets over their seed corpus, plus the regular tests. go test ./...# Search for new failing inputs. One target at a time. go test-run= 'XXX' -fuzz= 'FuzzRenderer$' -fuzztime=5m
Because these tests live outside the root module, go test ./... from the
repository root does not run them. They have to be invoked here.
What is tested
Each target decodes its input as a small program of renderer operations, replays it, and checks a property of the result. Go's fuzzer mutates the raw bytes and uses coverage feedback to steer toward programs that reach new renderer code, which is what finds the combinations nobody thinks to write by hand.
| Target | Property |
|---|---|
FuzzRenderer | An incrementally rendered screen matches a full repaint of the same frame. |
FuzzRendererIdempotent | Rendering an unchanged buffer twice leaves the screen alone. |
FuzzRedrawResyncs | A forced repaint recovers from any state the renderer drifted into. |
FuzzScreenShowsContent | Every cluster the buffer holds on the last drawn row reaches the screen. |
Each target also runs both ways the renderer can own a screen, chosen per program:
-
Fullscreen, where the frame is the terminal and the renderer moves the cursor absolutely.
-
Inline, where the frame is shorter than the terminal and the cursor moves relatively. There is no absolute move to fall back on, so a model that loses track of the cursor has nothing to recover with, and the rows below the frame belong to whatever was on the screen first. The screen is read back in full, past the bottom of the frame, because a frame that shrinks has to clear what it no longer covers and the abandoned rows are where the residue lands.
Rows of someone else's output sit above every inline frame, and the screen is read back over them too. They are the point rather than scenery: an inline renderer finds the top of its frame by counting rows upward from the cursor, and a count that overshoots erases into them. With nothing up there, an erase that reached a row too far would read back as blanks, which is what blank rows look like anyway, and the mistake would not show.
An inline frame may also collapse to no rows at all, which is the case most likely to reach above itself, since there is no last row for the erase below the frame to start from.
Inline screens are never narrowed, only widened. Narrowing makes the terminal rewrap the rows it holds, carrying them, and the cursor among them, somewhere a relative move cannot find again. What survives that is a property of drawing inline rather than a defect, so asserting on it would only produce failures nobody can act on.
Every target runs against two emulators, because they disagree about how wide a grapheme cluster is and that disagreement is the subject of these tests:
- libghostty models legacy widths per codepoint, the way stock terminals do, and implements DEC mode 2027 so both width models can be exercised.
- x/vt always measures whole clusters, so it behaves like a terminal with mode 2027 enabled.
Neither is redundant. A bug that only appears under legacy widths is invisible to x/vt, and a disagreement between the two is itself worth investigating.
The corpus
testdata/fuzz/ holds the inputs the nightly run has found. Each one was a real
renderer bug when it landed, so keeping it there is what stops the bug coming
back: the test job replays the whole corpus on every push, and Go validates it
before any mutation starts.
Add to it rather than pruning it. When a nightly run fails, download the
crashers-* artifact and commit the new file alongside the fix.
1# Renderer conformance tests 2 3Fuzz tests that drive the terminal renderer against real terminal emulators and 4compare the resulting screen against ground truth. 5 6## Why this is a separate module 7 8The renderer keeps a model of what the terminal is currently showing and writes 9only the difference between that model and the next frame. When the model drifts 10out of step with reality the output is still valid ANSI and still looks 11plausible; it just paints the wrong thing. Asserting on escape sequences cannot 12catch that, because there is no single correct sequence to compare against. 13 14Catching it needs a real terminal emulator to read the screen back from, and the 15most accurate one available is [libghostty], ghostty's own VT engine. That is a 16cgo package requiring Go 1.26 and a C library, and neither cost belongs in the 17library that ships to users. 18 19Keeping it in its own module means the root module never sees it: no dependency, 20no cgo, no Go version bump. Nothing here reaches anyone who imports ultraviolet. 21 22## Running 23 24Everything needs `libghostty-vt`. Build it once with [zig] and CMake, from this 25directory: 26 27``` shell 28# Build the commit go.mod depends on. go-libghostty's CMakeLists pins the 29# ghostty revision it fetches, and a newer one exports a different set of 30# symbols than these bindings call, so building its HEAD fails at link time. 31version=$(awk '$1 == "go.mitchellh.com/libghostty" { print $2 }' go.mod) 32rev=${version%+incompatible} 33[[ $rev =~ -([0-9a-f]{12})$ ]] && rev=${BASH_REMATCH[1]} 34git clone https://github.com/mitchellh/go-libghostty /tmp/go-libghostty 35git -C /tmp/go-libghostty checkout "$rev" 36make -C /tmp/go-libghostty build 37export PKG_CONFIG_PATH="/tmp/go-libghostty/build/_deps/ghostty-src/zig-out/share/pkgconfig" 38``` 39 40CI does the same thing from `.github/actions/libghostty-vt`, reading the pin out 41of the same line of `go.mod`. Splitting on the last dash is not enough: only a 42pseudo-version ends in a commit, so a tag has to be used as it stands. 43 44Then, still from this directory: 45 46``` shell 47# Run the fuzz targets over their seed corpus, plus the regular tests. 48go test ./... 49 50# Search for new failing inputs. One target at a time. 51go test -run='XXX' -fuzz='FuzzRenderer$' -fuzztime=5m 52``` 53 54Because these tests live outside the root module, `go test ./...` from the 55repository root does not run them. They have to be invoked here. 56 57## What is tested 58 59Each target decodes its input as a small program of renderer operations, replays 60it, and checks a property of the result. Go's fuzzer mutates the raw bytes and 61uses coverage feedback to steer toward programs that reach new renderer code, 62which is what finds the combinations nobody thinks to write by hand. 63 64| Target | Property | 65| ------------------------- | -------- | 66| `FuzzRenderer` | An incrementally rendered screen matches a full repaint of the same frame. | 67| `FuzzRendererIdempotent` | Rendering an unchanged buffer twice leaves the screen alone. | 68| `FuzzRedrawResyncs` | A forced repaint recovers from any state the renderer drifted into. | 69| `FuzzScreenShowsContent` | Every cluster the buffer holds on the last drawn row reaches the screen. | 70 71Each target also runs both ways the renderer can own a screen, chosen per 72program: 73 74- **Fullscreen**, where the frame is the terminal and the renderer moves the 75cursor absolutely. 76- **Inline**, where the frame is shorter than the terminal and the cursor moves 77relatively. There is no absolute move to fall back on, so a model that loses 78track of the cursor has nothing to recover with, and the rows below the frame 79belong to whatever was on the screen first. The screen is read back in full, 80past the bottom of the frame, because a frame that shrinks has to clear what 81it no longer covers and the abandoned rows are where the residue lands. 82 83Rows of someone else's output sit above every inline frame, and the screen is 84read back over them too. They are the point rather than scenery: an inline 85renderer finds the top of its frame by counting rows upward from the cursor, 86and a count that overshoots erases into them. With nothing up there, an erase 87that reached a row too far would read back as blanks, which is what blank rows 88look like anyway, and the mistake would not show. 89 90An inline frame may also collapse to no rows at all, which is the case most 91likely to reach above itself, since there is no last row for the erase below 92the frame to start from. 93 94Inline screens are never narrowed, only widened. Narrowing makes the terminal 95rewrap the rows it holds, carrying them, and the cursor among them, somewhere 96a relative move cannot find again. What survives that is a property of drawing 97inline rather than a defect, so asserting on it would only produce failures 98nobody can act on. 99 100Every target runs against two emulators, because they disagree about how wide a 101grapheme cluster is and that disagreement is the subject of these tests: 102 103- **libghostty** models legacy widths per codepoint, the way stock terminals do, 104and implements DEC mode 2027 so both width models can be exercised. 105- **[x/vt]** always measures whole clusters, so it behaves like a terminal with 106mode 2027 enabled. 107 108Neither is redundant. A bug that only appears under legacy widths is invisible 109to x/vt, and a disagreement between the two is itself worth investigating. 110 111## The corpus 112 113`testdata/fuzz/` holds the inputs the nightly run has found. Each one was a real 114renderer bug when it landed, so keeping it there is what stops the bug coming 115back: the `test` job replays the whole corpus on every push, and Go validates it 116before any mutation starts. 117 118Add to it rather than pruning it. When a nightly run fails, download the 119`crashers-*` artifact and commit the new file alongside the fix. 120 121[libghostty] : https://github.com/mitchellh/go-libghostty 122[x/vt] : https://github.com/charmbracelet/x/tree/main/vt 123[zig] : https://ghostty.org/docs/install/build