How performance is measured

The cost model states the numbers. This page states the machinery that produces them, the gates that keep them true, and the instruments an application can point at its own frames. It assumes the guide and reads better after Architecture overview.

Three instruments

A band

A band is the comparison, written down once and applied by the harness that takes the measurement. It has two shapes, because the numbers are of two kinds (crates/zgui-bench/src/scenario/band.rs):

// Internal to the measurement harness, not public API.
enum Band {
    /// Microseconds. `tolerance` is a fraction: 0.4 admits 40 % above the baseline.
    Time { baseline: f64, tolerance: f64 },
    /// A ceiling, with no tolerance at all.
    Count { ceiling: u64 },
}

A time is a property of the machine as much as of the code. It moves with the core the scheduler picked, with what else is running, and with the state of the caches. Its band is therefore a multiple of a recorded baseline, and the multiple has to clear the ordinary spread of the measurement on an ordinary machine.

A count is a property of the design. "One row hovered, one row restyled" reads the same on a slow machine, a fast one and under a debugger. A count band is therefore a ceiling with no tolerance at all: the moment a frame emits a two-hundredth primitive, something stopped culling. Widening a count band would admit exactly the regression the count exists to catch.

Three tolerances exist, and each states why it is that wide:

A budget

A budget is what the schedule wrote for a quantity before it was measured. It does not move with the measurement.

That is the statement a band cannot make. A band is written against what was measured, so it fires on a change and stays quiet on a machine that is merely slower than the one the numbers came from — which means a number can sit inside its band for ever while never once having met the budget. Reporting both is the only way to tell those two states apart.

The ratchet

cargo xtask perf has three obligations, separable on purpose, and it performs them in this order (crates/zgui-bench/src/scenario/record.rs):

It always runs in release. A band is an assertion about time, and one measured in an unoptimised build is an assertion about a program nobody runs.

perf sits in cargo xtask ci after the budget gates and before the documentation gate, because it writes documentation: a build whose numbers moved leaves the change in the working tree where a reader will see it.

The five scenarios

A scenario is one interaction driven over one document, headless, on a virtual clock, with a null renderer. Everything it reports carries a band.

Every band, from the source rather than from a run:

Measured values live in docs/performance.md, and the cost model reads them out.

What each scenario had to get right

• Each runs in its own process. A cold start is only cold once, and every scenario mounts an application through thread-local state that a second mount adds to rather than replaces. The runner spawns itself once per scenario and reads the numbers back off its own output.
• quiet() before measuring. A document that has just opened owes a first cascade, a first layout, a first shaping of every run and a first emission of every primitive. Measuring before that has settled measures the opening.
• The median, not the mean. One sample of an interaction is occasionally the scheduler's rather than the framework's, and a mean carries that outlier into every later comparison.
• idle is not the gallery. The gallery ships an indeterminate progress bar, and a document with a running animation in it is supposed to draw every refresh. A ceiling that admits sixty frames a second admits every regression this scenario exists to catch.
• hover-storm asserts that it hit something. A pointer aimed at empty space costs nothing and would pass every band. The scenario fails when no element was restyled by 240 crossings.
• scroll buckets its frames by one counter and asserts three others. A frame is a translation when boxes_rebuilt is zero. What is then claimed of it is restyles, relayouts and hit-index rebuilds, so the classification and the claim do not share a number.
• kitchen-sink runs its interactions in a fixed order. The keystroke goes first, because the field it types into is found by tabbing from the top of the document. The resize goes last, because every step of it leaves the window a different width.

Every scenario also records three counters — primitives emitted, primitives culled, draw-order insertions — whether or not that scenario is about emission. A number that exists only where somebody expected it proves nothing about anywhere else.

The five standing gates

Each is a step of cargo xtask ci and a subcommand of its own, so it can be run alone while the thing it covers is being changed (CONTRIBUTING.md).

One reference workload is deliberately absent from that list. The 42-step gallery script has a gate of its own — cargo xtask verify, which compares a running window's display list and resolved geometry against one rebuilt from nothing — and running it twice under two names would mean two places to relax it.

The reference workloads

A reference workload is a document a later claim is measured against. One rule governs every criterion, quoted in docs/perf/reference-workloads.md from docs/COMPOSITOR-PHASES.md:

A criterion is a slope or a same-run ratio, never an absolute microsecond count. Absolute numbers are recorded as an advisory artifact keyed by machine and gate nothing. A standing gate that can only pass by re-recording its baseline gates nothing.

Both halves of a ratio are taken in the same process, over the same documents, minutes apart at most. A machine twice as fast divides both halves by the same number and leaves the ratio exactly where it was. That is why a duration would be the wrong criterion: it would either fail on every machine that is not the maintainer's, or be relaxed until it asserted nothing.

Every criterion the gates apply (docs/perf/reference-workloads.md; the constants live beside the verdict that applies them, not in that page):

A ceiling is used wherever the ideal is zero and the failure is one: a two-sided band recorded at today's value would fail an engine that got better. A band is used where the number is a proportion between two real pieces of work and either half moving is worth knowing about.

A missing number is not a pass. A workload whose measurement stops working returns nothing, and a criterion written as "not worse than" holds against nothing at all. The verdict types therefore report Broken, and broken fails.

The workload that is deliberately not a gate

unvirtualised-probe mounts every one of twenty thousand rows.

It is the only document in the repository that would make a change to scrolling look important. Wiring it into ci would let that change justify itself against a document no application would ship: the number would be real, the improvement would be real, and the conclusion drawn from it would be false. xtask/src/workloads/subject.rs carries a named test asserting the probe is not in the gate's list, and a second asserting the same of glide-split.

glide-split is out for that reason and one more: it asks the offsetting walk to make its descents one duty at a time, so what it times is not the shape of the walk a real frame makes.

Both stay runnable on demand. They are diagnostics.

Running all of it

cargo xtask ci          # the whole definition of done
cargo xtask perf        # the five scenarios, the ratchet, docs/performance.md
cargo xtask skips       # every counter of avoided work names its pair
cargo xtask budget      # a cache over its soft limit comes back under it
cargo xtask cadence     # one frame per refresh at 60, 75 and 240 Hz
cargo xtask workloads   # the reference workloads against their same-run ratios
cargo xtask resize      # the resize slope against a same-run baseline
cargo xtask verify      # a live window's display list against a rebuild's
cargo xtask wall-clock  # the absolute-duration budgets, in release

The harness underneath, for a number that has gone red:

# The ratchet itself. This is exactly what `cargo xtask perf` runs.
cargo run --release -p zgui-bench --bin zgui-bench -- scenarios

# One scenario, printed, with no verdict.
cargo run --release -p zgui-bench --bin zgui-bench -- scenario hover-storm

# One phase at one document size, with a latency trace.
ZGUI_LATENCY=/tmp/trace.jsonl \
  cargo run --release -p zgui-bench --bin zgui-bench -- click s13 24

The phase names are idle, hover, click, scroll, glide, keys, resize and scroll-resize, plus the differential phases that compare a live window against a rebuilt one. The sizes are s0, s1, s2, s4, s8, s13 and s26; s13 is the shipped gallery. all in place of a size runs every size, smallest first, and stops at the first that fails.

The reference workloads run on their own:

cargo run --release -p zgui-bench --bin static-slope
cargo run --release -p zgui-bench --bin list-slope
cargo run --release -p zgui-bench --bin resize-slope
cargo run --release -p zgui-bench --bin unvirtualised-probe   # about ten minutes
cargo run --release -p zgui-bench --bin glide-split           # about forty seconds

ZGUI_BENCH_GPU=1 swaps the null renderer for this machine's graphics device, offscreen.

The profiling counters

The counters are the sharper half of every claim on this page. A time measures the machine as much as the code; a count measures the design.

They live in zgui-profile. That crate is not re-exported through zgui, so an application that wants to read counters names it directly:

[dev-dependencies]
zgui-profile = { version = "0.1.0", features = ["counters"] }

How they are compiled in

// crates/zgui-profile/src/counter/store/mod.rs
pub const COUNTERS_ENABLED: bool = cfg!(any(feature = "counters", debug_assertions));

When COUNTERS_ENABLED is false, every counter function is an empty inlined body and the storage behind it does not exist.

In a shipped release build it is true. crates/zgui-runtime/Cargo.toml names

zgui-profile = { path = "../zgui-profile", version = "0.1.0", features = ["counters"] }

as an ordinary dependency, not a dev-dependency, so cargo's feature unification turns counters on for every application built on the framework.

The shape of the table

One macro declaration builds the enum, the metadata and the snapshot struct together, so a counter cannot exist in one and be missing from the other. Each of the 70 rows is a variant, a snapshot field and a group (crates/zgui-profile/src/counter/table.rs):

ElementsRestyled => elements_restyled, Group::BackendNeutral;
LayoutsHeld      => layouts_held,      Group::Skip { done: Counter::NodesRelaidOut };
DrawCalls        => draw_calls,        Group::RendererSpecific;

The group answers two questions that go quietly wrong when they are written down somewhere else:

Six counters are skips today: LayoutsHeld, SizesHeld, PlaceWritesWithoutReemit, ChunksTranslated, PrimitivesCulled and SpritesResolvedAtPush.

The reading API is small:

use zgui_profile::{Counter, counter};

let _guard = counter::exclusive();   // the counters are process-wide atomics
counter::reset();
let before = counter::snapshot();

// … drive one interaction …

let moved = before.delta(&counter::snapshot());
assert_eq!(moved.get(Counter::GlyphsRasterised), 0);

Counters prints only its non-zero fields, which makes a Debug of a delta a readable account of what one interaction did. Nothing resets the counters per frame on its own.

Four counters distinguish the spike paths covered by the text and upload caches:

The non-vacuity rule

A counter of work performed is falsifiable alone: it reads zero when the stage did nothing and a number when it did something.

A counter of work avoided is not. It reads zero when the stage is perfect, zero when the stage has been deleted, and zero when nobody ever wired it up. So skipped <= n is green from the day it is written, whatever happens to the code underneath.

assert_non_vacuous is what makes such a counter mean something. It requires three things:

use zgui_profile::Counter;
use zgui_profile::counter::non_vacuity::{Scenario, assert_non_vacuous};

assert_non_vacuous(
    Counter::ChunksTranslated,
    Scenario::new("scrolling a document already painted once", scroll_a_painted_document),
    Scenario::new("the first paint of a fresh document", open_a_fresh_document),
);

cargo xtask skips searches every test target in the workspace for calls to this function and fails the build for any skip counter that has none.

The latency trace

The counters say how much work happened. The Phase spans say how long each stage took. Neither says anything about the wall time between one stage ending and the next beginning, or about the time an event spent waiting for a frame to be asked for. Those gaps are what a person means when they say an application feels slow.

So the latency trace records instants, not durations. Every gap between two consecutive instants is visible whether or not anybody thought to name the thing that filled it.

Turn it on with an environment variable and nothing else:

ZGUI_LATENCY=/tmp/trace.jsonl cargo run --release

The winit backend calls start_epoch() when the event loop starts and flush() every time the loop parks, so a desktop application needs no code change. The output is newline-delimited JSON, one object per mark:

{"t":142900,"base_unix_ns":1785456583000000000,"stage":"b.why","note":"owed=1 spliced=1 noroot=false rebuild=false"}

Marks are named for the stage boundary they sit on: loop.wake, evt.in, f.begin, f.restyle, f.boxes, b.why, f.layout, f.fragments, p.emit, p.draw, r.tables, r.vectors, r.plan, r.encoder, r.buffers, r.record, pres.out, f.end, wait.in, wait.out. docs/perf/endtoend.md lists the full vocabulary in the order one turn produces them.

The r.record mark has a note only when the upload belt allocated chunks. The note gives the allocation count for the r.buffers interval immediately before it. This turns an occasional Update GPU buffers spike into a specific growth or backlog signal instead of an undivided time.

When nothing is listening a mark costs one relaxed atomic load. That only holds if the note is built after the check, so a note that allocates is written with note_with, which takes a closure:

use zgui_profile::latency;

latency::mark("f.fragments");
latency::note_with("f.restyled", || restyled.to_string());

A mark whose note is built before the enabled check is paid for on every frame of every run, recording or not, which makes the instrumentation part of what it is measuring.

There is a second sink for a reader inside the process, because a tool drawing the shape of the frame on screen cannot read a file it is itself still writing. latency::retain(n) keeps the last n marks in a ring; latency::recent() returns them as Recorded { stage, note, at_ns }; latency::retain(0) stops. The capacity is fixed by the first non-zero call.

Measuring your own frames

A window's interesting state exists only for the instant between the frame that produced it and the next frame that overwrites it. A frame probe is called once at the end of every frame and handed the window as it stands.

use std::cell::Cell;
use std::rc::Rc;
use zgui::prelude::*;
use zgui::runtime::{FrameProbe, Window};

/// Counts frames and remembers how many rectangles the last one redrew.
#[derive(Default)]
struct Frames {
    frames: Cell<u64>,
    rects: Cell<usize>,
}

impl FrameProbe for Frames {
    fn frame_ended(&self, window: &Window) {
        self.frames.set(self.frames.get() + 1);
        self.rects.set(window.damage().len());
    }
}

fn main() -> Result<(), zgui::Error> {
    app()
        .with_probe(Rc::new(Frames::default()))
        .run(|| view! { column() })
}

The trait has one required method and no return value. It takes &self, so a probe that wants to keep something writes it through a cell or a signal — which is also what lets one probe serve several windows. A probe that mutated the window would change the thing it was measuring, and a probe that could refuse a frame would be a second frame loop. WindowOptions::probe is an option and not a list: one seam, one occupant.

What a probe can read off the Window:

Nothing in the framework implements FrameProbe. The bundled inspector does, and an open inspector redraws every frame, because what it shows changes every frame.

Environment variables

ZGUI_FULL_DAMAGE exists so that "is this artefact a damage-tracking bug?" is one restart away from an answer. It is the first thing to try when a visual artefact is reported. ZGUI_INVARIANTS costs a lookup per distinct raster per replayed fragment, so it is opt-in rather than compiled in.

How cost scales

Two results from docs/perf/reference-workloads.md, both measured on the shipped documents rather than on the probe.

A one-control update does no work proportional to the document — and still walks it. At every one of the four sizes, a single-property change on one control of ten thousand restyles one element, lays out none, diffs six fragments, emits one primitive and takes one draw-order insertion. It also visits controls + 17 nodes. That walk is the whole of the seventy-odd nanoseconds per control in the advisory table. Counter::NodesVisited exists for exactly this case: every other counter records work performed, so a traversal that touched ten thousand clean nodes to service one dirty one would satisfy all of them.

A glide over a virtualised list costs about nine tenths of repainting the same rows from their styles (LIST-glide = 0.92). Carrying rows past a port is very nearly as expensive as drawing them again.

The cliff is what virtualisation buys, and the numbers are worth reading twice:

Every one of those numbers is true. Only the first is about a document somebody ships, which is the whole reason the probe is not a gate.

docs/perf/glide-split.md puts the same finding another way: at 70 047 boxes a glide frame is 7.25 ms on the maintainer's machine, so a rigid scroller crosses a 16.6 ms budget at roughly 160 000 boxes, about 23 000 rows of that shape.

A wheel over a virtualised list costs, per model row, between −1.7 and +2.5 ns — indistinguishable from zero, which is the point, and what LIST-virtualisation-wheel gates at ≤ 1.05 for eight times the data.

Next