Caches and memory
Every cache in the pipeline, what fills and invalidates it, what a miss costs, how large it may grow, and how the budget keeps memory bounded.
zgui keeps the result of each pipeline stage between frames and reuses whatever is still valid. This page lists each of those stores in pipeline order, from style to drawing, and then explains how their memory is bounded. The cost model gives the cost of each kind of change; the architecture pages linked from each section describe the mechanisms in depth.
Inventory
| Cache | Holds | Bounded by |
|---|---|---|
| computed styles | one allocation per distinct cascade result | the live document |
| box tree | the boxes, spliced rather than rebuilt | the document |
| per-box layout | one final layout, 11 size answers, intrinsic answers per axis | fixed slots per box |
| shaped paragraphs | one shaped result per distinct content and style | 16 384 entries, soft |
| recall buffer | line-breaking results | 4 per paragraph |
| hit index | one entry per fragment | the fragments |
| paint-style lowering | one lowered paint description per distinct style | distinct styles since the last scale change |
| paint records | the primitives each fragment drew | 64 MiB, soft |
| glyph cache | placement and extent per glyph, blank glyphs included | the atlas, plus 4 096 blank answers and 4 096 eviction records |
| atlas | glyphs, pictures, small vector masks, path-glyph sheets | 64 MiB, soft |
| vector caches | parsed drawings, placements, recognitions, payloads, CPU layers, encodings | see the section |
| scene side tables | clips, paints, text paints, shader parameters, coordinate systems | reference counts and generations |
| bounds tree | draw order for this frame's primitives | this frame |
| render targets | offscreen group targets and scroll-shift scratch | the target pool's own ceiling |
| device memory | live renderer allocations and zgui-owned callback-surface textures | pinned; never evicted |
Eight of these report to the budget registry, which measures retained memory and releases what can be rebuilt. The rest are bounded by the document, by the frame, or by a fixed number of slots.
Computed-style sharing
A computed style is the cascade's result for one element. It holds one shared allocation per property group (font, background, border, and so on), and elements that cascade to the same result share those allocations.
| Filled by | the cascade |
| Invalidated by | a restyle that produces a different result; the old allocation is freed when its last holder drops it |
| A miss costs | one more pass of whatever a consumer does per style |
| Bounded by | the live document |
Later stages key their caches on these pointers, so they work once per distinct style rather than once per element. In a component library that is one to two orders of magnitude fewer.
- Equal pointers prove equal values, but unequal pointers prove nothing. Cascade workers can each build their own copy of an identical group, so a cache keyed on pointer identity needs a content-hashed fallback. Paint-style lowering is an example.
- An address is an identity only while the allocation lives. A table that outlives a frame holds
a
PinnedGroup, which keeps the group alive so its address cannot be reused. Text colour slots are keyed this way.
See Computed-style sharing in the style engine.
The box tree
Layout runs over the box tree: one or more boxes per element, plus anonymous boxes. Fragment reuse, geometry diffing, paint records and damage are all keyed by box.
| Filled by | the box builder |
| Invalidated by | an element that owes a rebuild after a structural change |
| A miss costs | building that element's boxes and splicing them in; a whole build when the change cannot be confined |
| Bounded by | the document |
A whole build gives every box a new identity, so nothing keyed by box can be reused: every fragment compares as changed and the damage becomes the whole window. zgui therefore splices each changed element's boxes into the existing tree, rewrites text runs in place where it can, and builds the whole tree only when a change cannot be confined to the element. The most common case that still falls back to a whole build is an out-of-flow box whose layout parent and paint parent differ. Updating the box tree has the rules.
Counters: BoxesRebuilt, and BoxTreeBuilds for whole-tree builds (the first one included). The
b.why latency mark records how many elements owed a rebuild, how many were spliced, and whether the
whole tree was built.
Per-box layout
Layout algorithms ask a box several questions before placing it: how narrow it can be, how wide it wants to be, how tall it is at a given width. Each unanswered question is a nested layout of the box's subtree, so each box caches its answers.
| Filled by | every layout question a box answers |
| Invalidated by | marking the box dirty, and a change of device scale, which empties every box's cache |
| A miss costs | one nested layout of the box's subtree |
| Bounded by | one final-layout answer, 11 size answers, one intrinsic answer per axis |
- One final layout. Performing a layout writes positions into every descendant, so only the most recent answer can be trusted.
- Eleven size answers, which exist mainly for grids: track sizing asks each item several min-content and max-content questions. When all eleven are full, the least recently used is evicted, which costs a measurement and never a wrong answer.
- Equivalent questions share an answer. A growing sibling does not evict answers that cannot depend on it.
- Intrinsic answers persist. A box sized by
fit-content,min-contentormax-contentkeeps its min-content and max-content answer per axis across frames and viewport changes, so a settled document measures nothing.
See Measurement caches.
Counters: SizesHeld against SizesMeasured, LayoutsHeld against NodesRelaidOut.
Shaped paragraphs
Shaping turns characters into positioned glyphs. It is the most expensive text stage, and it is cached separately from breaking, which decides where lines end.
| Filled by | layout reaching a paragraph that has not been shaped |
| Invalidated by | eviction of inactive entries, an element moving to a different text-paint slot, or an explicit reset |
| A miss costs | one shaping pass over the paragraph |
| Bounded by | 16 384 entries, soft; paragraphs in the current layout are never evicted |
The key covers the text, the device scale, the runs and their shaping properties, the widths of inline boxes, and the base direction. Two things are deliberately not in it:
- The colour. The key holds the text-paint slot the glyphs draw through, not the colour in the slot. Recolouring text, through a theme change or a colour transition, rewrites the slot and reshapes nothing. An element whose colour diverges from others sharing its slot gets a slot of its own, which reshapes that element's paragraphs only.
- The width. Width belongs to breaking, so a resize re-breaks lines without reshaping.
An edited paragraph leaves its old entry behind until the budget needs room, and eviction removes only inactive entries, least recently used first. When a frame has 16 or more paragraphs to shape, they are shaped on the layout workers before layout runs; fewer are shaped on the frame thread, where that is cheaper. See The paragraph cache.
Counters: TextShaped and TextBytesShaped (one long paragraph and one short label are each one
shape, but not the same work), ParagraphsEvicted, ParagraphsForgotten.
The recall buffer
Each shaped paragraph remembers the last four widths it was broken at.
| Filled by | every breaking pass |
| Invalidated by | a fifth distinct width, which drops the oldest |
| A miss costs | one breaking pass over the shaped glyphs |
| Bounded by | four results per paragraph |
Layout asks three questions in a row (how narrow, how wide, how tall at the final width) and repeats them on its next iteration, so three slots make the repeat free. The fourth covers the extra width a nested grid adds. See Line breaking.
Counter: TextRebroken.
The hit index
The hit index answers which element is under a point.
| Filled by | the fragment pass, one entry per fragment, updated in place |
| Invalidated by | a change of paint order, or churn past a quarter of the entries |
| A miss costs | a rebuild over every fragment, in paint order |
| Bounded by | one entry per fragment, one spatial tree per coordinate system |
A scroll or transform moves a whole run of entries by the same vector, which does not count as churn, so scrolling never rebuilds the index. See The hit index.
Counters: HitEntriesUpdated, HitEntriesMovedInPlace, HitEntriesReinserted, HitIndexRebuilds.
The scroll scenario holds hit-index rebuilds at zero.
Paint-style lowering
Lowering turns a computed style into what the paint stage draws from: colours resolved, lengths in device pixels, shadows and borders in primitive form.
| Filled by | the paint walk, once per distinct style |
| Invalidated by | a change of device scale, which clears it |
| A miss costs | one lowering, plus a content hash when the pointer lookup misses |
| Bounded by | the distinct lowerings since the last scale change |
The cache is looked up by the style's group pointers first and by a hash of the lowered content
second, so styles that are equal but separately allocated still share one entry. The result is a
PaintStyleRef, a Copy index that a paint record can store and compare on the next frame. The
lookup is described under
Computed-style sharing.
Counters: StylesLowered against StylesLoweredFromCache.
Paint records
A fragment is one rectangle of one box after layout. Each fragment's paint record keeps a copy of the primitives it produced, so an unchanged fragment costs a replay instead of an encoding.
| Filled by | every fragment the paint walk encodes |
| Invalidated by | a change in any compared paint input, a change of kind or size, fragment retirement, or budget eviction |
| A miss costs | encoding the fragment again from its lowered style and geometry |
| Bounded by | 64 MiB, soft; records used this frame are never evicted |
A record compares one small signature of everything that can change the painting: the lowered style, clip, transform, decorations, animations, inherited opacity, caret and selection highlights, the device scale, and what the fragment draws. A text line carries its paragraph's identity, so one changed character re-encodes the line even at the same width.
A replay can be offset, which is how a scrolled row is drawn without encoding. Replays still go through this frame's clipping, culling and draw-order assignment, so a row that was outside a scrollport can enter it without re-encoding. A record holds each atlas tile its primitives draw, so eviction cannot free a tile a replay still needs. Records survive frames that do not visit them; under memory pressure the least recently used records outside the current frame are evicted. See Recording and replay.
Counters: ChunksTranslated against ChunksReencoded, ChunksRetainedUnvisited, ChunksEvicted,
FragmentsRetired, and the gauge PaintChunkBytes.
The glyph cache
A glyph is one character's shape in one face at one size. Rasterising it runs the face's hinting program.
| Filled by | the paint walk placing a glyph |
| Invalidated by | eviction of its atlas tile, and a lost device |
| A miss costs | one rasterisation and one upload |
| Bounded by | the atlas for tiled entries, plus 4 096 blank answers and 4 096 eviction records |
The key holds the face, glyph index, size, subpixel phase, synthesis and coverage form, so the same letter at the same phase is one tile however often it appears, and text scrolled by whole pixels keeps all its tiles. The cache stores each glyph's placement as well as its tile, so a repaint never rasterises just to find out where the pixels go. It also stores blank glyphs: a space has no pixels and no tile, and without a separate answer every space would be rasterised on every full repaint. See The glyph cache.
Per glyph (docs/perf/endtoend.md) | |
|---|---|
| rasterised | 32.0 µs |
| served from this cache | 0.32 µs |
Counters: GlyphsPlaced against GlyphsRasterised, and RebuiltAfterEviction for glyphs rasterised
again because their tile was freed. Any GlyphsRasterised on a repaint of unchanged text means the
cache is not being used.
The atlas
An atlas is a large texture that many small images are packed into, so a thousand glyphs draw from one texture. One atlas serves glyphs, pictures, small vector masks and path-glyph sheets, because they compete for the same memory and a fixed split would suit some documents and not others.
| Filled by | glyph, mask and path-glyph rasterisation, and image upload, during the paint walk |
| Invalidated by | eviction, and a lost device |
| A miss costs | producing the content again and uploading it |
| Bounded by | a soft limit of 64 MiB per window (ATLAS_SOFT_BYTES) |
A new texture is 1 024 pixels square, or larger for content that does not fit, up to the device's
maximum texture size. A pool holds at most 16 textures (crates/zgui-atlas/src/atlas/limits.rs). The limit is soft: the atlas evicts back towards it but
never refuses an allocation the current frame needs, so a frame whose working set is larger than the
limit stays over it.
- Each frame is a generation. Eviction frees the untouched, unreferenced entries of the oldest generation, one generation at a time.
- Memory is returned only when a whole texture empties.
- Eviction reports every key it frees, so the glyph and mask caches drop their entries at once.
Decoded pictures have a separate budget (DecodedImages, 64 MiB soft). The variant of each source a
live image shows is kept; other variants go first, then sources nothing shows, coldest first. See
Atlases.
Counters: AtlasTilesEvicted, RecordTilesRetained and RecordTilesReleased. A growing gap between
the last two means something holds tiles and never releases them.
Vector caches
Vector content (the vector element, path notation, SVG documents and canvas scenes) is drawn by
the cheapest route that can draw each shape, and
each route has its own cache.
| Store | Holds | Keyed by | Bound |
|---|---|---|---|
| parsed drawings | the shapes read from one source text | the text; shared by every element drawing it | while any element draws it |
| placed drawings | one drawing per node, with its fit | the source and the six coefficients of the fit matrix | the live nodes; 16 384 entries (VectorResources) |
| recognitions and shape payloads | analytic quads and mark payloads recognised from a shape | the shared path allocation | 4 096 entries; entries no frame looks up expire after a few frames |
| series payloads | one built payload per series part | the data allocation | 64 entries, 600 frames unused |
| mask tiles | coverage for small solid paths | geometry and density, not colour or position | the atlas; a per-frame budget for new masks |
| path-glyph sheets | each repeated outline at 16 subpixel positions | the outline and its style, per device scale | the atlas; at most 64 new sheets per frame |
| CPU layers | whole drawings rasterised into the atlas's image pool | the drawing, scale and 1/16-pixel phase | four surfaces of RGBA8, 16–128 MiB (VectorLayers) |
| general encodings | one encoding per distinct path, paint and style | the shared path allocation | 64 MiB, least recently drawn first |
Three properties follow from the keys:
- Colour is never part of a geometry key, so a recoloured drawing reuses all its geometry.
- Geometry is identified by its allocation, so the same source text or
Arcreaches the same entries everywhere. - Moving reuses everything. Shapes stay in their own coordinate space and the fit travels beside
them, so a new box size is a new fit, not a new path, and a moved drawing replays at an offset
(
VectorReplaysMoved).
New rasterisation is also budgeted per frame. Content over the budget is drawn by another route or a
frame or two later, and counted (VectorMaskBudgetOverflow, VectorLayersDeferred,
VectorLayerTilesDeferred). Writing fast interfaces
turns these properties into guidelines.
Scene side tables
Clips, paints, shader parameters and coordinate systems are interned in persistent tables: the same content returns the same identifier. Text paints are mutable slots, so a text colour can change without reshaping. Each table is reference-counted and generation-marked, and frees the coldest entries nothing refers to. A coordinate system is named after the box that establishes it, so moving a box writes one entry, and a thousand untransformed rows share one.
The bounds tree
The bounds tree assigns draw order by finding which already-inserted rectangles a new one overlaps. It is cleared each frame, so its cost follows the primitives emitted, never the document. Content that does not overlap shares low orders, so a hundred disjoint boxes all get order one and can be drawn in one batch. Inserts and queries reuse their buffers and do not allocate. See Draw order.
Caches keyed on device pixels
A change of device pixel ratio invalidates everything measured in device pixels
(crates/zgui-runtime/src/window/scale.rs):
| State | What a scale change does |
|---|---|
| computed styles | nothing; they are in CSS pixels |
| per-box layout | emptied explicitly |
| shaped paragraphs | keyed by scale, so they miss and reshape |
| rasterised glyphs | keyed by device size, so they miss and rasterise |
| lowered paint styles | cleared on the first lowering at the new scale |
| paint records | carry the scale in their signature, so they re-encode |
| scroll offsets | multiplied by the change, so the view stays in place |
The layout cache has to be emptied explicitly because it is keyed by the question, and some questions, such as a min-content probe, carry no size and look identical at every scale.
The budget registry
Retained state without a budget only grows. Eight caches register with each window's budget registry, each with its unit and level:
| Cache | Unit | Level |
|---|---|---|
PaintChunks | bytes | 64 MiB |
VectorLayers | bytes | four surfaces of RGBA8, clamped to 16–128 MiB |
GlyphAtlas | bytes | 64 MiB |
DecodedImages | bytes | 64 MiB |
ParagraphShaping | entries | 16 384 |
VectorResources | entries | 16 384 |
RenderTargets | bytes | none; the target pool enforces its own ceiling |
DeviceMemory | bytes | none; pinned renderer allocations and zgui-owned callback-surface textures |
16 384 sits above the two workloads that bound it: the largest document held entirely live (about
5 000 nodes) and the most distinct content a run leaves behind (a 10 000-row scroll list). No
workload in the repository evicts at that level; it is a margin, not a measured capacity
(crates/zgui-runtime/src/budget/limits.rs).
DeviceMemory excludes pooled targets already reported by RenderTargets. Textures owned by an
embedded surface's producer are reported by Window::embed_memory_report() and never released by the
budget.
How a cache comes back under its level
Once per frame, after the paint walk and the atlas flush, the registry records what each cache did, collects what each holds (with its pinned share and rebuild cost), and enforces the level on caches that are over it. On an ordinary frame that is none.
Eviction takes speculative content first, then the coldest content, with ties broken by the lowest rebuild cost. Rebuild cost is only the tie-break: ordering by it alone would evict the glyphs on screen, which are cheap precisely because they are drawn every frame.
| Cache | When over its level |
|---|---|
| paint chunks | evicts records not used this frame, oldest first, releasing their holds |
| vector layers | removes layers nothing holds, then drops the coldest records that hold layers |
| atlas | frees cold generations; stops at the frame's working set and anything a record holds |
| decoded images | removes inactive variants, then unseen sources, coldest first |
| paragraph shaping | removes inactive entries, least recently used first; active text stays |
| vector resources | drops all placements; they are rebuilt from the same source and box |
Two seconds after the last frame, an idle window also releases its working memory (unused offscreen targets, scratch buffers, completed uploads) without drawing; see Idle maintenance.
Read and change the limits
let report = window.last_budget_report();
for line in report.lines() {
// line.id, line.report.resident, line.report.pinned, line.limit
}Window::last_budget_report returns the report the frame's budget step already took, so it costs
nothing. Window::cache_limits and Window::set_cache_limits read and change the levels in
CacheLimits (shaped paragraphs, placed drawings, decoded-image bytes, paint-chunk bytes); the atlas
level is set with ContentCache::set_soft_bytes. Window::forget_caches drops everything that can
be dropped, damages the whole surface and requests a frame.
The budget gate
cargo xtask budget runs four test targets against real windows on the headless platform:
| Target | Claim |
|---|---|
evict_budget | a cache over its soft limit comes back under it within a bounded number of frames, without thrashing |
evict_replay | no replayed record uses a raster its cache no longer holds |
evict_pinned | the most aggressive eviction cannot take a raster something holds |
budget_registry | every registered cache reports itself, comes back under its level, and can be emptied |
evict_budget holds a window to 2 MiB and drives 200 turns of 250 never-seen characters. Resident
bytes may stay over the limit for at most 4 turns in a row, and over a later 60-frame window,
re-rasterised glyphs must be under a twentieth of glyphs placed.
What grows with the document
| Per | Held |
|---|---|
| document node | an 8-byte invalidation word, a dirty-child record, the core record, and entries in the side tables it uses |
| box | a box node and a layout record (the layout cache slots, baselines, any resolved inline lines) |
| fragment | the fragment, a hit-index entry, and a paint record until retirement or eviction |
| distinct style | one lowered paint description |
| distinct paragraph | one shaped result and up to four breaking results |
| distinct glyph key | one cache entry, and one atlas tile unless it is blank |
| interned name | one shared copy, for the life of the process |
The shipped gallery is 1 851 boxes and 2 363 fragments, 17.6 % of the boxes anonymous.
Three stores hold more than the current frame's working set, which is why the budget exists: shaped paragraphs (every distinct string a virtualised list has shown), paint records (fragments that are not visited keep theirs), and interned names. Interned names are never freed, so never intern unbounded or user-supplied text; see Interned names.
Nodes, boxes and fragments live in arenas with stable addresses, and the box arena is recycled, so a whole box-tree rebuild reuses slots instead of growing. See Stable addresses.
Next
Measurements
Measure your own application, and the benchmarks and gates behind the numbers.
Writing fast interfaces
Guidelines that follow from what these caches are keyed on.
Cost model
What each kind of change costs, in numbers.
Paint and scene
The display list, the stacking-order walk, and how a paint record is replayed.