Performance¶
valgebra compiles a schema once into a Rust validator tree and crosses into Rust exactly once per validation call. This page records how that is measured, a reproducible baseline against other validators, the honest limits of the numbers, and the techniques behind them.
A speed claim is only as good as its methodology. Every number here states the harness, the dataset, the library versions, and the machine class. Re-run the harnesses on your own hardware before relying on a ratio: absolute times move with the CPU, and the comparison points do different amounts of work.
What is measured¶
Two harnesses, one per side of the boundary:
- Core micro-benchmarks (
crates/valgebra-core/benches/core.rs, criterion) time the pure-Rust schema transformations — the index remap behind validator composition and the recursive open/closed record transform — and the relations, a rule's answer (subtype_*,is_empty_*) beside the set representation's (lower_*). No Python is involved. - End-to-end benchmarks (
benches/, pytest-benchmark) time a single boundary-crossing validation call through the public API, over synthetic shapes that each stress one cost dimension.
Run them with:
# Core micro-benchmarks (Rust):
cargo bench --bench core
# End-to-end and comparison benchmarks (Python); install the bench group first,
# without the project, which the wheel below provides:
uv sync --locked --no-install-project --group bench
# To match the published figures, build the same PGO wheel the release ships
# (needs the llvm-tools rustup component) and install it; a plain build is slower.
# Every `uv run` from here on takes --no-sync: a sync reinstalls the project over
# the wheel, and the run then times a different build from the one installed.
uv run --no-sync --group bench maturin build --release --pgo --out dist
uv pip install --no-deps --reinstall dist/*.whl
uv run --no-sync --group bench pytest benches/bench_validate.py
uv run --no-sync --group bench pytest benches/bench_compare.py --benchmark-group-by=group
Comparison is not apples-to-apples¶
The comparison runs the same shapes through three checkers that do different work. Read the ratios with that in mind:
- valgebra checks membership of the object already in hand: no copy, no
coercion.
is_validreturns a bool through the membership fast path. - jsonschema (
Draft202012Validator.is_valid) is also a pure check with no coercion — the closest semantic analog — but it is pure Python. - pydantic (
TypeAdapter.validate_python, strict mode) validates and constructs a value. Strict mode disables coercion, but it still builds and returns output, so it does strictly more work than a membership check. It is the relevant point of comparison because it is the Rust-cored validator most users reach for.
The record shape compares valgebra's closed record against a pydantic
TypedDict and a jsonschema object with additionalProperties: false, so all
three check the same set of named fields.
Two libraries check an object in hand against its annotation, as valgebra does,
and the matrix leaves both out: typeguard and beartype. Neither reads
every element by default. typeguard checks the first item of a collection
unless its collection_check_strategy is ALL_ITEMS, and beartype checks one
item drawn at random, so [1] * 999 + ["x"] passes as a list[int] on every
call of the first and on most calls of the second. A ratio against a check that reads one element measures what
sampling buys, not the same work; typeguard under ALL_ITEMS does the same
work, in Python.
Baseline matrix¶
Machine class: AMD Ryzen 7 PRO 7840U (Zen 4, 8c/16t, up to 5.1 GHz, a 2023-era
mobile part) under WSL2 on Linux 6.18. Toolchain: rustc 1.100.0 (the build these
numbers are measured on; the supported minimum is the lower rust-version in
the manifest), CPython 3.15.2 built from source with
--enable-optimizations --with-lto, CC=clang and
CFLAGS=-march=native -mtune=native, and the GIL enabled
(sysconfig.get_config_var("Py_GIL_DISABLED") is 0; the free-threaded build
of the same version takes 1.4 to 2.8 times as long to walk a container, so a
figure measured on one is not comparable with the other). Every package version -- the two
benchmark harnesses and the libraries compared against -- is whichever the bench
group resolves. uv.lock owns them and scripts/compare_gate.py prints them
beside the figures, so none is written here: a version copied into prose is
stale the next time the lock moves, and a reader cannot tell a stale one from a
current one.
sysconfig.get_config_var("CONFIG_ARGS") reports that build, and is how to
check you are on it. The native tuning is the flag that matters when reproducing
these numbers: a stock distribution interpreter is a different binary, and so is
the one uv sync provisions for this repository, which is a
python-build-standalone image rather than a source build. Point the bench
environment at the interpreter you mean -- the sync rebuilds .venv on it --
uv sync --locked --no-install-project --group bench --python /path/to/that/python
uv run --no-sync --group bench maturin build --release --pgo --out dist
uv pip install --no-deps --reinstall dist/*.whl
uv run --no-sync --group bench python scripts/compare_gate.py
-- because a figure measured against another binary is not comparable with a figure here.
The extension is the PGO release build — the profile-guided, fat-LTO wheel
the release ships, which the maturin build --release --pgo step above builds.
pydantic's PyPI wheels are likewise PGO-built, so this is a release-to-release comparison.
Read no figure taken from a debug build as either -- maturin develop
without --release installs one, it is indistinguishable from the release
extension at the Python prompt, and a timing of it reads an order of magnitude
slow. scripts/compare_gate.py refuses such a build outright, and so does
benches/ from its own conftest.py; a figure timed by hand has only the habit
to protect it.
Whether to add PGO is a question about your shapes, not a setting to turn
on. Build both, install each in turn, and time the shapes you run on it -- the
sequence the pgo compare lane in .github/workflows/ci.yml runs:
uv sync --locked --no-install-project --group bench
uv run --no-sync --group bench maturin build --release --out plain
uv run --no-sync --group bench maturin build --release --pgo --out profiled
uv pip install --no-deps --reinstall plain/*.whl
uv run --no-sync --group bench python scripts/pgo_compare.py \
--record plain.json --label plain
uv pip install --no-deps --reinstall profiled/*.whl
uv run --no-sync --group bench python scripts/pgo_compare.py \
--record profiled.json --label pgo
uv run --no-sync --group bench python scripts/pgo_compare.py \
--compare plain.json profiled.json
That script times every shape the competitive gate judges -- it reads them from
scripts/compare_gate.py -- so the two tables speak of the same workloads. It
refuses a reading from a debug build and refuses to compare two readings from
different interpreters, because a ratio between builds cancels the machine and
not the interpreter.
Measured that way on the box above, six readings of each build interleaved, the
profile buys on six shapes, by the median of the six: the list nested
twenty-five deep 36%, the JSON document 25%, the one-int call 19%, compilation
18%, the error report 5%, and the accepting walk over a fifty-field record 4%,
between 3% and 5% across the six. The ten-thousand element list[int] reads
10% slower, between 5% and 13%. A single reading of
one shape is not a verdict either way: the first reading after an install can
part from the five after it by 12%.
The shape the profile costs is the one with the least left to arrange, and what moves it is where its code lands rather than what its code does. A list of one scalar kind is read by a loop of its own for that kind, a type test per element, and that loop counts within 0.9% with a profile and without one. On CPython 3.15 it takes 0.94 ns an element in the plain wheel and 1.04 in the profiled one. Two profiled wheels built from one source count it to the instruction and take 1.0 and 1.2 ns, so this figure follows where a build lands the loop, the effect aligning code to 64 bytes reads on another list. The shapes that gain are the ones a call crosses several functions to answer: the boundary itself, a descent through twenty-five levels, a parse and then a walk.
The reading the release matrix is decided on is taken where the release
builds. The pgo compare lane builds both wheels from one source, times every
shape on each, and uploads the two readings, on both interpreters the bench jobs
use -- a global lock changes what a per-element loop costs, and the shapes a
profile serves best are the per-element ones:
The training workload is the lever, not the flag: the profile is taken over
scripts/pgo_workload.py, which pyproject.toml's pgo-command names, and
what that workload spends its time on is what the layout is arranged for. A
change to it is measured on both sides of the comparison above, since a profile
re-weighted toward one shape is a profile taken away from another.
A loop the profile holds no counts for is laid out by chance. A homogeneous
list takes a loop of its own, so a workload reading only homogeneous lists holds
no counts for the general scan every list of records and every nested list is
read through, and whether that scan stays inside the recursive walk is the
inliner's guess. A change to the list arm's code can turn it: untrained on the
scan, the comparison gate's list nested twenty-five deep takes 64% longer in the
PGO wheel, 405 ns against 245 on one box, while the instruction gate's plain
workloads, which build without a profile, read 0.9%. So the workload reads a list
of records and a list nested twenty-five deep, and trained on them that shape
reads 0.101 of pydantic's time on CPython 3.15. The instruction gate holds the
training too, counting the comparison gate's shapes on the PGO wheel built at a
change and at its base (perf_gate.py --profiled): built with the nested list's
training taken out, that shape reads 50.89% more instructions.
A record inside a list is laid out by the lists of records the workload reads, and weighted lightly, its reading moves with code beside it that it never runs: trained a quarter as often, on records of scalar fields alone, a list of two hundred records whose record holds a list reads 1.3% more instructions on the profiled wheel. So the workload reads its lists of records at the weight a document's records are read with, one of them of records holding a list.
A path's weight moves a path it does not touch. Trained at 2,000 rounds, a list of refined floats reads the list nested twenty-five deep 6% to 9% dearer in cycles at one count, the instructions on that list's path the same and laid out otherwise, on two builds of one tree. At 500 rounds that list reads between 1% cheaper and 4% dearer across two builds, the spread of a shape of that size, for 3% more instructions on the floats, and untrained the floats read 54% fewer instructions than through the rich comparison where trained they read 69%.
A parsed JSON array of records is read through a loop of its own
(record_array_matches), and untrained it is laid out by chance too: two builds
of one source read the comparison gate's document 10% apart in instructions, and
a source path one character longer is enough to move it. Trained, every build
reads one count, within 0.2%, and the document takes 12 to 14% less time than
untrained on CPython 3.14.
The same holds for a reader the workload never enters: the lists a reader of their own settles -- a union of literals, one class, a union of scalars, a tuple of scalars, a named tuple of them -- sit beside the general scan, and untrained, the readers for literals and classes move the PGO wheel's walk of a nested list by 6% in instructions. The workload reads a list of each kind, which takes that to 3%.
And for a door the workload never calls. x in v reaches the check through a
slot of its own, into which the build inlines a copy of it; untrained, that copy
took a mispredicted branch on every call and read 13% slower, and the workload
calls it at no cost to any shape the profiled gate counts. load parses into
Python objects through a builder no other call takes, and it is left untrained,
where it reads 49% slower than a build trained on it, on the comparison gate's
document and through validate_json alike, which parses the same way. That is a
trade the gate refuses: trained on a document of records, at every weight tried,
the profile lays the general list scan out for long lists, and the list nested
twenty-five deep, one element a level, reads 3.5 to 7% more instructions on the
profiled wheel and 2.5% more time.
What the matrix does with the reading. --pgo ships on five targets, and
that is a claim about those boxes rather than a default. Where the lane reads a
gain on the shapes the release serves, the matrix keeps pgo: true and this
page names the shapes it costs. Where it reads the record walk slower on both
interpreters -- the shape most callers spend their time in -- pgo: true leaves
the matrix and this page says so. Either way the decision is the reading's, and
the reading is the lane's.
That is one microarchitecture and one interpreter, and the release lane builds
on five targets none of which is this one, so it is a reason to measure your own
build rather than a ranking. scripts/compare_gate.py against each wheel is how.
Every wall-clock figure on this page, and in the changelog, is a release build on an idle machine, read over repeated runs -- a median of ten for a table, taken as two sets of five, and the comparison gate's minimum for a ratio, taken twice. The two guards above hold the first of those and neither holds the other two: a timing taken while something else has the CPU reads slow, and one taken once reads whatever that run did. Either is enough to put a published figure out by a factor, so the discipline is written down rather than assumed.
How much PGO adds over a plain --release build is not a constant this page can
state, and on some shapes it is not a gain at all. It is whatever the profile can
still arrange that fat LTO did not, so it shrinks as the hot paths get shorter,
and where the layout it picks suits one shape it can cost another.
The figures are measured on the wheel carrying valgebra's full feature set — the per-validator precompute (record-field lookups, literal-union dispatch) and native string patterns — which leaves these shapes unchanged: the features earn their keep elsewhere, not by regressing the core.
Method¶
Each cell is the median of ten independent runs of the comparison
benchmark, two sets of five, each run reporting pytest-benchmark's own median
over its rounds. The +/- is the half-range across the ten runs, not a standard
deviation: it states
the observed spread rather than modeling one. With the release wheel installed
as above:
The ratios have their own gate, which measures only valgebra against pydantic and takes the minimum over many repeats rather than a median:
That script owns the per-shape ratio ceilings (scripts/perf_compare.json)
-- what the project claims it stays under rather than a recorded measurement --
and the table below is the absolute record. The two estimators can part by a
quarter of a multiplier — a minimum sits below a median by however much the run
was disturbed, and the 3.15 results below read both — so read a cell here
against the same cell, not against the gate's output.
One command re-reads every figure on this page and on the
JSON page, on the checkout as it stands or on the
commit --rev names:
uv run --no-sync python scripts/page_figures.py \
--python-315 /path/to/that/python --python-314 /path/to/its/3.14
It needs uv, the llvm-tools component the PGO build needs, and the five
interpreters the tables name: the two passed, CPython 3.15 and 3.14 built from
source alike, and CPython 3.12 and the free-threaded 3.14 and 3.15 as uv
installs them, unless --python-312, --python-314t and --python-315t name
others. It builds everything it times under
target/page-figures/, every wheel from one directory, takes the readings one
at a time, and prints each figure under the name of the section that states it.
--stage runs one of those three steps alone, and --help says what each
holds. MEASURE in the script holds the repeats this section states, and
tests/test_page_figures.py holds the page to it.
On the machine above the build takes about ten minutes and the readings about seven, and the readings are a wall clock: run them on an idle machine, with nothing beside them. The script edits no page. A figure is copied in by hand once read, because a run something disturbed reads slow, and only a reader can tell that from a figure that moved. Of the figures under How the speed is made, it re-reads the three that state a cost of this tree: the reference pool's pair, the fifty refined fields' build and the report on a deep value. A figure stating a technique's before and after is a pair built where the technique landed, and the changelog's figures are each fix's own pair, on the build its entry names; it re-reads neither.
The cheapest door, and where the floor is¶
x in v is v.is_valid(x) through the container protocol, and it is the
cheaper call: 12 ns against 18 ns for a scalar on the machine above, the
best of fifteen timeit repeats of a million calls, because the interpreter
reaches a container slot directly and a method by its call protocol. Neither
number is the check. Validator(anything).is_valid(1) -- the schema that
answers True without looking -- costs 18 ns, nearly as much, since a validator
of one scalar kind is its type test and enters no walk, so what is left is the
boundary a Python call crosses. For reference on the same run,
isinstance(1, int) is 13 ns and a call to an empty one-argument Python function
is 17 ns.
Read that as the floor it is: a per-call check cannot be much cheaper than a Python call, and the way to spend less is to make fewer calls -- validate the list, not each element -- rather than to look for a faster scalar.
Results¶
End-to-end validation of a value that passes (lower is better):
| Shape | valgebra | pydantic (strict) | jsonschema |
|---|---|---|---|
list[int], 10,000 elements |
10.4 +/- 0.18 us | 87.2 +/- 1.2 us | 25,410 +/- 650 us |
| Closed record, 50 int fields | 0.481 +/- 0.015 us | 2.02 +/- 0.12 us | 133 +/- 3.1 us |
Nested list[...], depth 25 |
0.183 +/- 0.037 us | 2.04 +/- 0.045 us | 75.6 +/- 2.4 us |
valgebra relative to pydantic on this machine, under the CPython 3.15 the matrix above names, as the medians above divide: 11.2x faster on deep nesting, 8.4x on the large flat array, 4.2x on the wide record. The comparison gate's minimums read the same three at 10.2x, 8.4x and 4.6x (the 3.15 column below). It is consistently far ahead of pure-Python jsonschema — 2,438x on the array, 413x on the nesting and 276x on the record. The nesting's half-range is its slowest run of the ten, which read 0.250 us; the other nine read 0.177 to 0.193.
pydantic does strictly more work on the record (it constructs output), so read that shape as a margin over a heavier operation, not a like-for-like loss for pydantic.
One of those margins moves with the interpreter¶
A ratio cancels the machine — a slower box slows both sides — and it does not
cancel the interpreter. Running the comparison gate on one box against five of
them -- CPython 3.12.16 and the free-threaded 3.14.10 and 3.15.2 as uv installs
them, beside the 3.15.2 build named above and a 3.14.10 built the same way -- as
the fraction of pydantic's time each shape takes, the median over twelve runs on
3.12 and 3.14, eighteen on 3.15 and seven on each free-threaded build:
| Shape | CPython 3.12 | CPython 3.14 | CPython 3.15 | 3.14 free-threaded | 3.15 free-threaded |
|---|---|---|---|---|---|
list[int], 10,000 elements |
0.159 | 0.107 | 0.119 | 0.143 | 0.135 |
| Closed record, 50 int fields | 0.224 | 0.212 | 0.219 | 0.234 | 0.234 |
Nested list[...], depth 25 |
0.105 | 0.090 | 0.098 | 0.229 | 0.237 |
One int |
0.160 | 0.155 | 0.171 | 0.163 | 0.175 |
The element is what moves, not the check. A list hands out each of its items
as an owned reference — a count written on the object when the handle is made
and again when it drops — and the free-threaded build writes each count through
a call into the interpreter. A schema nested twenty-five deep is twenty-five
containers of one element, so it is almost nothing but that cost, and it takes
2.6 to 2.8 times as long there as under a global lock. A flat array of ten
thousand is read through a snapshot of the list under 3.12 and the free-threaded
builds, which pays the counts in two loops inside the interpreter and none in the
walk; CPython 3.14 and 3.15 with their global lock make the counts cheap enough
that the walk reads the list in place (snapshot_pays in
crates/valgebra-py/src/check/walk/sequence.rs). The array's cell parts those two
by where each build lands the loop: it reads 0.97 ns an element in the 3.14 wheel
and 1.04 in the 3.15 one, and two builds of one source read
1.0 and 1.2.
The margin carries across all five.
That is why the ceiling file holds a second set for the free-threaded build: what the project claims of that build is what that build can hold.
Threads on the free-threaded build¶
A Validator is immutable, so every thread can share one. What decides how
the free-threaded build scales is what the values the threads check share,
and the walk's reading of a container says how. A list, a dict and a set are
read under that container's own lock (with_critical_section in
crates/valgebra-py/src/check/walk/), and a list hands out each element as an
owned reference, whose count the free-threaded build writes on the element
itself:
| What the threads check | How it scales |
|---|---|
| one value, shared | not with the threads: they take turns on the lock of every list, dict and set they share |
a copy.deepcopy each |
with the threads, less the more elements the copies share: deepcopy returns an immutable element such as an int as it is, so every thread reading it out of its list writes that element's count |
| a value each, of its own objects | with the threads |
| a JSON document each, parsed from one shared string | with the threads: the parser takes a string cache from a pool behind one lock, and holds the lock only to take it and put it back |
So a server checking each request's own payload scales with its threads, and threads checking one shared structure do not. The installation page says what to do about the second.
The closest races¶
The table above holds four of the shapes the competitive gate judges;
scripts/perf_compare.json names every one with the ceiling the project claims
for it, and scripts/compare_gate.py prints each beside its ceiling. A record of
only the wide margins would be a selection, so here are the two closest, as the
fraction of pydantic's time each takes, on the PGO CPython 3.15 build above:
| Shape | ratio | spread across runs |
|---|---|---|
| JSON document, 200 records parsed and checked | 0.52 | 0.044 over eighteen runs |
| Error report, 50-field record with one wrong field | 0.46 | 0.024 over eighteen runs |
The JSON document is a single pass over bytes for both libraries, which is why
valgebra takes about half of pydantic's time here rather than a fraction:
neither is spending its time in the check. A document's free-form sections are
dict[str, V], and covering their keys is read two ways -- in place for a
narrow object, through a table of last values for a wide one
(docs/dev/walk.md).
Across the eighteen runs the JSON document spreads 0.044, the widest of the
seven; beside it the nesting 0.037, seventeen of its runs within 0.006 and one
reading 0.132, the error report 0.024, the record 0.020, the scalar, which sits
near timer resolution, 0.016, the array 0.008 and the build 0.0004. The
error report is the only shape timing a path that raises and formats a Python exception,
so a Python exception's cost is inside the number. It is excluded from the
gate's drift ratchet, and scripts/perf_compare.json gives the reason rather
than leaving an absent entry to mean it: a spread on another build wider than
any regression worth catching. The ratchet holds the JSON document and the other five shapes,
armed from a window of the bench lane's readings rather than one run, since the
lane reads one tree on several classes of machine further apart than a
tolerance (docs/dev/tooling-ci.md has the window); the error report is
judged against its ceiling alone.
A failing validation walks the value twice here -- once to decide, once to
say which field -- where pydantic-core walks it once and collects as it goes.
That is a deliberate trade for the passing path, which is the common one and
which walks once. The second walk does not repeat the first: it resumes where
the deciding walk stopped, since a field that matched has no violation to report
(docs/dev/walk.md). Resuming rather than restarting is worth
36.8% of that walk on the instruction gate, which
scripts/perf_gate.py --binding-explain measures. What the report pays beyond
the deciding walk is one walk of the fields after the failure, and the
exception.
The scalar shape is absent from the table because it sits near timer resolution, and most of it is the call rather than the check, as the floor above shows: on CPython 3.15 the competitive gate measures it at a 32.0 ns median over eighteen runs, between 31.4 and 33.8 ns, around 5.9x. That gate measures it; this record does not.
Core micro-benchmarks (criterion, release+LTO, indicative single run):
| Operation | Corpus | Median |
|---|---|---|
reindexed |
64-field pool-indexed record | ~1.2 us |
with_records_open |
record spine, depth 32 | ~4.2 us |
Honest limits¶
- The numbers are a single machine class. They establish relative behavior, not a universal ranking. Shared CI runners are too noisy for a tight wall-clock budget, so the merge gate measures a deterministic instruction count instead.
- The margins against pydantic come with the caveat that the two tools do different work: pydantic constructs output, valgebra only checks membership. The ratios answer "how fast is each tool's validation step," not "how much faster is membership than construction." The record's margin is the narrowest on every interpreter measured, and which of the other two is the widest moves with the interpreter: nesting leads on CPython 3.12, 3.14 and 3.15, and the array on the free-threaded builds.
- The ratios are not the whole argument for using valgebra. They are real,
and they are also a regression gate. But the reasons to reach for a membership
check over a parser are additionally semantic — an already-held object is
re-examined rather than passed through, a value stays checkable after it is
mutated, and schemas can be compared as sets — and every one of those would
hold even at a ratio of 1.0.
tests/test_pydantic_boundary.pypins them as verdicts, with no timer involved. - The comparison measures different operations (check vs check-and-construct vs pure-Python check). It answers "how fast is the validation step for each tool," not "are these tools interchangeable" — they are not. See the README for what valgebra is and is not for.
- These figures are for the object path — validating a value already in hand. The JSON input path is measured separately, on the same machine class, in the JSON page.
How the speed is made¶
Every technique below changes how an answer is reached and never the answer.
Where a cheap reading stands beside the general one, a test asks both and holds
them to one verdict, so a rule changed in one and not the other fails. Where
only the cost moves, no test can tell the readings apart, and a shape of the
instruction gate holds the cost instead: scripts/perf_gate.py --against <rev>
measures it against a base, and the flag a figure below names is the shape that
reproduces it. A percentage compares a technique with the reading it stands
beside or replaces; where that reading stays in the tree, it is the fallback,
and where a test keeps it, the test holds the two to one answer.
Each entry names the code that owns it. The developer pages in the repository
carry the arguments in full: the walk (docs/dev/walk.md), the frontend
(docs/dev/frontend.md), the relations (docs/dev/decision.md), the
error model (docs/dev/errors.md) and the schema representation
(docs/dev/schema-ir.md).
Crossing into Rust¶
- One crossing per call.
is_valid,validateandis_valid_jsonenter the extension once and walk the whole value in Rust. Python runs during a walk only where the schema or the value brings code of its own: a predicate, a class whose metaclass computesisinstance, a property read as a field, a value's own__eq__. What the crossing itself costs is the floor the cheapest door measures. - No reference pool. PyO3 keeps a pool of reference-count decrements
deferred while a thread is detached, and once any of its lazy initializers
has detached and attached again -- the first interned name does -- every call
into the extension locks that pool's mutex to ask it for them: forty
instructions and eight branches on
Validator(int).is_valid(1). The binding never detaches, so the pool is always empty, and.cargo/config.tomlcompiles it out;tests/test_build_flags.pyholds the flags there. The instruction gate's boundary shape calls the walk without crossing PyO3's call machinery, so the comparison gate'sscalaris the instrument that reads it: 32.1 ns with the pool gone against 33.9 with it, on CPython 3.15 and the machine named above. - A failure is raised in one step.
into_pyerrincrates/valgebra-py/src/errors.rsbuilds theValidationErrorinstance and hands it to PyO3 as the error's value, so the raise is onePyErr_SetObject. A lazily built error is normalized by releasing the interpreter lock and taking it back, which lets another thread run in the middle of a raise and costs a failingvalidateof a fifty-field record 5.4% more instructions. The exception being handled is chained as__context__either way, and a test holds that on every interpreter the suite runs. - Every fixed name is one the interpreter already holds. An attribute, key
or method name crosses as an interned string (
intern!), which carries its hash; a name handed over as Rust text is decoded into a new string and hashed on every lookup. An attribute that may be absent is asked for withgetattr_opt, which answers without building an exception wherever the interpreter offers a lookup that does (PyObject_GetOptionalAttr, from 3.13). - Internal maps hash with FxHash. A validator's own maps are keyed by its
field names and by object identities, so they use
rustc-hashrather than the standard hasher and its per-map random seed. The one table a document's own keys fill -- the last entry of each undeclared key in a wide parsed object -- hashes withahashinstead, keyed per process, because an unseeded hash lets a document carry keys computed to collide; it is also filled once a key, the position a repeat displaces read off the insert, which makes the wide object 9% to 14% cheaper than an FxHash table.
Building a validator¶
Compiling happens once per schema, so these are startup costs. They matter to a program that builds validators per request, and to no validation call.
- A type is dispatched before any
typingintrospection, and an exactbool,int,float,strorbytesconstant, or aValidator, is read before the frontend's dispatch (builtin_constantincrates/valgebra-py/src/build.rs). ALiteralof two thousand string codes compiles in about three million instructions; read through the whole dispatch one constant at a time, it costs four times that. The pool indexes its constants by type and value or by address (Pool::intern_keyed), so interning one is a lookup rather than a scan. - The
typingforms are resolved once per interpreter (formsinbuild.rs), anddataclassesis imported on the first dataclass a program compiles (DATACLASS_READINGinbuild/classes.rs). Held among the forms, its import would leave tracked objects behind that lengthen every later garbage-collection pass, and a build that compiles no dataclass would read 6.45% dearer on--binding-build. The table of loaded modules is held too (loaded_modulesinbuild.rs): aTypedDictfrom CPython 3.15, and onetyping_extensionsbuilds on any release, asks it once per class for the sentinel that says noextra_itemswere given, and importingsysto ask is 12% of compiling a three-field one. - An alias of a builtin container, and a union written with
|, are read off the object (builtin_aliasincrates/valgebra-py/src/build.rs).typing.get_originandtyping.get_argsread the same two attributes for them through three Python calls, after an attribute no such form carries is asked and answered by anAttributeErrorraised, formatted and cleared: compilingint | str | list[int] | dict[str, int] | Nonereads 68% fewer instructions on the profiled CPython 3.15 wheel, and a test holds the two readings to one answer. - A class is asked whether it is a dataclass by its attribute
(
is_dataclassinbuild/classes.rs).dataclasses.is_dataclassishasattr(cls, "__dataclass_fields__")once its argument is a class, on every interpreter the matrix runs, so the attribute is asked in its place: a class node pays no Python call for it, and a schema naming classes and no dataclass never imports the module (--binding-object,--binding-protocol). - A class's annotations are read as written wherever evaluating them would
hand each one back unchanged (
annotations_as_writteninbuild/classes.rs).typing.get_type_hintsexists to evaluate forward references, and on a class with none it still copies every base's namespace and walks every annotation in Python. The reading takes the call's steps and declines every case the call would rebuild -- a string, aForwardRef, a nesting deeper thanMAX_ANNOTATION_DEPTH-- so an answer or an error a caller sees is the call's. A Rust interpreter test compares the two over a corpus of classes on every supported interpreter. A static builtin base --object,tupleunder aNamedTuple,dictunder aTypedDict-- holds no annotations and is not asked for them (annotates_nothing): from 3.14 the asking is a Python call that raises and drops an error for each, and skipping it makes aNamedTuplecompile in 45% fewer instructions on CPython 3.14 and 3.15, a five-fieldTypedDictin 30% fewer, and a dataclass in 16% fewer. A class of the program's own is read throughtype's own__annotations__descriptor, whichannotationlib.get_annotationsasks first and whosedictit copies (own_annotations); the call, two Python functions, answers only what the descriptor leaves to it. From 3.14 that makes a three-fieldTypedDictcompile in 19% fewer instructions, aNamedTuplein 14% and a dataclass in 7% (--binding-protocol,--binding-object). - A dataclass's fields are read as
dataclasses.fieldsreads them (fields_as_declared), and an order bound of exactlyint,floatorboolis placed without asking thenumbers.NumberABC, whose__instancecheck__is a Python function (is_a_numberinbuild/refine.rs). A subclass is still asked, because the ABC reads a value's__class__, which a subclass may answer with code of its own. - A refinement marker is read by its type, once. A marker carries one or
two of ten optional attributes and not the rest, and below 3.13 asking for one
it does not have is answered by raising. So the frontend asks the marker's
type which names it carries and remembers the answer, up to
MAX_MARKER_TYPEStypes, which costs no exceptions on any interpreter: fiftyAnnotated[int, Ge(0)]fields compile in about 13 us on CPython 3.15, 13 on 3.14 and 13 on 3.12, on the machine named above. - A
TypedDictfield's qualifiers are read once. The reader that says whether a key is required walks theRequired,NotRequiredandReadOnlyaround a field's hint, below the first that answers so that two which disagree are refused, and hands back the type below them, which the field is built from (qualified_requiredinbuild/classes.rs). The builder reaches a qualifier only after asking the hint every form before it, so it is spared that walk: fifty refined fields, five of themNotRequired, compile in 12% fewer instructions than built from the hint as written (--binding-annotated). - A protocol reads its classes' namespaces once, and a member annotated
with a class
typemade is neitherClassVarnorFinalwithout callingtyping.get_origin, a Python function (protocol_membersinbuild/classes.rs;--binding-protocol). - A class's missing attribute raises nothing on CPython 3.12. A build asks
a class for attributes it nearly always lacks -- the
__origin__of aTypedDictfield's hint, the__unpacked__of a tuple's type arguments,_is_protocol-- and below 3.13 PyO3 answers a miss with anAttributeErrorbuilt, its message formatted for a class, and cleared. On 3.12 a class is asked through the builtingetattrwith a default instead (optional_attributeinbuild/classes.rs), whose lookup answers without the error: a fifty-fieldTypedDictofintcompiles in 59% fewer instructions there, and the binding gate's protocol build in 8.61% fewer (--binding-protocol). From 3.13 PyO3 asks without the error, and 3.11 builds it on either road. - A validator is accepted on one walk of each tree.
Schema::measureincrates/valgebra-core/src/ir.rsreads the nesting depth, the node count and the self-reference marker in one level-by-level walk against two buffers, and descends no native stack. - What the walk needs is built once per validator.
ValidatorIndexincrates/valgebra-py/src/check/index.rsholds each record's name-to-position plan, its keys and attribute names interned, each literal union's tables and each compiled pattern. It is keyed by the address of the schema's own buffers, which are stable for the life of the immutable schema, so a copied validator never inherits another's index; and a node the build traversal does not reach falls back to reading its own text, so no verdict depends on the index being complete. A report does -- a record with no plan is read by its entries, and which branch of a union of records is explained follows that reading -- so the traversal reaches every node a record can sit under. A dataclass instance's attributes are read through the interned names, which--binding-object-walkcounts.
Walking a Python value¶
- The leaf decision is a test where the answer is.
admitincrates/valgebra-py/src/check/walk/scalar.rsis inlined, and recording a violation is a#[cold]function of its own, so the accepting path is one comparison and the allocating path sits apart from it. - A scalar is its type test. A union's scalar branch, a mapping clause's key
and value, a record's scalar field and the elements of a sequence of a union
of scalars ask the type test directly (
scalar_member, andfield_holdsincheck/walk/record.rs), reading the depth level and the fatal-signal flag as the general walk does, so each refuses exactly where the walk refuses. A thousand-elementlist[int | None]costs 87% fewer instructions that way. A mapping whose one clause is two scalars reads the level and the flag once for the dict, and its entries as two type tests each (scalar_map_matches): a hundred-entrydict[str, int]costs a third fewer instructions that way on the profiled wheels, and the binding gate's sixty-four entries 43% fewer (--binding-mapping). A mapping of several such clauses asks each clause's two tests in turn (scalar_clauses_match): a hundred-entry{str: int, int: bool}costs 41% to 42% fewer instructions than the scan on the profiled 3.12 and 3.14 wheels. A Rust test holds the direct reading to the walk's verdict for every scalar schema against every kind of value. - A literal's check stays out of the walk's every node. A relation walks a
literal's constant, so a profile reads
check_literalas hot, and inlined intomemberits explaining half puts 300 instructions and 48 bytes of frame in every node of every walk: a list nested twenty-five deep reads 1.2% more instructions and 4.2% more cycles. It is out of line, so the profile a relation trains cannot bring it back. - An instance of the class itself is read off its type pointer.
type(obj) is Canswersisinstance(obj, C)beforeC.__instancecheck__is asked, which is the test CPython'sPyObject_IsInstancemakes first (is_exactly_aincheck/walk.rs). PyPy implementsisinstanceotherwise, and there the call answers. - A literal's own constant is answered by its address. A literal written in
a program's source is usually the very object it validates: an interned
string, a cached small integer. An exact
str,int,bool,bytesorNonethat is the pooled constant equals it, sois_the_constantanswers without a comparison. Afloatis excluded, since a NaN is not equal to itself. Either way the value is compared where the walk holds it, writing no reference count, and only a parsed JSON value is built into an object (check_literal): a check against oneLiteralcosts 2.4% fewer instructions on CPython 3.14 and 4.1% fewer on 3.14t, where PyO3 reaches a reference count through a call into the interpreter. - A validator of one scalar kind is its type test. A schema that is one kind at
the root is answered before the walk's state and context are built
(
root_scalarincheck/walk/scalar.rs): a walk that has not begun holds no signal and every level, so nothing around the test can refuse.is_validon anintreads 22% fewer instructions on the profiled CPython 3.15 wheel, the interpreter's own call included,validateof a value that belongs takes the same road, and every other schema pays a load and a comparison. - A refinement borrows its operand and renders nothing on a pass. The bound
is read out of the pool as a borrow for the whole check, and the value from
the walk, so a passing check writes no reference count; a scalar base that
admits the value is its type test rather than a walk (
check_refine;--binding-refined). A violation's message, which needs the bound'srepr, is built only at the site that records one (Expectedincheck/walk/scalar.rs):tests/test_refinements.pycounts the bound'sreprcalls, and a hundred passing checks make none. A constraint is decided without its code and expectation, whichexplainedbuilds only for a failure a walk that explains records, and the walk's context is read through the frame rather than copied onto the stack once per constraint: a thousand integers under two bounds cost 11% fewer instructions, a thousand strings under two length bounds 17%. - An order bound over a float compares two floats. An exact
floatunder an operand a float holds exactly, an exactfloator an exactintup to2^53, is the two read where they lie, and no Python runs (floats_comparedincheck/walk/scalar.rs, with the operands read once a validator byfloat_operandincheck/index.rs). A thousand floats under one bound read 69% fewer instructions on the profiled CPython 3.15 wheel,--binding-refined-float52% on the build without a profile, and a record of two such fields beside a refinedint18%. Any other value pays a load and a comparison, and a test holds the reading to the comparison over every bound, a value of every kind and an operand of every kind. A float the two floats admit is admitted where the constraint is read, with no call into the bound's own reading, which a refusal and every other value still take: a record of three refined fields reads 0.7% to 1% fewer instructions on the profiled wheels than the 0.0.30 wheels, asscripts/perf_inspect.py countreads itsrefined_fieldsshape. - An error the walk answers with is dropped out of line. Dropping one asks
whether the thread is attached to the interpreter, a thread-local the
extension reaches through a call into the dynamic linker, and inlined into a
loop, the compiler takes that address ahead of the loop, on the path where
nothing raises. So the errors a constraint and an attribute record answer
with are dropped in functions of their own (
set_asideandpredicate_raisedincheck/walk/scalar.rs,attribute_missingincheck/walk/record.rs): in the extension, a thousand integers under two bounds cost 2.8% fewer instructions on 3.12, 3.14 and 3.15, strings under two length bounds 3.6%, and a list of dataclasses 1.2%. The instruction gate's workload is an executable, which reads a thread-local without the call, so it sees none of this; a probe of the built extension does. - A fast walk asks in the frame it holds. A walk carries two buffers, for
where it is and what it has found, and a fast walk writes neither; each is
still a value with a destructor. A union's branch is walked in the union's
own frame (
check_unionincheck/walk.rs), and a mapping's scan keeps one fast frame for every entry its clauses are asked of (coveredincheck/walk/record.rs), rather than a pair built and dropped per branch and per key: a recursive schema's descent costs 10% fewer instructions that way (--binding-memo). A record's deciding pass reads over its caller's buffers in a fast frame, and keeps its progress only where the walk explains a record it refuses (check_keyed_map): a list of two hundred records costs 4.8% fewer instructions that way, and a record the walk reaches alone 2.9%. The scan reads the frame's context where it lies, asking each half of a clause as a record asks a field (field_holds), rather than copying the context once an entry. Where a fast walk has no frame to hold --is_validandis_valid_jsonthemselves, a mapping's scan, a parsed object -- its pair is anUnwrittenone, never dropped because it never owns anything:is_validon a scalar costs 6% fewer instructions that way on CPython 3.15,in9%, and the instruction gate's call boundary 13% (--binding-boundary). - The context a level hands on is seven words. It holds the pool, the
definitions, and one reference each to the validator's index and the call's
state (
Ctxincheck/ctx.rs), and a table or a cell lies at a fixed offset of what is referred to, so each is read with the loads a reference of its own would take. A walk that descends copies less a level: on the CPython 3.12 PGO wheel a list nested twenty-five deep costs 12.9% fewer instructions, a scalar 5.4%, a literal 4.8% and a JSON document 1.5%, and the deepest walks fit a thread of 844 KiB. On the instruction gate the call boundary reads 1.9% fewer and a recursive schema 1.2%. - The recursion trail is a stack, reserved on first use. Entering a
reference pushes a
(value, definition)pair and leaving pops it (Trailincheck/ctx.rs), so nothing is hashed. The first level reservesFIRST_TRAILpairs in one small request, and a walk that enters no reference allocates nothing (--binding-recursive). - A leaf is decided by a value's type and number. A union, meet or
complement of scalar kinds and of refinements of
intandfloatby bounds and steps -- JSON Schema'sintegerandnumber-- is specialized to each exact builtin type once a validator (LeafPlanincheck/index.rs): a constant, or the test of the number left when each kind folds away. A list of one reads each element's type and, where the type leaves it open, its number (leaf_list_readincheck/walk/sequence.rs), and walks any other element in the walk's own mode. The list arm asks it behind a test of the index's one word, so a validator holding no reading keeps the other readers' tail calls, and a union of scalar kinds, which has a reader of its own, holds no plan. A parsed array of one is read by each item's tag and number (leaf_array_read). On the profiled CPython 3.15 wheel a list of a thousand integers againstintegercosts 92% fewer instructions, a thousand floats againstnumber83%, and a document of a thousand integers 67%, the parse included; a test holds the reader to the general loop's verdict, stop, moves, fatal signal and report in every mode. A leaf overfloats that is order bounds alone is held as the interval they admit (Intervalincheck/index.rs), two comparisons where the bounds were each a test through the leaf's tree: a thousand floats againstnumbercount 44% fewer instructions again, and a list or a mapping of a tree answers an exactfloatby the interval at once. A mapping ofstrkeys to a leaf is read the same way (leaf_dict_read): valgebra-jsonschema'sstr -> integermap of a thousand entries checks in 12.5 us where the scan took 31. A leaf may hold a reference to a tree of scalars (Leaf::Tree), the JSON values a translation meets a keyword's guard with: a scalar is the tree's leaves' answer, a list or a dict the tree reads on is walked, and the reader takes it where the trail is below its bound and no memo is armed (leaf_reads_hereincheck/walk/scalar.rs). On the profiled CPython 3.15 wheel, a thousand integers againstif: integer, then: minimum 0check in 7.3 us where the walk took 60.7 (1,410,105 to 165,119 instructions a call), againstminimum: 0with notypein 7.4 where it took 42.8, and againstnot: stringin 2.35 where it took 19.9 (list_guarded89% fewer instructions). - A tree of scalars is read by each value's type. A definition whose union
holds only leaves refusing every list and dict, a list of the definition and a
mapping of
strkeys to it -- the valuesjson.loadsreturns,int | list[t]-- is asked each value's exact type once (tree_memberincheck/walk/tree.rs, overTreePlanincheck/index.rs), where the walk spends a reference, its pair on the trail, the union's level and a branch per member around the same question. On the profiled CPython 3.15 wheel the JSON values over two hundred records of every kind of JSON value cost 73% fewer instructions andint | list[t]over a tree twelve deep 61%, and--binding-recursive46% on the build without a profile. A value of no exact builtin type, one near either bound and one inside itself are handed to the walk where it would have reached them, a walk that explains or settles a complement reads the definition generally, and one with a memo armed reads its scalars alone, noting the reach their walks would note; a test holds the reading to the walk's verdict, stop, moves and fatal signal. The index holds the plans of leaves and trees behind one word, and the reference's arm asks whether it holds any and whether the value is a Python one, so a validator holding none pays a load and a comparison a reference. - A list or a mapping of a tree is read by each value's type.
list[J]anddict[str, J], which a JSON Schema translation meets every array and object with, read each value as the tree's reading does at the value's reference, the bounds asked once for the container (tree_list_readandtree_dict_readincheck/walk/tree.rs). On the profiled CPython 3.15 wheellist[J]over a thousand integers takes 2.5 us where the reference's arm took 6.8, anddict[str, J]over a thousand entries 6.2 us where it took 13.2: each reader keeps the type of the last value its leaf admits whatever the number, so a value of that type is one comparison of its type pointer. The list arm asks it behind the index's word, since a list of a reference in a validator holding no reading paid the reader's call: the memo's shape read 3.5% dearer without it, and reads 1.6% dearer with it on the build without a profile, a list of records 0.9% and a list nested twenty-five deep 0.5%; on the profiled wheel every shape counts within 0.5%. - A kept walk reads a tree's scalars by type. A JSON Schema translation
meets each array with
list[J]and a list of its items, each record reachingJtoo, so the meet's second member is walked with the memo armed, and there every reading of the tree declined: each value cost a hash insert and the general walk. The readings read a scalar there by its exact type, noting the reach its reference's walk would note, and leave a container to the walk, which keeps it for the deeper meets that reachJat it (kept_scalar_read,kept_list_readandkept_dict_read). On the profiled CPython 3.15 wheel valgebra-jsonschema's array of two hundred records counts 25% fewer instructions a call and checks in 52 us where it took 67, itsif/thenarray 29% fewer and 64 us where it took 96, and its taggedoneOf21% fewer; a chain of sixty objects against a recursive$refchecks in 22 us where it took 25. The kept list and mapping readers take five and four arguments, asked once the mode and the prefix are read: asked with the prefix where the reading declines, they cost the plainlist[J]reader an instruction an element.
Sequences and sets¶
- A sequence of one scalar kind is a loop per kind.
scalar_list_matchesincrates/valgebra-py/src/check/walk/sequence.rsreads the element kind once and runs a loop whose type test is a constant, compiled out of line from the rest of the sequence walk so that an edit elsewhere does not move its register allocation (--binding). A set, a frozenset and a parsed JSON array take the same reading, a set with a scan per kind in both the deciding and the explaining walk (--binding-set). A list of one scalar kind orNonehas a loop per kind too, both tests constants inside it (nullable_list_matches;--binding-nullable): a thousand-elementlist[str | None]costs 80% fewer instructions than through the union's branches on CPython 3.14 and 3.15, underis_validandvalidatealike, and 72% on 3.12. - A list is read through a snapshot where that pays. Reading a list element
hands out an owned reference, a count written on the element twice; a tuple
copy pays those writes in two loops inside the interpreter and none in the
walk.
snapshot_paysdecides it per interpreter and per length, as the margins above describe. The instruction count of a snapshot reading is higher than in place -- the copy is instructions, and the stall it removes is not -- which is why the wall clock decides this one. Before 3.12 allocating the copy may run a collection, and a finalizer it runs may empty the list under the copy, so on 3.11 the copy holds the collector off for its own length: three calls, about three hundred instructions a list, 1.4% to 1.6% of a thousand integers on the profiled wheels. - A tuple's elements are borrowed, because a tuple cannot change while the
caller holds it; a list cannot be read that way. On CPython,
PyTuple_GET_SIZEandPyTuple_GET_ITEMread a tuple's storage whatever its type overrides, so every tuple, subclass or not, is read where it lies and its type is asked nothing: aNamedTuplevalidates at what a tuple does (--binding-subclass). PyPy'scpyextanswers those accessors through a subclass's own__len__and__iter__, so there the walk asks the type whether it inherits both and copies any subclass that does not. A length bound asks the type whether its__len__is the tuple's own on every interpreter, since it counts what the value holds rather than what an override answers. - A tuple of scalar positions is one type test a position
(
scalar_positions_tuple_matches), and a list whose element is a union of literals, one class, a tuple of scalars or a named tuple of them is read through that union's table, the type-pointer test, a test of each position, or the class's pointer test and a test of each position, with the positions read once for the list (element_list_matches). An element the test does not settle is walked, which may run Python, so the list is read in place unless a snapshot settles it entirely. A thousanddatetime.datevalues cost 76% fewer instructions than through the general loop, a thousandtuple[int, str]61% on 3.12 and 3.14, 68% in the 3.14 PGO wheel, and a thousand two-fieldNamedTuples 76% on 3.12, 3.14 and 3.15, 81% undervalidate. A list of refinements calls each element's own check, with the level the walk opens for an element held across the list (refined_list_matches): a thousand integers under two bounds cost 12% fewer instructions, and a thousand strings under two length bounds 19%. Deciding over a base ofint,floatorstr, its kind is read once and has a loop of its own (refined_scalar_list): strings under a pattern read 7.4% fewer instructions on the profiled CPython 3.15 wheel. One pattern, alone or beside length bounds, is read from the index once for the list, and an exactstris its text matched against it (pattern_list): two hundred strings under one pattern read 15% fewer instructions on the profiled CPython 3.12 wheel than with the pattern found at each element, asscripts/perf_inspect.py countreads itspatternshape. Order bounds on anintwhose operands a machine word holds are one range read once for the list, and an exactintthat fits a word is compared with it where it lies (machine_range_list): a thousand integers under one bound read 74% fewer instructions there, and 77% on the build without a profile (--binding-refined). A value past the word is read by the comparisons, once its read has raised, and that ends the range for the list. Order bounds on afloatwhose operands a float holds are one interval read once for the list, each end open or closed, and an exactfloatis compared with it where it lies (float_range_list): a thousand floats under one bound read 43% to 49% fewer instructions on the profiled wheels of CPython 3.11 to 3.15 than the 0.0.30 wheels, which read each float through the constraints' loop, asscripts/perf_inspect.py countreads itsrefined_floatshape. Length bounds on astrare one range of code points read once for the list, and an exactstr's length is compared with it (length_range_list): a thousand strings under a minimum and a maximum length read 77% fewer instructions on the profiled CPython 3.15 wheel. Any other element is its own check. A meet ofstrand its length refinements is the same one range (text_meet_list), whatever else the validator holds: valgebra-jsonschema's array of a thousand strings underminLengthandmaxLength,list[str & MaxLen & MinLen], checks in 3.7 us where the walk through the meet took 24. A list of records reads each element by its record's check, inlined into the reader, with that level held the same way (record_list_matches): two hundred three-fieldTypedDicts cost 7.5% fewer instructions on 3.12 and 3.14. What its node says of every element, its plan and what its clauses admit, is read once for the list (RecordReading), which is 6.6% of that shape's instructions on the build without a profile (--binding-record-list). A list of dataclasses walks an element of exactly the class as the record of its fields, without dispatching the meet, the class and the record around it (class_record_list_matches): 12% fewer instructions for a hundred five-field dataclasses, in both modes. - A scan holds its container's lock once on a free-threaded build.
PyO3's list and dict iterators take the container's critical section around every step, each a re-entry of the section the scan already holds; theirfind_maptakes it once, and the free-threaded build's list and dict scans step inside it. In the 3.14t PGO wheel,is_validon a hundred-entrydict[str, int]takes 18% less time, and on a thousand-elementlist[tuple[int, str]]19%. Under a global lock both scans keep their hand-stepped loops: throughfind_map, alist[int]read in place on 3.14 costs 59% more instructions, and the record shapes the dict scan is inlined beside 1.4% more.
Records¶
- A closed record probes the keys it declares. Membership is settled by
asking the dict for each declared key and counting: the value belongs exactly
when each key it holds matches its field and it holds nothing else, which the
dict's own length says, since a dict cannot repeat a key. A declared key is
interned with the validator and carries its hash, so each probe is the probe
alone, and a key is resolved the way Python resolves one -- a
strsubclass, or a key of another type, is the field exactly where indexing the dict finds it, in this reading and in the scan beside it. - A field is read beside its key. The record's declared fields and its plan's
interned keys are one length, so the loop steps both, with no bound test and no
Optiona field (keyed_map_asks_for_its_keys): a fifty-field record reads 8% fewer instructions on the profiled CPython 3.15 wheel, and 2.2% on the build without a profile (--binding-record). - The probe is the floor for this shape. Profiled under callgrind on CPython 3.12, fifty probes of a fifty-field record are about 143 instructions each and 64% of the accepting call. Iterating the dict once and resolving each key by name instead measures 47.9% dearer: an iteration step increments two reference counts, casts and decodes the key and hashes it, where a probe on an interned key carries its hash already.
- Interned keys meet on a pointer, and Python interns most of them. A dict
probe compares the key it is given with the key it holds by pointer before it
compares hashes or bytes. A dict written as a literal, one built from
**kwargsand an object's__dict__all carry interned keys, so a record walk over them settles each field in one comparison: on a fifty-field record, interned keys read about 28% cheaper than keys that are not (--binding-keysagainst--binding-record). Keys parsed from JSON or built with f-strings are not interned and take the hash-and-compare path, which is what the figures on this page are measured over;sys.internon the keys of a dict you validate in a loop is worth trying if that loop is your bottleneck. - An open record is read by its keys too. A
TypedDictadmits keys it does not declare through the clausestr: anything, and what that clause says about an undeclared key -- it is admitted exactly when it is astr-- can be said without reading its value (Undeclared::ofincrates/valgebra-py/src/check/walk/record.rs;--binding-open). A record whose clause reads a key together with its value -- thestr: Ja JSON Schema translation writes foradditionalPropertiesabsent -- is read by its keys as well, and where the value holds a key it does not declare, those keys alone are scanned against the clause (undeclared_held): a list of two hundred records{id: integer, name: str, tags: list[str], str: J}, as valgebra-jsonschema writes an array of objects, reads 21.5% fewer instructions on the profiled CPython 3.15 wheel (16.97 to 10.68 us), a list of two such records' taggedoneOf26.9%, and a$reftree of 511 such nodes 17.1%. - A dataclass's attribute names are interned once per validator, so
getattris handed an object the interpreter already holds. The object is read borrowed, as the walk holds it, so reading a record writes no reference count on it, and a scalar attribute is its type test asked in the loop over the attributes, the level it needs read once for the record (check_attr_recordincheck/walk/record.rs): a list of dataclasses costs 2.7% fewer instructions that way on CPython 3.12 and 3.15.
Unions and literals¶
- A union of literals is a table. A union whose members are all literals --
a
Literal["a", "b", ...], a discriminator -- is compiled once into value-keyed sets, one for its integer literals and one for its string literals (UnionPlanincrates/valgebra-py/src/check/index.rs). An exactintorstrvalue is one set lookup, so the cost stops growing with the number of literals. The integer set is consulted only for an exactint(never abool) and the string set only for an exactstr; any other value -- abool, afloat,None, a subclass instance, a JSON value -- falls back to the scan, which remains the single source of truth, and tests hold the two to one verdict over the cross-type cases. The table also keeps its constants' addresses, so a value that is the constant itself is answered without reading its text: a thousand-element list of four string literals costs 37% fewer instructions than reading each value's text. - The closest branch is searched within a bound. A failing union reports
the branch whose first failure lies deepest, and the search re-walks at most
CLOSEST_BRANCH_PROBE_LIMITbranches, so a very wide union bounds the cost of its own report.
JSON¶
is_valid_jsonwalks the parsed document in place. The walk reads a value that is either a Python object or a parsed JSON value, so a document is checked without building Python objects for it, except the value at a node that compares it with one: a literal, an instance or object check, a refinement's comparison or predicate. The two sources run one walk and stay equivalent by construction, which a property test holds over random schemas and documents (the JSON page has the figures).- A closed record reads a document's keys through its plan, once. Each key
of the parsed object is resolved through the record's name-to-position plan
and every declared field's value gathered before any is checked, because a
repeated key means the document's last entry, as
json.loadshas it. The gathered table sits on the stack for a record of up toFOUND_ON_STACKfields. - A parsed object's keys are strings. A clause keyed by
strtherefore admits every key of a parsed object, and covering a free-form section is a check of its values alone. An object of up toSMALL_OBJECTentries is covered where it lies -- an entry is the one the document means exactly when no later entry repeats its key -- and a wider one through a table of last values. - A length bound counts a parsed container where it lies. An array's
length is its item count, and an object's is its distinct keys, counted by
the same two readings (
distinct_keysincheck/walk/record.rs), since the dictjson.loadsbuilds holds a repeated key once. The bounds are asked before the elements, and a document runs no code that could change it, so a refinement whose every constraint is a length bound builds nothing to ask them again (parsed_for_constraintsincheck/walk/scalar.rs). Two hundred records underMaxLen(8)cost 65% fewer instructions than building each one into a dict, once to count it and once to ask the bound again. The building is out of line, and a list of refined Python integers costs 1% fewer for it.
Reporting a failure¶
validateis one explaining walk, and the cheap readings serve it. An explaining walk records nothing of what it admits, so an element that passes its type test is answered by the test (admitted_quietlyincheck/walk/scalar.rs), and a sequence of scalars is read as its tests, an element that fails being walked at its own position (list_explained,tuple_explained).validateon a thousand-elementlist[int]that belongs executes about a fifth of the instructions the general element loop takes, and within a twentieth of whatis_validtakes: each scalar kind has a loop of its own in the explaining reader as in the deciding one, which on CPython 3.14 and 3.15, where the list is read in place, is a fifth ofvalidate's instructions on a list of integers and 15% of its time. A tuple of one scalar kind takes a test per kind the same way (homogeneous_tuple_explained), andvalidateon a thousand-elementtuple[int, ...]costs 46% fewer instructions than through the reading of each position's schema.- A refused list is read once up to where it fails. Where the explaining
walk reads a list through a snapshot, the snapshot's test stops at the first
element it refuses, and the walk reads the list in place from that element
on (
list_explainedincheck/walk/sequence.rs): nothing has run before it, so the elements the test passed are not read again. On CPython 3.12,validateon ten thousand integers refused at the last element costs 41% fewer instructions than read in place from the start, refused at the middle 22% fewer,fail_fastthere 28% fewer, and a list of one class refused at the last 42% fewer. A list that belongs costs what it cost, and one refused at its first elements within a few percent of it: alist[int | None]refused at its second element, 2.9% more. CPython 3.14 and 3.15 read the list in place from the start, where no snapshot pays. Reading only the elements the snapshot refused is not taken, since the walk of one may change the others (docs/dev/walk.mdin the repository). - A union explains no branch for a value it admits. Only a refused
value's report reads the branches that did not match, so a branch that fails
at the union's own location -- a scalar, a literal, a container of another
kind, a class -- is decided by its test (
decided_quietlyincheck/walk.rs), and a record branch by its deciding pass, its explaining pass waiting until no branch has admitted the value (explain_union).validateon a hundred values of a tagged union of threeTypedDicts costs a fifth of the instructions it costs with each refused branch explained in turn, close to whatis_validcosts, and a union of dataclasses about as much, on CPython 3.12 and 3.14 alike: the report a refused branch builds summarizes the value, and a dataclass's summary is its__repr__, Python code. Tests count the reprs a member's report would run, and hold a refused union's report to the one its chosen branch gives alone. - The explaining walk resumes where the deciding walk stopped, as
the closest races describe: a field the first walk
passed has no violation to report (
--binding-explain). - A report builds each key and string once. The keys of the error model are interned, a violation's message and path are built once and shared between the error item and the attribute that mirrors it, and a path segment shares the field name the schema already holds.
- A value summary is bounded while it is built. Every violation summarizes
the value it is about, and a container is rendered through
reprlibunder a depth and width bound (BOUNDED_REPRincrates/valgebra-py/src/errors.rs) rather than rendered whole and cut, so reporting on a value nested twenty thousand deep takes about 23 us. A value small enough to print renders exactly asreprwould.
What a relation between two validators costs¶
is_subtype_of, relation_to and is_equivalent are not the membership walk,
and they have a cost of their own with two clear levels. A pair a rule
decides costs about a microsecond: the rules recurse over the two schemas and
answer from their shapes. A pair the rules decline goes to the set
representation, which lowers both sides into automata and takes tens to
hundreds of microseconds. Inside the core the two answers to one question are
about three orders of magnitude apart -- scripts/perf_gate.py
--decision-rules and --decision-sets ask one corpus of each, and
scripts/perf_budget.json holds both counts -- and a call through
relation_to adds its own fixed cost to either. That gap is the reason a rule
that stops declining is worth writing.
Which pairs land on which side is the interesting part, and it moves. Most do not reach the sets: a mismatch of kinds, a record missing a required key, a sequence of the wrong arity, a subject outside a refinement's base. What still reaches them is a schema whose atom only the bindings can read on one side and a structure on the other -- a class deriving from no builtin, a complement as the subject -- and a bound that has to be compared rather than matched, such as a list against a length-bounded list of the same element.
- The cheap questions are asked first. A node is a subtype of itself, which
a pointer comparison answers before anything else; a universe on the right is
read off the region set the query already holds; a schema below a union is
first looked for among the union's branches, which keeps widening a literal
union linear in its members; and the readings that refute a pair no rule
places are asked in the order they cost, the walk of both subtrees last
(
crates/valgebra-core/src/decision.rs). A kind refutation reads the subject's whole kinds first, since most subjects hold none and the supertype then goes unwalked (spills_a_kind_into). - The set representation is asked only where the rules are silent. Both deciders answer in three values, so the descriptor is asked where the rules answer unknown, never where they proved the answer.
- Records compare by a cursor. Two name-sorted field lists are walked
together, each lookup one ordering comparison, with no table built per call
(
FieldCursorindecision/records.rs); a record whose fields repeat one schema asks that goal once. - Every query spends a work budget (
spendindecision.rs), and returns the conservative answer once it is spent, so a deeply nested Boolean combination stops rather than running unbounded. - A query carries its state behind one pointer. The oracle, the
definitions, the budget, the trail and the
visitinglist are one context per query (Cxindecision.rs), handed down by reference: a rule's call passes one pointer, and an emptiness question a goal asks reads the query'svisitinglist rather than starting one. The meets being unfolded stay off it, on a list only an unfolded meet reaches, since a field of the context is built and dropped by every query. - The set representation shares what it does not edit. An automaton holds
its edge guards and its word automata by handle, a complement of a minimal
automaton keeps it minimal, and a question about which kinds a descriptor
reaches is read off its components (
Descr::within,Descr::reachesindescr/mod.rs) rather than answered by building a meet and testing it. - A word automaton is minimized by halves. The states no word tells apart
merge once blocks of states are cut by the transitions read backwards, and
only the smaller part of each cut is queued to cut by next
(
equivalent_statesindescr/regular.rs, Hopcroft's algorithm):k n log nwork overnstates andkbyte classes. Signing every state afresh each round reaches the same partition and spends a round per state on a chain, which is what a literal is: a literal of four thousand bytes takes about 1.5 ms to build on a release build, where signing every state each round takes 603 ms, and a product that keeps its chain is minimized again at the same cost.--decision-setsreads 6.12% fewer instructions, the signatures the rounds allocated. - A pattern is built once a thread. A lowering reads a schema's patterns
and length bounds again for every difference a relation asks, so a thread
keeps each word automaton it built under the text it was built from, a
refusal among them, up to sixteen mebibytes (
RegularSet::compileindescr/regular.rs). Two JSON Schema documents translated into validators, holding three patterns and three length bounds between them, ask for their automata 322 times and build six, which hold 85,678 bytes: the relation declines in 0.17 s on a release build, against 4.2 s when every request builds. A pair whose one pattern the build refuses decides in 0.46 s against 2.14 s, since a refusal spends the most of any build. The instruction gate's decision workloads read no more than before. - A pattern the oracle walks is compiled once a query. A literal is
asked of a pattern by walking its constant through a schema no validator
built an index for, so a table of strings compiled the pattern once a
constant: 1,024 strings against the alternation of the same words took 1.4 s
on a release build. The query keeps each pattern it compiled under its text
and drops them when it is answered (
matches_unindexedincheck/index.rs), and the pair takes 2 ms. An oracle that walked nothing, which is most relations, drops with no thread-local read (walkedinoracle.rs), and a relation reads its answer off the oracle by reference, its size being past what a move does inline. - A value the last asking reads of a schema is read once a query. The
asking that reads a pair's leaves lowers each refinement, meet and complement
the rules leave open, alone, for a value, and a query asks the same ones
again: one question over drawn JSON shapes asked the sets of a meet of nine
nodes 3,609 times, and read for 94 ms on a release build against 6.2 ms
without the reading. The query holds each answer under its number
(
Cx::reads_a_valueindecision.rs) and charges the steps each time it is asked, so the memo moves no answer, and the question reads 8.7 ms. The instruction gate's decision workloads read within 0.4% of the last release's. - A length bound costs a state an element and little else. A bound over a
list is an automaton of one letter repeated, so the letter is held by one
handle and asked whether it takes every value once (
chainindescr/symbolic.rs); a chain whose letters all hold a value is minimal as built and skips the merge; a row with no guarded edge leaves everything, so meeting its else edge is the guard itself (Rest); a product keeps the rests and the meets it has asked for the pairs to come, since a chain repeats its rows (Rests,Met); and a side that is the whole language or the empty one is answered without a product. A bound of forty elements overlist[int]lowers on 135 units of the allowance, so two bounds are compared up to the state bound, 2,046 elements, with allowance to spare;MinLen(4000)refuses in 4 ms against 1.2 s.--decision-lengthsreads 88.31% fewer instructions and--decision-sets21.80% fewer. - A sequence automaton's merge starts from three distances, and signs only
the states that share a block. Before its first round, each state is put
in a block by its distance to an accepting state, its distance to a
rejecting one, and the longest walk it can take while acceptance stays in
reach (
seedindescr/symbolic.rs). States the merge joins agree on all three, so the start changes no table; what it changes is the rounds. From one block, the rounds part a chain a state per round, and a chain is what a length bound is; from the distances its states are apart before the first round, a lower bound's by how far each is from accepting and an upper bound's by how far each can go. A block of one state cannot split, so a round signs only the members of larger blocks (refineindescr/automaton.rs): two tuples of a thousand positions that differ only at their last take a round per position, which the distances do not save, and their union reads 1.06 billion instructions against 2.11 billion when every round signs every state.MinLen(1000)overlist[int]lowers in 1.8 ms against 87 ms on a release build,MaxLen(1000)in 1.6 ms against 104 ms, andMinLen(500) ⊆ MinLen(1000)decides in 5.9 ms against 146 ms.--decision-lengthsreads 19.85% fewer instructions and--decision-sets0.88% fewer. - A union's members are joined in rounds. A lowering reads every member of a
union and joins them two at a time, each round halving the list
(
joined_in_roundsincrates/valgebra-core/src/lib.rs). A join admits what it holds against what it has kept, so joining the whole so far with each member in turn reads the cube of the width, and a member past the bound on a line's atoms retries the full line. A relation between two unions of 2,048 tagged records, which the node bound refuses, declines in 37 ms on a release build, where the fold takes 5.9 s, and--decision-unionsholds it: the fold reads 1,825 times the instructions. Up to three members, those are a fold's joins, and they hold no list, since a list of descriptors is a request past the allocator's bins whose consolidating costs more than the union it would hold. - A complement is built only where it is read. Most sets hold a kind or two, and
a complement builds a top for every kind a set misses. Whether a letter takes
every value, and whether an else edge is dead, ask only whether a complement holds
a value, and a kind holding no line answers that with nothing built (
leavesindescr/mod.rs). A difference reads only the kinds its subject holds (minus), and a product holds what an edge's neighbors take in place of their complement. A meet starts from its first member rather than from every value. The relations of the JSON Schema containment suite read 37% fewer instructions on a release build,--decision-sets29.82% fewer and--decision-lengths66.37%, and what a relation keeps of its allowance decides a few a dearer reading declines. - A pair the sets read whole is not read again. Their reading of a pair spends no step,
so it reads alike however often it is read. Neither the last asking's root nor a second
asking lowers it again, and
dict[str, int]below a dict of 4,096 literal keys reads 761 million instructions where it read 1.87 billion, a release's being 719 million. - A wide union asks the rules before it lowers a branch. A branch the rules refute
is one the sets never prove, and what the root keeps of that fits 64 branches, so a
wider union lowered the subject once a branch:
frozenset[Literal[0, ..., 511]]below the union of the 512frozenset[Literal[i]]read 8.9 billion instructions. Asked of the rules first it read 350 million, with no answer moved over the families and the instruction gate within 0.05%. - A product of two automata is charged by its table. One unit of a build's allowance
bought a product of any size, thousands of entries for a Unicode class, so a pattern
over
\wbeneath four lists read 3.7 s before a bound declined the whole. Charged by its entries, the reading of the whole refuses early, the last asking reads the leaves, and the pair answers in 0.5 s, with no answer lost over the word families. - A table of constants beside its kind is the table. A translation from JSON Schema
writes an
enumof strings asstrmet with the literals, and the relation between two read as a product of the automata of their words: 60 words against 400 took 79 ms, where the two tables' own relation takes 0.2. A meet whose other members hold each constant is read as the table, in 1.6 ms, and a meet holding none pays a test a member. - A supertype proved empty needs no difference.
intbelow a recursive attribute record with no value read for up to a second, on the second reading's unfoldings, for a questionis_emptyanswers at once. The relation reads the supertype's emptiness first, where its shape can hold none and it holds a reference. - A leaf is read by the sets last, and alone. A goal the rules decline, with no reference on either side, is the sets' to answer where the pair's own reading stopped at a reference beside it. Asked of every such goal, it cost the relation matrix ninety-six times its instructions, since a goal declined on the way to another rule's answer is the common case, so it is asked in a last asking of a pair everything else declined, and only of one holding a constructor or a reference. Lowerings are charged steps, so an asking reads at most sixty-four.
- A step's table is charged by what it holds, and shared rather than copied. An
operation over a
MultipleOf's table reads a list per residue, so one unit of a build's allowance bought up to 4,096 lists and each automaton edge kept them. A unit per thousand residues and spans leaves every table under that at no charge, and two list bounds over a step of 4096 decline in under a second, where they read for a minute, kept gigabytes, and ended a process short of memory. A descriptor holds its integers' and floats' tables by handle, since no operation edits one:--decision-matrixreads 0.80% fewer instructions and--decision-sets1.11%. - A class an operation hands back is fitted as it stands. An operation over a
step's table reads each residue's spans and fits the result to that residue, a
search per end for a float of the class. A class is mostly nothing or every float
of its residue, so the result is mostly one of the two lists it was made from, each
fitted already (
refittedindescr/floats.rs):--decision-stepsreads 10.8% fewer instructions. - A descriptor is equal to itself at its own address. A product of two unions
sorts what it keeps, and its atoms hold their fields by handle, so most of what a
comparison reaches is one allocation read on both sides, walked to its depth
field by field. The order reads the address first (
Descrindescr/mod.rs):--decision-unionsreads 1.12% fewer instructions, and a relation an undecided meet of a recursive schema reads in 13 ms takes 6.7. - A language is empty where no state accepts. Every table a set holds is minimal
or the complement of one, so a word reaches every state, and whether the language
holds a word is a scan of the accepting flags (
RegularSet::is_emptyindescr/regular.rs), where a walk asks a queue of each table a kind holds. A float set read at its own period is read as it stands (split_atindescr/floats.rs), since a set built at a period holds its classes fitted, and fitting them again finds what it holds.--decision-stepsreads 23.51% fewer instructions and--decision-keys8.53%, and the containment relations 1.9% fewer.
A pair's constants are read once per validator, not once per question.
Relating two validators pools one's constants into the other's, and the key a
constant is pooled by is read out of the interpreter. Each validator keeps the
keys of its own pool from the first relation it is a side of, so asking about a
pair again reads neither pool. Relating two tables of ten thousand codes costs
5.4 million instructions that way, 2.3 million of them the rules' own answer;
reading the keys afresh for each question costs 15.0 million. Two validators
built alike compare equal in one pass over their members and their recursive
definitions, each definition against the one at its own index. Two whose
definitions were numbered apart take a second reading only where each holds two
definitions or more and their bodies agree as multisets, and that reading pairs
the definitions in the order construction numbered them before the trees are
read, so each body is read once (schemas_equal in
crates/valgebra-py/src/equality.rs). A validator keeps its hash once
computed, as a frozenset does.
A refutation about a class reads the class. It stands on a direct instance
of the class, and what that instance can carry is read off the namespaces of
the class's __mro__: which names each defines, whether each is a slot or a
data descriptor, and whether instances carry a __dict__. The namespaces of
object, the builtins a kind is read from and BaseException are static types
no assignment reaches, so each is read once for the process. A class's own
namespace is read once and kept: an assignment to the class between two
relations moves what the second one reads, so the next relation walks the
namespace and takes the kept reading only where it holds the same names, in
their order, each read as the same member, which is the reading a fresh one
would build. The walk spends what reading the namespace's items spends, and no
copy of a name, no sort and no hasattr on a builtin type: a refutation between
two dataclasses reads about 40% fewer instructions than with its classes read
afresh, on CPython 3.15 and 3.12, as scripts/perf_gate.py --binding-relation
counts it. A kept reading holds strings alone, so a class a program drops is
freed as it would be.
There is no wall-clock gate on this, and that is deliberate: a relation's
wall-clock cost is a property of the pair, and pinning one would be pinning a
number the next rule changes. The instruction gate covers the decision
workloads crates/valgebra-core/examples/ holds -- one for each path a relation
takes, which MODES in scripts/perf_gate.py names -- and those are what a
change to the rules moves. What the bindings read off a class is no rule's, and
no decision workload reaches it, because their classes are identities with
nothing behind them; --binding-relation relates two dataclasses through
relation_to and counts it.
The schema representation¶
- A node carries its children by a shared handle. A child is a shared
pointer and a node's lists are shared slices, so carrying a subtree across a
rebuild is a reference count rather than a copy; a field's name is an
Arc<str>, atomic because a validator is shared across threads on a free-threaded interpreter. - A transform hands back what it was given when nothing changed, so opening a schema that holds no record, or shifting the half of a composition whose indices do not move, returns the caller's own handle.
- An operand whose indices do not move is not walked. Composed into a pool
and a table that hold nothing yet, or into a table holding its definitions at
its start with its constants at their slots, an operand is the handle it holds
(
Placed::Keptincrates/valgebra-py/src/build.rs), where the remap would walk every node to find that none moves. A record nested sixty deep, each level met with the JSON values, composes in about half the time, since each level paid that walk over the levels below it, and--binding-compose, sixteen such levels, counts 21% fewer instructions than with the walk. - A composition moves the incoming half in one pass. Its definitions are
found as a block where the table already holds them, or appended whole, and
either way one shift moves every reference to them in the pass that moves
its constants; only a placement that finds some of them and appends the rest
renumbers the tree a second time (
place_definitionsincrates/valgebra-py/src/build.rs). - A node built twice is one node. A member list, field list, clause list or
child built twice is one handle, held in a per-thread, direct-mapped table of
weak references (
crates/valgebra-core/src/ir/intern.rs). Sharing is decided by a test stricter than structural equality -- the same allocation, or the same value in every field -- so a caller cannot observe which handle it got; and the table is what makes two equal schemas cheap for a relation to compare. - Two handles of one node compare at once, under the order as under
equality. A union and a meet keep their members sorted, and
Schema's order asks whether two nodes are one allocation before it reads them (impl Ord for Schemainir.rs), asArc's equality does. A subtree shared many times is a tree many times its size, and an unfolded fixpoint is one: sorting a union of its prefixes walked each shared child as the whole tree, where a relation over forty such prefixes spent 85% of its instructions comparing a child with itself (callgrind on a profiling build,<Schema as Ord>::cmp). - An unfolding builds one copy of a body a level (
Schema::unfolded): a reference's unfolding is built once per definition, unfoldings left and polarity, and every reference holds that handle, so a body naming its definitionmtimes unfolds twice into two copies rather thanmofm. - A record opened pays one test.
clauses_overanswers a node with no clauses in its caller and reads a clause list out of line (clauses_over_a_listinir.rs): with the whole function a call, opening the core workload's records reads 1.9% dearer. - Lists are assembled in per-thread buffers and moved into the node, and the lists every record shares -- an open record's catch-all, a closed one's empty one -- are allocated once for the process.
The build¶
The release profile in Cargo.toml links with fat LTO into one codegen unit and
wraps rather than checks integer overflow, which the dev and test profiles trap;
[profile.profiling] is the same build with its symbols kept, for a profiler.
The published CPython wheels for glibc Linux, macOS and Windows x64 are
profile-guided, trained by scripts/pgo_workload.py, which pyproject.toml
names as pgo-command, and the musllinux, Windows arm64 and PyPy wheels are
plain release builds (.github/workflows/release.yml says why); what that buys, and on which
shapes it costs, is measured above. A path the training
workload never enters is laid out by chance, so the workload reads lists of
scalars, of literals, of one class, of tuples, of named tuples, of dataclasses
and of records, records open and closed, the closed ones built two hundred times
each so the frontend holds counts of its own, a deeply nested list, scalars
through is_valid and through x in v, the explaining walk, a JSON array of
records and a recursive
JSON document, a set, a named tuple read against a tuple schema, and relations
at both of their levels. TRAINED in the workload names, for each shape a gate
measures, the calls that take its reading, and UNTRAINED names the shapes none
does with what each reads: a value walked against a recursive schema and a union
read over a JSON array among them.
Measured and not taken¶
Each of these reads as an obvious gain and measures as a loss on a shape the
gate holds, under scripts/perf_gate.py --against unless it says otherwise:
- Resolving a record's keys by iterating the dict, rather than probing each declared key: 47.9% dearer, as above.
- Reserving a record's field vector from the dict's length while building it: the build shape reads 9% dearer, because one large request leaves glibc's fastbins, where growing from empty is served from bins already warm.
- An unstable sort for the canonical orders: 7% dearer on the build shape, since its swaps over a record's fields cost more than the stable sort's scratch buffer.
- A memo over every goal of a query: up to 18% dearer on the queries whose goals do not repeat, which is most of them; the memo kept is one entry, in the one loop where a goal repeats.
- Comparing an order bound as a native integer, a value at a time: 28% dearer
on
--binding-refined, since reading a Python integer out costs more than the rich comparison it would replace. Read once per value and once per bound as a machine word, it is still 4.6% dearer on CPython 3.14, and anintpast the word raises on every read, nearly four times the cost. Afloat, whose value is a field and whose read cannot raise, is taken (Walking a Python value), and a list reads its bounds once and stops reading words at the first that raises, which is what makes a list's range a gain. - A JSON string cache of keys only, or none: jiter's default caches every string it parses, and parsing alone on CPython 3.14, a 1.5 MiB document of ninety thousand distinct strings reads 18% faster caching keys only and 28% faster caching none. The others read slower: 1.8 MiB of records 7.5% and 9.4%, 1.1 MiB of strings drawn from fifty twice the time, sixty thousand one-key objects 4.6% and 29%.
- Answering a union of scalars by the element's exact type, from a table
of the builtin types the union's kinds admit built once per list: a list of a
thousand
int | strcosts 25% fewer instructions in the PGO wheel, but an eight-element list costs 26% more for the table, a homogeneous tuple ofint | None26% more, and--binding-nullablereads 6.8% dearer on CPython 3.12, where the member test the table replaces is already inlined. - Remembering a string's length across a refinement's bounds:
Annotated[str, MinLen(1), MaxLen(64)]reads the length once for each bound, and a length kept for the second costs a list of a hundred such strings 11.6 to 15.5% fewer instructions on CPython 3.14. Kept as a parameter of the bound check, reset where the walk runs code, or keyed by the value's address, it costs--binding-refined2.0 to 4.3% more; read in one pass over the run of length bounds, it costs that shape nothing and the list of strings 12% more. A list folds its bounds into one range instead, read once for the list (Sequences and sets). - A two-word walk context: one reference to everything a walk reads,
beside its mode, travels in two registers and is never copied, but a read
through a frame takes one load more:
--binding-boundaryreads 8.7% dearer,--binding-refined4.1% and--binding-subclass3.4%, where the JSON shapes read about 1% cheaper. - A replacement global allocator: up to 28% cheaper on the JSON shapes and 12% dearer on the call boundary, past the gate's ceiling, for a C dependency on every wheel target.
- A snapshot of a list on CPython 3.14: 1.65 ns an element against 1.28 in place, timed, because that interpreter makes the reference counts cheap.
-
Aligning code to 64 bytes: a function starts on a 16-byte boundary, so a change anywhere in the library can move it 16 bytes across the processor's 64-byte fetch lines. A list of a thousand refined integers read 2% more cycles on CPython 3.14 in one build than in another at an equal count, nearly all of it frontend stalls: its function's code is identical and starts 16 bytes later in its line. Loop heads aligned to 64 bytes leave it 1.7% dearer, and functions aligned to 64 bytes 2.0%, for a library 1.5% larger; every other shape moves about 1% either way under either, inside its own shape's spread. Read with
perf_inspect.py cycles, thirty rounds. -
Reading a meet's complemented member as a difference while lowering: the relations of the JSON Schema containment suite read 0.6% fewer instructions, since the complement a meet stops building is mostly built again by the record that holds it, for five more lines at the build's busiest match.
- Forcing the emptiness entry into its callers:
#[inline]onverdict_recorempty_and_regionchanges no count, and#[inline(always)]moves the call one frame down and reads the relation matrix 0.06% dearer. -
A list of dataclasses reading its record's names once, rather than once an element:
--binding-object-walkreads 0.67% cheaper, and drawn apart from the record's check, what is left of that check is inlined into the walk's every node, which reads a list nested twenty-five deep 8% to 12% dearer in cycles at one count. -
A tagged union's test at the first branch walked, rather than before the scan: a union of scalars reads 0.37% cheaper, and a recursive schema 0.83% dearer and a union of literals 0.18% dearer, for the same two tests.
- A definition's tree plan asked by its bit in a
u128: the shift of two words is thirteen instructions at every reference, which a recursive schema with no tree of scalars pays, and--binding-memoread 2.1% dearer on the build without a profile. The arm asks whether the index holds a plan at all. - A leaf evaluated node by node at each element:
number,(int | float >= -inf) & ~bool, walked as its meet, union and complement over afloatcosts about 240 instructions where its specialization tofloatis one comparison, and the JSON values over two hundred records read 18.5% dearer than with the tree's own bounds. The plan specializes the leaf to each exact type at build. - A parsed value read as a tree of scalars: each value's tag is its exact
type, and the JSON values over a document of two hundred records cost 35%
fewer instructions to check, the parse included. Taking it means the
reference's arm stops asking whether its value is a Python one, and the
profiled CPython 3.15 wheel then laid
memberout so that a list nested twenty-five deep, which enters no reference, read 1.8% more instructions and 12% more cycles, thirty rounds of thirty, its frontend stalls three times as many. - A reference's arm asking past one word whether a tree is held: the plans of leaves and trees share one word of the index, and asking for the trees behind it is a load more in an arm inlined into every walk, which the profiled wheel read three instructions a level dearer on a list nested twenty-five deep, a walk that enters no reference. The arm asks the word alone, and a validator holding leaves and no tree pays a call that declines.
- The tree plans held in two words of the validator's index: its fields
fill 248 bytes under an alignment of 16, so a second word grows it to 272,
and every frame holding a validator is laid out anew:
--binding-boundaryread 2.9% dearer on the build without a profile, though it walks no reference. One word fits the alignment's padding. - Leaving a tree of scalars out of the memo: a value meets at most one of a
tree's members, so a walk of it reads each value below once, and a JSON
Schema translation's meets of
list[J]anddict[str, J]arm the memo for the tree alone. Not remembered, the tree is read whole again at every level of such meets, where kept, each container is read once: on the profiled CPython 3.15 wheel valgebra-jsonschema's record array checked in 43.6 us where the kept walk took 79, but a chain of sixty objects against a recursive$reftook 56.7 us where it took 26, a chain of forty arrays of objects 124 where it took 38.5, and its$reftree eight deep 220 where it took 196. A kept walk reads the tree's scalars by type instead (a kept walk). - Reading a record's leaf field by its type in the record's field test: a
field JSON Schema's
integertranslates to, a meet, is walked wherescalar_memberanswers a bare kind. Read there by its leaf, valgebra-jsonschema's record array counts 4% fewer instructions on the profiled CPython 3.15 wheel, but the field test is inlined into every record's field loop and every default clause's, and the arm moved them all though no gate shape reaches it: a record walked by interned keys read 2.8% dearer, a mapping of lists 11.5%, and the other record shapes 1.2% to 1.9%. - Reading a record's leaf field at the meet's arm of the walk: the same field
read by its plan where
memberdispatches a meet, behind the index's one word, as the reference's arm asks it. valgebra-jsonschema's$reftree of 511 nodes counts 18% fewer instructions and its record array 18%, but the arm lays outmember, which every shape crosses: on the profiled CPython 3.15 wheel the list nested twenty-five deep read 6.73% more instructions, a record 1.53%, a dataclass 1.67%, a literal 1.62%. - Training the loop over a leaf holding a tree: a line in
scripts/pgo_workload.pyreading a list of(~NUMBER | Ge(0)) & Jtakes theif/thenarray from 165,119 instructions a call to 110,191, but the list nested twenty-five deep reads 2.36% more, at a hundred rounds as at five hundred: past the profiled gate's ceiling. The loop runs untrained.
Regression gate¶
The wall-clock numbers above are for humans reading results; they are too noisy
on shared CI runners to gate a merge. The merge gate is instead a deterministic
instruction count: scripts/perf_gate.py --against runs each fixed workload
under cachegrind at the change and at its base, in one job with one
toolchain, and fails on a rise past the band it allows a change. The count is
identical across runs of a given build, so the gate does not flake, and
measuring both sides in one job cancels what another machine or another
toolchain would add, while an algorithmic regression is far larger than the
band. The counts in scripts/perf_budget.json are a record of one
environment rather than a gate, since the same commit re-measures several
percent away on another machine: the gate reads one only where the base has no
count to compare with -- a shape the base does not carry, or a re-recorded
workload the base cannot build -- and a step the file records excuses the one
comparison that crosses it (docs/dev/tooling-ci.md).
The gate holds one workload per surface, because a gate only catches what it
exercises. crates/valgebra-core/examples/ holds the pure-Rust ones: the schema
transformations (perf_workload), and the decision procedures, one workload for
each path a relation takes, which MODES in scripts/perf_gate.py names -- a
proof, a refutation, a repeated goal, a shape only the structural readings
decide, and one corpus asked of the rules and of the sets each walk a different
path, and a workload that asks only one of them measures only that one. The binding's shapes are
BindingShape in crates/valgebra-py/src/workload.rs, run by
crates/valgebra-py/examples/binding_workload.rs, and BINDING_SHAPES in
scripts/perf_gate.py names every one the gate measures: the membership walk
over a live value, the call boundary, the record walks closed, open and over
interned keys, the builds, the JSON document and the explaining walk among
them. The walk is the shipped hot path neither pure-Rust workload reaches, and
construction and the open record are each a cost no other shape's count
carries. A surface with no shape of its own can move by percents a release at a
time with every gate green.
Each binding shape embeds CPython, whose startup is not a fixed count, so the gate measures the difference between two iteration counts. What a shape's loop holds is part of what its count means, and is read before the number is: a build shape that assembles its fifty fields inside the loop spends most of its count on the harness naming them and none of it on the annotation walk, which is the half a build gate exists to measure.
One thing an embedded interpreter brings with it is its string hash seed, drawn per process; a shape that probes a dict of string keys executes a different number of instructions under every seed, and on the difference of two counts that is a few percent -- the size of the ceiling. The gate fixes the seed where every measurement passes through, so a reading is of the code and not of the interpreter's draw. A number in this page is a wall-clock figure for a human; the counts belong to the budget file and are not repeated here.
The end-to-end wall-clock suites run on the same CI lane with timing disabled, as a smoke test that they keep working.
The headline claim — that valgebra's check is as fast as pydantic's — is gated
too, by scripts/compare_gate.py. For each shape in a matrix it measures the
ratio of per-call time (valgebra over pydantic's strict
TypeAdapter.validate_python, called through a lambda as a caller holds it),
taking the minimum over many repeats, and holds it under the ceiling
scripts/perf_compare.json states for that shape -- what the project claims,
not what it measured -- and, where the file carries a ratio recorded in the same
environment, within that shape's tolerance of it. The lambda and the adapter are
a Python frame a direct pydantic-core SchemaValidator call does not pay, and
on a scalar that frame is most of pydantic's time: against SchemaValidator
the scalar shape reads about 1.0, several times what its ceiling against the
adapter allows (ceilings in scripts/perf_compare.json), on one box with a release build and no profile. So the ratio is to what a caller
reaches, not to pydantic-core. A
ratio cancels the runner's absolute speed: if the machine is slow, both
libraries are slow in proportion, so the comparison survives the shared-runner
noise an absolute budget cannot. A shape fails the merge gate when valgebra's
ratio crosses its ceiling, or drifts past its recorded ratio where one is armed
— a competitive regression, whether from valgebra slowing down or ceding ground.
Re-record the counts after an intentional change with:
uv run --no-sync python scripts/perf_gate.py --update # the core count; --decision, --binding-record, ... for the others
uv run --no-sync --group bench python scripts/compare_gate.py --update # one run's ratios, a reading for a window
uv run --no-sync python scripts/compare_gate.py --record-window WINDOW.json # arms the ratchet from the lane's window
Reading one build against another¶
The gates above each hold one reading of one build against its base.
scripts/perf_inspect.py reads any build against any other -- a release on
PyPI, a wheel, or a revision built as the release builds it -- over the shapes
scripts/perf_shapes.py holds, on the interpreter it is given, and says which
of three readings each figure is:
| reading | what it counts | what it cannot see |
|---|---|---|
count |
instructions per call under cachegrind, as the difference of two runs, in the instruction gate's environment | a cache, the branch predictor, the alignment of a loop: time |
cycles |
the processor's own cycles, instructions, branch mispredicts and frontend stalls per call, through perf_event_open, the environments interleaved |
a difference under the spread between two readings of one build, and any machine whose kernel gives no counter |
symbols |
the functions, or with line tables the source lines, that spend the instructions one build spends over the other; with --per N, the instructions run N times a call, disassembled |
time, as count cannot |
uv run --no-sync python scripts/perf_inspect.py env last --python "$(uv python find --managed-python 3.15)" --release 0.0.19
uv run --no-sync python scripts/perf_inspect.py env head --python "$(uv python find --managed-python 3.15)" --build
uv run --no-sync python scripts/perf_inspect.py count last head
uv run --no-sync python scripts/perf_inspect.py cycles last head --shapes deep records
uv run --no-sync python scripts/perf_inspect.py symbols last head --shape deep --per 25
env makes an environment under target/perf-inspect/ holding the standard
library and one valgebra; --build builds the checkout, or the revision it
names, with the training's hash seed fixed, from a directory whose path is one
length whatever the tree, so two builds differ in their sources and nothing
else. Every reading is against the first environment named. count exits 1
where --tolerance is given and a shape rose past it; every reading exits 2
where it cannot be taken. It needs valgrind, uv for --release, and for
--build the llvm-tools component the PGO build needs.
Read a change against the last release on every interpreter the wheels
ship. Each interpreter's wheel is trained and laid out on its own, so a
reading on one says nothing about another: one release read a list nested
twenty-five deep 3.1% dearer than the release before it on 3.12 and 0.2% on
3.14, and a hundred-entry dict[str, int] 5.4% dearer on 3.14t and 2% to 3%
cheaper on every build with a global lock. One env per interpreter and side, and one
count per pair, is the reading; a survey of one interpreter is a reading of
that one.
Decide on the count, and confirm in cycles. On a build with the GIL a count
is deterministic: two builds of one source and one toolchain read it to the
instruction between environments whose names are one length, so a difference is
the code's -- on a shape that allocates, where it reads the same at a second
size of the environment, as the next paragraph says. On a free-threaded build, a
shape of a few thousand instructions is not exact: two counts of one pair on
3.15t read literal -0.03% and -13.5%, and scalar +6.3% and +8.7%, where
validate read +1.9% both times. Count such a cell twice, and read one that
does not repeat as nothing shown. A count does not predict time on every shape:
that nested list read 0.2% more instructions in one release than in the one
before it, and 4% more cycles. cycles reads what the count cannot and carries
the machine's noise, so it starts each round at the next environment, prints the
median of the rounds, and beside it the rounds in which each environment read
slower: a cell inside the spread of the same build read twice has shown nothing.
On the machine above that spread is about 2%, 4% on a shape of a few hundred
cycles, and it is a shape's own: two environments of one build read list[int |
str] 14% apart, at every round of two runs, where list[int] read 0.6% apart.
So the control is the base in a second environment, read in the same run, and
each cell is weighed against its own shape's control.
A shape that allocates is counted at a second size. It reads the heap its
loop starts from, and everything run before the loop leaves that heap, the
environment included: one release in two environments named a character apart
read explain_record 3.1% apart and json_records 0.7%, and to the instruction
at one length. Two builds may move together as the environment grows, or apart.
Two that build a validator to the instruction alike differed by the same count,
within 0.02%, at every size up to 8,200 bytes larger on 3.11, 3.12, 3.14 and
3.15, where each alone spread up to 3.7%; the builds of two releases read
json_deep anywhere from 1.5% cheaper to 8.0% dearer by the environment's size
alone. count --pad BYTES adds that many bytes to every run's environment, so a
shape that allocates and reads dearer is counted again at two sizes, --pad 600
and --pad 1100: a difference that is the code's reads the same at each.
Trimming the heap before the loop, as the binding gate does, is not the remedy
here: it took explain_record's 3.1% to 0.02% and moved json_deep's from 0.3%
to 0.7%.
Read a change against its base built the same way. A base --build made
differs from the change in its sources alone; a release also differs in the
container and the path it was built in, which the layout follows, and in the
binary a shape that allocates hands the C library's allocator: on 3.11 PyPI's
0.0.19 read explain_record 0.84% below a build here of its own commit, 167
instructions a call in _int_malloc, and 0.39% above it with the environment
1,100 bytes larger. So a change is read against its base's revision built here,
and charged in cycles only that way. A release's own wheel is read by count, for
what it ships, and once published in cycles against the last release's, for what a
user meets: step 9 of docs/dev/releasing.md names a move there and charges it
to no change.
symbols names where, not why. A release wheel is stripped, so it names
the extension's functions only between builds --build made; against a
release it names the C library's alone, which is where an allocator's luck
shows -- a realloc that copies, read as memcpy. A function inlined into
another reads as its caller, which is why --per reads past functions to the
instructions themselves. --lines adds line tables and leaves the code as it is -- a list
nested twenty-five deep counts the same with them and without -- while the
length of the path a tree is built from moves it: the same tree built from a
path two characters longer read that list 1% cheaper. --build unpacks every
tree at one length for that reason. tests/test_perf_inspect.py holds the
readers of both output formats to hand-written files.