Refinements¶
A refinement narrows a base type to the subset satisfying one or more
constraints. Write it with Annotated[T, ...markers]; the base T is checked
first, then each constraint. valgebra reads the
annotated-types markers
structurally, so it has no runtime dependency on that library.
That is a fact about valgebra, not about your environment: the examples on
this page import annotated_types, and pip install valgebra does not bring it
in. Install it alongside — pip install annotated-types — or write the same
constraint with valgebra.Regex, which needs nothing beyond valgebra.
from typing import Annotated
import annotated_types as at
from valgebra import Validator
adult = Validator(Annotated[int, at.Ge(18), at.Le(150)])
assert adult.is_valid(21)
assert not adult.is_valid(5)
Refinements built from bound and length markers also take part in the decision procedure: a refinement is a subtype of its base and of a looser refinement, and a bound conjunction that cannot be satisfied is detected as empty.
from typing import Annotated
import annotated_types as at
from valgebra import Validator
assert Validator(Annotated[int, at.Ge(0)]).is_subtype_of(int) # refinement <= base
assert Validator(Annotated[int, at.Ge(0), at.Le(10)]).is_subtype_of(
Annotated[int, at.Ge(0)] # a tighter bound is a subtype of a looser one
)
assert Validator(Annotated[int, at.Ge(10), at.Le(0)]).is_empty() # no such int
A predicate marker is checked at validation time, and nothing is inferred from it
— its satisfiability is undecidable in general, so two refinements relate through
a predicate only when they carry the same one. A decision query may still call it:
asking whether a literal is a subtype of a refinement asks whether that literal's
value belongs to it, which runs the predicate. A slow or side-effecting predicate
is one is_subtype_of pays for.
Supported markers¶
| Marker | Constraint | Failure code |
|---|---|---|
Ge(n) |
value >= n |
greater_than_equal |
Gt(n) |
value > n |
greater_than |
Le(n) |
value <= n |
less_than_equal |
Lt(n) |
value < n |
less_than |
MinLen(n) |
the value's length is >= n |
too_short |
MaxLen(n) |
the value's length is <= n |
too_long |
MultipleOf(n) |
value % n == 0 |
multiple_of |
Regex(p) |
the string fully matches the regex p |
string_pattern_mismatch |
Predicate(f) |
f(value) is truthy |
predicate_failed |
Which base each marker can be asked of¶
A marker narrows a base by asking its values a question, so the base has to be one the question can be put to. This is which:
| Narrowing | The bases whose values answer it |
|---|---|
MinLen, MaxLen |
str, bytes, list, tuple, set, frozenset, dict — the kinds with a length |
Regex |
str — the kind with text for a pattern to match |
MultipleOf |
bool, int, float — the kinds with a number to divide |
Ge, Gt, Le, Lt |
every kind whose values compare with the bound: the numbers against a number, str against str, bytes against bytes, a list against a list, a tuple against a tuple, a set or frozenset against a set. Not None and not dict, whose values compare with nothing |
Predicate |
any base: the predicate is a black box and the value is handed to it whole |
A bound is the row to read twice, because it is the only marker whose answer
depends on both values. Annotated[set[int], Ge(0)] is refused although a
set has an order of its own, because a set and an integer do not compare;
Annotated[set[int], Ge({1})] is the same base narrowed by a bound its values
can be asked about, and it admits the supersets of {1}.
A marker put to a base outside its row is refused when the validator is built,
for the reason above: the check would raise at every value, the walk reads a
raise as a non-member, and the schema would admit nothing while reading like a
narrowing. tests/test_constraint_matrix.py drives the whole of this table,
cell by cell.
Interval and Len expand to the bounds they carry (below), and Not wraps a
predicate. A marker from annotated_types that is not in this table names a
constraint valgebra does not check — Timezone and Unit are the two — and is
refused when the validator is built rather than ignored: ignoring it would leave
a schema that admits exactly the values the marker was written to exclude.
A bound excludes nan. Every comparison against a not-a-number is false,
>= included, so a bounded float set never holds one while the bare kind does.
This is the ordinary reading of the marker rather than a special case, and it is
worth knowing because it is the one narrowing that removes a value the bound
does not name.
from typing import Annotated
import annotated_types as at
from valgebra import Validator
nan = float("nan")
assert Validator(float).is_valid(nan)
assert not Validator(Annotated[float, at.Ge(0)]).is_valid(nan)
assert not Validator(Annotated[float, at.Le(0)]).is_valid(nan)
A value has one length. For a list and a tuple it is the number of
elements the value holds — the same number a sequence schema counts when it
walks them. For everything else it is __len__, which is how a str, bytes,
set and dict are read anyway. The distinction is visible only for a list or
tuple subclass that overrides __len__: its MinLen is measured against what
it stores, not against what it reports, so Annotated[list[int], MinLen(5)] and
the shape [int, int, int, int, int] narrow by the same count. A length two
parts of one schema disagreed about would not be a property of the value, and a
set defined by one would not be a set.
A constraint must also be one the base can answer. Asking an int for its length
raises, and a raise reads as a non-member, so Annotated[int, MinLen(1)] would
denote nothing at all while looking like a narrowing. A constraint no value of
the base can answer is therefore refused too; where some value can — a union with
a str branch, or a class that may define what the constraint asks for — it is a
narrowing and stands.
from typing import Annotated
import annotated_types as at
from valgebra import Validator
try:
Validator(Annotated[int, at.MinLen(1)])
except NotImplementedError as error:
assert "length" in str(error)
# One branch can answer it, so this narrows rather than empties.
narrowed = Validator(Annotated[int | str, at.MinLen(1)])
assert narrowed.is_valid("a")
assert not narrowed.is_valid(5)
Regex is valgebra's own marker (from valgebra import Regex), since
annotated-types defines none for strings. The match is anchored — the whole
string must match, like re.fullmatch — and runs natively in Rust with a
linear-time engine (no catastrophic backtracking), so unlike a Predicate it
stays on the fast path and never crosses into Python per value. An invalid
pattern is rejected when the validator is built, not at first use. A compiled
re.Pattern works as metadata too:
import re
from typing import Annotated
from valgebra import Regex, Validator
oid = Validator(Annotated[str, Regex(r"[0-9a-f]{24}")])
assert oid.is_valid("0123456789abcdef01234567")
assert not oid.is_valid("0123456789abcdef0123456X") # not hex
assert not oid.is_valid("0123") # not the full 24 characters
assert Validator(Annotated[str, re.compile(r"\d+")]).is_valid("123")
# A compiled pattern's flags are part of the pattern and are carried over.
assert Validator(Annotated[str, re.compile("abc", re.I)]).is_valid("ABC")
A pattern is matched against text, so a bytes pattern is refused. So are
re.ASCII, re.LOCALE and re.DEBUG: the first two change what a character
class means in ways this engine spells differently (see the dialect section
below), and the third asks the other engine to report on itself.
re.IGNORECASE, re.MULTILINE, re.DOTALL and re.VERBOSE are written into
the pattern, which is how this engine spells them.
The dialect is Rust's, not re's¶
Running natively is what buys the linear-time guarantee, and it is also what makes the dialect the Rust engine's. The two languages are close but not equal, and a pattern both engines accept can denote different sets. Compiling successfully is therefore not a test of which language a pattern is in. Four places the two engines part, which a ported pattern should be checked against — this is where they differ, not a complete audit of what a pattern denotes:
import re
from typing import Annotated
from valgebra import Regex, Validator
def admits(pattern: str, text: str) -> bool:
return Validator(Annotated[str, Regex(pattern)]).is_valid(text)
# POSIX bracket expressions. Python has no such class, so it reads a character
# class followed by a literal `]` and needs two characters.
assert admits(r"[[:alpha:]]", "a")
assert re.fullmatch(r"[[:alpha:]]", "a") is None
assert re.fullmatch(r"[[:alpha:]]", "a]") is not None
# Case folding. Both fold the ASCII pair; only Python folds the Turkish one.
assert admits("(?i)i", "I")
assert not admits("(?i)i", "\u0131") # dotless i
assert re.fullmatch("(?i)i", "\u0131") is not None
# Property escapes. `\p{...}` is a pattern only this engine accepts.
assert admits(r"\p{L}+", "ab")
# Class-set operators. `--`, `&&` and `~~` combine classes here and are literal
# characters to Python, so a class carrying one is refused rather than read.
try:
admits(r"[\w~~\d]", "a")
except ValueError as err:
assert "set symmetric difference" in str(err)
# A nested set is refused for the same reason: a union here, four literals
# there.
try:
admits(r"[a[bc]]", "a")
except ValueError as err:
assert "nested set" in str(err)
The third is the loud case: re.compile(r"\p{L}+") raises, so a pattern that
works here fails there and a reader finds out at once. The first two are the
quiet ones — both engines build the pattern and answer differently — and they
are the reason a library holding itself to re's decisions cannot adopt Regex
behind a fallback that triggers on compile failure.
The fourth is neither, because it is not accepted: a character class carrying
--, &&, ~~ or a nested [ raises a ValueError naming the operator, in
the words re warns about it with. Python reserves all four and warns that it
may one day read them as operators; this engine reads them as operators now, so
the same pattern denotes two sets and nothing about compiling it says which.
Refusing is the loud direction, and it is the one a reader can act on. Escape
the characters to mean them literally — [\w\~\~\d] — or write the classes
out. A POSIX class ([[:alpha:]]) is the one nested [ that is read rather
than refused, because it is the first divergence above and is documented there.
A pattern Python spells and this engine does not is refused, which is the
loud direction and the one to prefer. A lookaround ((?=...), (?<=...)), a
backreference (\1, (?P=name)), an inline comment ((?#...)), a named
character (\N{BULLET}), the ASCII flag (?a), the \Z anchor and the
open-ended {,n} repetition all raise a ValueError naming the parse error,
rather than being read as something else.
Agreement with re is not ASCII¶
Checking a pattern against the list above tells you the two engines agree. It
does not tell you the pattern denotes what you meant, and the character classes
are where that bites: \d, \w and \s are Unicode-aware here, exactly as
they are in re. The two engines agree, and a reader who wanted digits gets
every decimal digit Unicode defines.
import re
from typing import Annotated
from valgebra import Regex, Validator
year = Validator(Annotated[str, Regex(r"\d{4}")])
assert year.is_valid("٢٠٢٦") # Arabic-Indic digits are decimal digits
assert re.fullmatch(r"\d{4}", "٢٠٢٦") is not None # `re` agrees
assert Validator(Annotated[str, Regex(r"\w")]).is_valid("é")
assert Validator(Annotated[str, Regex(r"\s")]).is_valid("\u00a0") # no-break space
Write the ASCII set when you mean the ASCII set. [0-9] is the portable
spelling; (?-u:\d) is the same thing with the Unicode flag turned off for that
group, and is this engine's syntax rather than re's:
from typing import Annotated
from valgebra import Regex, Validator
assert not Validator(Annotated[str, Regex(r"[0-9]{4}")]).is_valid("٢٠٢٦")
assert not Validator(Annotated[str, Regex(r"(?-u:\d){4}")]).is_valid("٢٠٢٦")
assert Validator(Annotated[str, Regex(r"[0-9]{4}")]).is_valid("2026")
This is the shape that reaches production: a timestamp, an identifier or a
version pattern written with \d admits strings the parser downstream rejects,
and every engine involved agreed the pattern was fine.
A bound against nan is rejected with a ValueError, and so is
MultipleOf(nan): every comparison with nan is false, so such a bound admits
no value at all. That is the empty set written as a bound, which no caller
means, and dropping the marker instead would admit every value of the base. A
bound that is empty because the order says so — Gt(inf) on a float — is kept,
because emptiness is then an answer rather than the absence of one.
MultipleOf(n) requires a nonzero number: no value is a multiple of zero,
so MultipleOf(0) is an unsatisfiable constraint, and a step that is not a
number is unsatisfiable for the same reason one step further on. The constraint
is value % n == 0, and a remainder that is not a number equals no zero —
timedelta(6) % timedelta(2) is timedelta(0), which is not the integer 0 —
so such a step names a schema no value belongs to. Both are rejected with a
ValueError when the validator is built, rather than rejecting every value at
check time. int, float, Decimal and Fraction steps all divide as they
read.
The compound markers Interval and Len expand to the bounds they carry, so
Interval(ge=0, le=10) contributes Ge(0) and Le(10), and Len(2, 4)
contributes MinLen(2) and MaxLen(4). They are read through the grouping
protocol annotated_types documents — a marker that stands for several
constraints answers for them — so a marker of your own written that way is
read the same way:
from typing import Annotated
import annotated_types as at
from valgebra import Validator
assert Validator(Annotated[int, at.Interval(ge=0, le=10)]).is_valid(5)
assert not Validator(Annotated[int, at.Interval(ge=0, le=10)]).is_valid(11)
assert Validator(Annotated[str, at.Len(2, 4)]).is_valid("abc")
assert not Validator(Annotated[str, at.Len(2, 4)]).is_valid("a")
class Percent(at.GroupedMetadata):
"""A marker of your own, standing for the two bounds it yields."""
def __iter__(self):
yield at.Ge(0)
yield at.Le(100)
assert Validator(Annotated[int, Percent()]).is_valid(50)
assert not Validator(Annotated[int, Percent()]).is_valid(101)
assert Validator(Annotated[int, at.MultipleOf(3)]).is_valid(9)
assert not Validator(Annotated[int, at.MultipleOf(3)]).is_valid(5)
Predicates: the slow path¶
A Predicate runs an arbitrary Python callable. It is the one refinement
constraint that leaves Rust for a caller's own code — literals, instance and
attribute checks, and comparison bounds also compare against Python objects, but
against fixed operators, not arbitrary callables — so it is a documented slow
path, never a silent fallback. is_valid runs it once per value it reaches;
validate may run it again, because a failing union is re-walked to find the
branch to report, and that walk asks the predicate a second time. A predicate
is user code, and every occurrence the walk reaches is a call: the walk keeps
no cache over it, because a cache would change what a predicate that does not
answer from its value alone observes, and the same reading is what keeps
A & ~A from folding when A carries one. A predicate that must run once per
value memoises on its own side. Use it for checks the markers cannot express:
from typing import Annotated
import annotated_types as at
from valgebra import Validator
even = Validator(Annotated[int, at.Predicate(lambda x: x % 2 == 0)])
assert even.is_valid(4)
assert not even.is_valid(3)
A predicate that raises is reported distinctly, as predicate_error rather
than an ordinary failure, so a buggy predicate is not mistaken for a rejected
value.
A bare callable is a predicate too, without the wrapper:
from typing import Annotated
from valgebra import Validator
positive = Validator(Annotated[int, lambda value: value > 0])
assert positive.is_valid(1)
assert not positive.is_valid(-1)
Predicate is the portable spelling — it is what pydantic, msgspec and cattrs
read — so prefer it in an annotation other tools also consume. The bare form is
valgebra's own convenience, and it excludes a class for the reason above.
A bare callable is metadata only¶
The bare form is read only in Annotated metadata position, and reading it
there at all is valgebra's own convenience: the libraries that share this
metadata channel each require a wrapper — pydantic AfterValidator, beartype
Is[...], msgspec Meta(...), annotated_types Predicate. The typing spec
leaves each consumer to say what its own metadata means, so this arm reaches
exactly as far as the metadata position and no further.
A schema language that reads a top-level callable as a predicate is expressing a
different rule for a position Annotated metadata does not cover; valgebra's
rule for that position is the one below.
Passed as a schema on its own, a callable is not a predicate. It is an object the frontend has no other reading for, so it takes the fallback literal form and denotes the one function object:
from typing import Annotated, TypedDict
from valgebra import Validator, intersection
class Record(TypedDict):
kind: str
def kind_is_known(value):
return value["kind"] in {"a", "b"}
checked = intersection(Record, kind_is_known) # NOT a refinement of Record
assert not checked.is_valid({"kind": "a"}) # a dict is not that function
assert not checked.is_empty() # nor is emptiness a warning: see below
refined = Validator(Annotated[Record, kind_is_known]) # the refinement
assert refined.is_valid({"kind": "a"})
assert not refined.is_valid({"kind": "z"})
The first schema admits nothing, and nothing reports it. is_empty returning
False is not a claim that the set is inhabited — a negative answer from any
decision is "no, or not yet proven" (see the
decidability boundary), and the meet of a
record with a literal is one it does not decide. So the failure mode is a schema
that silently rejects every value. Write the refinement as Annotated, and the
callable narrows the base rather than replacing it.
annotated_types.Not wraps a predicate and denotes the values it rejects:
from typing import Annotated
import annotated_types as at
from valgebra import Validator
odd = Validator(Annotated[int, at.Not(lambda value: value % 2 == 0)])
assert odd.is_valid(3)
assert not odd.is_valid(2)
A marker that is itself callable is called, which is what applies Not's
negation and what keeps a functools.partial's bound arguments. Only a marker
that is not callable is taken apart by its .func, which is the shape
Predicate has.
On classes¶
Refinements declared on a TypedDict, dataclass, or NamedTuple field are
enforced — the constraint travels with the field:
from typing import Annotated, TypedDict
import annotated_types as at
from valgebra import Validator
class Account(TypedDict):
balance: Annotated[int, at.Ge(0)]
assert Validator(Account).is_valid({"balance": 100})
assert not Validator(Account).is_valid({"balance": -1})
Unrecognized markers¶
Per the typing spec, metadata valgebra does not recognize as a constraint is
ignored — so non-constraint Annotated metadata, such as a documentation string,
is harmless and carries no membership meaning. The carve-out is the
annotated_types vocabulary itself: a marker from there was written to narrow
this schema, so one valgebra does not check is refused rather than ignored.
A class is among what is ignored. A marker carries its values on an
instance — Ge(0) holds ge = 0 — so the class itself holds no value to read
and calling it constructs rather than asks. A documentation marker written as a
class therefore carries no constraint, exactly as one written as an instance does
not.