Refinements¶
A refinement narrows a base type to the subset satisfying one or more
constraints. Write it with Annotated[T, ...markers]; the base T is checked
first, then each constraint, except a length bound on a container, which is read
before the elements it bounds (below). valgebra reads the
annotated-types markers by the
module they come from, so it has no runtime dependency on that library; a
marker another library writes for itself is that library's, and is
ignored.
That is a fact about valgebra, not about your environment: the examples on
this page import annotated_types, and pip install valgebra does not bring it
in. Install it alongside — pip install annotated-types — or write the same
constraint with valgebra.Regex, which needs nothing beyond valgebra.
from typing import Annotated
import annotated_types as at
from valgebra import Validator
adult = Validator(Annotated[int, at.Ge(18), at.Le(150)])
assert adult.is_valid(21)
assert not adult.is_valid(5)
Refinements built from bound and length markers also take part in the decision procedure: a refinement is a subtype of its base and of a looser refinement, and a bound conjunction that cannot be satisfied is detected as empty.
from typing import Annotated
import annotated_types as at
from valgebra import Validator
assert Validator(Annotated[int, at.Ge(0)]).is_subtype_of(int) # refinement <= base
assert Validator(Annotated[int, at.Ge(0), at.Le(10)]).is_subtype_of(
Annotated[int, at.Ge(0)] # a tighter bound is a subtype of a looser one
)
assert Validator(Annotated[int, at.Ge(10), at.Le(0)]).is_empty() # no such int
A predicate marker is checked at validation time, and nothing is inferred from it
— its satisfiability is undecidable in general, so two refinements relate through
a predicate only when they carry the same one. A decision query may still call it:
asking whether a literal is a subtype of a refinement asks whether that literal's
value belongs to it, which runs the predicate. A slow or side-effecting predicate
is one is_subtype_of pays for. The answer stands for the literal only where its
constant is its one value: Literal[0.0] also holds -0.0 and
Literal[Decimal("1.00")] holds Decimal("1.0"), which a predicate can tell
apart, so such a literal below a predicate is not proven.
Supported markers¶
| Marker | Constraint | Failure code |
|---|---|---|
Ge(n) |
value >= n |
greater_than_equal |
Gt(n) |
value > n |
greater_than |
Le(n) |
value <= n |
less_than_equal |
Lt(n) |
value < n |
less_than |
MinLen(n) |
the value's length is >= n |
too_short |
MaxLen(n) |
the value's length is <= n |
too_long |
MultipleOf(n) |
value % n == 0 |
multiple_of |
Regex(p) |
the string fully matches the regex p |
string_pattern_mismatch |
Predicate(f) |
f(value) is truthy |
predicate_failed |
A constraint is Python's own operation, and a value it raises against is
refused. The walk asks each constraint of the value and reads an ordinary
error as a failed check, so a bound over a base holding values it cannot
compare refuses them: Annotated[int | None, Ge(0)] admits the non-negative
integers and refuses None, which orders against no number, with
greater_than_equal. A length bound refuses a value with no length, and a
pattern a value that is not a string, the same way:
from typing import Annotated
import annotated_types as at
from valgebra import Validator
maybe_count = Validator(Annotated[int | None, at.Ge(0)])
assert maybe_count.is_valid(3)
assert not maybe_count.is_valid(None)
assert maybe_count.relation_to(Annotated[int, at.Ge(0)]) == "subset"
A length bound is read before the elements it bounds. On a list, tuple,
set, frozenset or dict, MinLen and MaxLen are asked once the value is of the
base's kind and before any element is read, which is where a fixed shape such as
[int, int] reads its length. A container too long for its bound is refused for
its length alone, whatever it holds, and its elements are never walked, so the
bound is a size guard too. The answer is the one either order gives, since a
member satisfies both.
The guard holds where the bound and the elements are one refinement, as in
Annotated[list[int], at.MaxLen(3)]. A bound carried in a meet of its own --
intersection(list[E], Annotated[list, at.MaxLen(3)]), or a validator in the
metadata, which builds that meet -- is one
member of the meet, and the meet reads its members in turn, so the elements of
an over-long list are walked before the bound refuses it. The answer is the
same; the cost is the length of the list, and the report names a bad element
beside too_long.
from typing import Annotated
import annotated_types as at
from valgebra import ValidationError, Validator
capped = Validator(Annotated[list[int], at.MaxLen(3)])
raised = False
try:
capped.validate([1, 2, "x", 4, 5])
except ValidationError as err:
raised = True
assert [item["code"] for item in err.errors] == ["too_long"]
assert raised
Which base each marker can be asked of¶
A marker narrows a base by asking its values a question, so the base has to be one the question can be put to. This is which:
| Narrowing | The bases whose values answer it |
|---|---|
MinLen, MaxLen |
str, bytes, list, tuple, set, frozenset, dict — the kinds with a length |
Regex |
str — the kind with text for a pattern to match |
MultipleOf |
bool, int, float — the kinds with a number to divide |
Ge, Gt, Le, Lt |
every kind whose values compare with the bound: the numbers against a number, str against str, bytes against bytes, a list against a list, a tuple against a tuple, a set or frozenset against a set. Not None and not dict, whose values compare with nothing |
Predicate |
any base: the predicate is a black box and the value is handed to it whole |
A bound is the row to read twice, because it is the only marker whose answer
depends on both values. Annotated[set[int], Ge(0)] is refused although a
set has an order of its own, because a set and an integer do not compare;
Annotated[set[int], Ge({1})] is the same base narrowed by a bound its values
can be asked about, and it admits the supersets of {1}. A complex bound or
step is refused with a ValueError when the validator is built: no value
orders against a complex number or is a multiple of one. Two bounds over lists
or tuples are not compared by the relations: a sequence orders item by item,
so [2 + 0j] is at least [2] and orders against [1] not at all, which
puts it in Ge([2]) and out of Ge([1]). Two sets compare by inclusion and
are decided, and so is a bound over str or bytes, the words on one side
of a word being a regular set.
A marker put to a base outside its row is refused when the validator is built,
for the reason above: the check would raise at every value, the walk reads a
raise as a non-member, and the schema would admit nothing while reading like a
narrowing. tests/test_constraint_matrix.py drives the whole of this table,
cell by cell.
Interval and Len expand to the bounds they carry (below), and Not wraps a
predicate. A marker from annotated_types that is not in this table names a
constraint valgebra does not check — Timezone is the one — and is refused when
the validator is built rather than ignored: ignoring it would leave a schema
that admits exactly the values the marker was written to exclude. Two of the
vocabulary's markers exclude no value and are ignored: Unit, which names what
a number is measured in and leaves the reading to the consumer, and the
documentation marker, so Annotated[float, Unit("m")] is float.
A bound excludes nan. Every comparison against a not-a-number is false,
>= included, so a bounded float set never holds one while the bare kind does.
This is the ordinary reading of the marker rather than a special case, and it is
worth knowing because it is the one narrowing that removes a value the bound
does not name.
from typing import Annotated
import annotated_types as at
from valgebra import Validator
nan = float("nan")
assert Validator(float).is_valid(nan)
assert not Validator(Annotated[float, at.Ge(0)]).is_valid(nan)
assert not Validator(Annotated[float, at.Le(0)]).is_valid(nan)
A value has one length. For a builtin container — a list, tuple, str,
bytes, set, frozenset or dict — it is the number of items the value
holds: the same number a sequence schema counts when it walks one. For any
other value it is __len__. The distinction is visible only for a container
subclass that overrides __len__: its MinLen is measured against what it
stores, not against what it reports, so Annotated[list[int], MinLen(5)] and
the shape [int, int, int, int, int] narrow by the same count. A length two
parts of one schema disagreed about would not be a property of the value, and a
set defined by one would not be a set.
A constraint must also be one the base can answer. Asking an int for its length
raises, and a raise reads as a non-member, so Annotated[int, MinLen(1)] would
denote nothing at all while looking like a narrowing. A constraint no value of
the base can answer is therefore refused too; where some value can — a union with
a str branch, or a class that may define what the constraint asks for — it is a
narrowing and stands.
from typing import Annotated
import annotated_types as at
from valgebra import Validator
raised = False
try:
Validator(Annotated[int, at.MinLen(1)])
except NotImplementedError as error:
raised = True
assert "length" in str(error)
assert raised
# One branch can answer it, so this narrows rather than empties.
narrowed = Validator(Annotated[int | str, at.MinLen(1)])
assert narrowed.is_valid("a")
assert not narrowed.is_valid(5)
The constraint narrows the whole union, so a branch whose values cannot answer
it is left out rather than let through. That is the reading of Optional too:
Annotated[Optional[int], Ge(0)] refuses None, where pydantic applies the
bound to the int alone and admits it. Put the bound on the branch it narrows,
and the None beside it stays:
from typing import Annotated, Optional
import annotated_types as at
from valgebra import Validator
whole = Validator(Annotated[Optional[int], at.Ge(0)])
assert not whole.is_valid(None) # `None` has no order against `0`
branch = Validator(Optional[Annotated[int, at.Ge(0)]])
assert branch.is_valid(None)
assert branch.is_valid(3)
assert not branch.is_valid(-1)
Regex is valgebra's own marker (from valgebra import Regex), since
annotated-types defines none for strings. The match is anchored — the whole
string must match, like re.fullmatch — and runs natively in Rust with a
linear-time engine (no catastrophic backtracking), so unlike a Predicate it
stays on the fast path and never crosses into Python per value. An invalid
pattern is rejected when the validator is built, not at first use, and a pattern
is read on its own before it is anchored, so a)|(b is invalid rather than an
alternation of two halves. A pattern matches the text of a string, and a string
holding a lone surrogate -- which json.loads makes of an escaped "\ud800" --
has no such text, so it is outside every Regex refinement, and str is never
below a pattern over it however much text the pattern matches
(what the soundness argument assumes).
A compiled
re.Pattern works as metadata too:
import re
from typing import Annotated
from valgebra import Regex, Validator
oid = Validator(Annotated[str, Regex(r"[0-9a-f]{24}")])
assert oid.is_valid("0123456789abcdef01234567")
assert not oid.is_valid("0123456789abcdef0123456X") # not hex
assert not oid.is_valid("0123") # not the full 24 characters
assert Validator(Annotated[str, re.compile(r"\d+")]).is_valid("123")
# A compiled pattern's flags are part of the pattern and are carried over.
assert Validator(Annotated[str, re.compile("abc", re.IGNORECASE)]).is_valid("ABC")
A pattern is matched against text, so a bytes pattern is refused. So are
re.ASCII, re.LOCALE and re.DEBUG: the first two change what a character
class means in ways this engine spells differently (see the dialect section
below), and the third asks the other engine to report on itself.
re.IGNORECASE, re.MULTILINE, re.DOTALL and re.VERBOSE are written into
the pattern, which is how this engine spells them. Regex itself takes its
pattern as a str and raises TypeError for anything else when it is made.
It pickles and copies as the pattern it holds, so a schema carrying one can be
sent to another process and built there, as
a validator cannot be.
The dialect is Rust's, not re's¶
Running natively is what buys the linear-time guarantee, and it is also what makes the dialect the Rust engine's. The two languages are close but not equal, and a pattern both engines accept can denote different sets. Compiling successfully is therefore not a test of which language a pattern is in. The places the two engines part, which a ported pattern should be checked against — this is where they differ, not a complete audit of what a pattern denotes:
import re
from typing import Annotated
from valgebra import Regex, Validator
def admits(pattern: str, text: str) -> bool:
return Validator(Annotated[str, Regex(pattern)]).is_valid(text)
# POSIX bracket expressions. Python has no such class, so it reads a character
# class followed by a literal `]` and needs two characters.
assert admits(r"[[:alpha:]]", "a")
assert re.fullmatch(r"[[:alpha:]]", "a") is None
assert re.fullmatch(r"[[:alpha:]]", "a]") is not None
# Case folding. Both fold the ASCII pair; only Python folds the Turkish one.
assert admits("(?i)i", "I")
assert not admits("(?i)i", "\u0131") # dotless i
assert re.fullmatch("(?i)i", "\u0131") is not None
# Property escapes. `\p{...}` is a pattern only this engine accepts.
assert admits(r"\p{L}+", "ab")
# Class-set operators. `--`, `&&` and `~~` combine classes here and are literal
# characters to Python, so a class carrying one is refused rather than read.
raised = False
try:
admits(r"[\w~~\d]", "a")
except ValueError as err:
raised = True
assert "set symmetric difference" in str(err)
assert raised
# A nested set is refused for the same reason: a union here, four literals
# there.
raised = False
try:
admits(r"[a[bc]]", "a")
except ValueError as err:
raised = True
assert "nested set" in str(err)
assert raised
# Word-boundary escapes. `\<` is the start of a word here and the character `<`
# to Python, so it is refused too.
raised = False
try:
admits(r"\<b\>", "b")
except ValueError as err:
raised = True
assert "the start of a word" in str(err)
assert raised
Five more are quiet, and each is in a class or an anchor a ported pattern is likely to carry:
\s:recounts the separators\x1cto\x1fas whitespace, asstr.isspacedoes, and this engine follows Unicode'sWhite_Space, which does not.\wand\b: this engine counts a combining mark as a word character andredoes not, andrecounts a numeric character that is not a decimal digit, such as², and this engine does not.$: outsideMULTILINE,realso matches$before a final newline, soa$\nadmits"a\n"there and nothing here.\zis the end of the text in both, from Python 3.14.\Bon the empty string: it matches here, and inreonly from 3.14.- The Unicode version: this engine's classes, case folding and properties
are Unicode 16.0's, and
re's are the interpreter's own (unicodedata.unidata_version), 14.0 on Python 3.11 up to 17.0 on 3.15. A character one version assigns and the other does not is read apart:\wadmits U+1C89, a letter 16.0 adds, here and inrefrom 3.14, and(?i)folds U+A7CE and U+A7CF, a case pair 17.0 adds, inrefrom 3.15 and not here.
import re
import sys
import unicodedata
from typing import Annotated
from valgebra import Regex, Validator
def admits(pattern: str, text: str) -> bool:
return Validator(Annotated[str, Regex(pattern)]).is_valid(text)
assert not admits(r"\s", "\x1c")
assert re.fullmatch(r"\s", "\x1c") is not None
assert admits(r"\w", "\u0301") # a combining acute accent
assert re.fullmatch(r"\w", "\u0301") is None
assert not admits(r"\w", "²")
assert re.fullmatch(r"\w", "²") is not None
assert not admits("a$\n", "a\n")
assert re.fullmatch("a$\n", "a\n") is not None
assert admits(r"\B", "")
assert (re.fullmatch(r"\B", "") is not None) == (sys.version_info >= (3, 14))
unicode = tuple(int(part) for part in unicodedata.unidata_version.split("."))
assert admits(r"\w", "\u1c89")
assert (re.fullmatch(r"\w", "\u1c89") is not None) == (unicode >= (16,))
assert not admits("(?i)\ua7ce", "\ua7cf")
assert (re.fullmatch("(?i)\ua7ce", "\ua7cf") is not None) == (unicode >= (17,))
The third of the first four is the loud case: re.compile(r"\p{L}+") raises,
so a pattern that works here fails there and a reader finds out at once. The first two are the
quiet ones — both engines build the pattern and answer differently — and they
are the reason a library holding itself to re's decisions cannot adopt Regex
behind a fallback that triggers on compile failure.
The fourth is neither, because it is not accepted: a character class carrying
--, &&, ~~ or a nested [ raises a ValueError naming the operator, in
the words re uses where it warns. re answers the four three ways:
[\w--\d] raises, [\w&&\d] and [\w~~\d] warn that it may one day read them
as operators, and [a[bc]] compiles without a word. This engine reads all four
as operators now, so a pattern re accepts denotes two sets and nothing about
compiling it says which.
Refusing is the loud direction, and it is the one a reader can act on. Escape
the characters to mean them literally — [\w\~\~\d] — or write the classes
out. A POSIX class ([[:alpha:]]) is the one nested [ that is read rather
than refused, because it is the first divergence above and is documented there.
Verbose mode parts the two the same quiet way inside a class. re.VERBOSE
keeps a space and a # there as members, and this engine ignores the space and
starts a comment at the #. So under verbose mode -- re.VERBOSE, (?x), or
(?x:...) for one group -- a class carrying either raises a ValueError naming
it. Escape it as \ or \# to mean it, which both engines read as a member.
Three escapes part the two outside a class. \< and \> are the start and the
end of a word here and the characters < and > to re, so \<b\> would
admit "b" here and admits "<b>" there; \b{start} is a named word boundary here and, to
re, a boundary followed by the text {start}. Each raises a ValueError
naming what it asserts here. Write <, > or \{ for what re reads, or \b
for a word boundary in both.
Two quantifier spellings part them the same way. A + after a quantifier
makes it possessive to re and repeats the quantified item again here, so
a?+ refuses "aa" there and would admit it here. And this engine reads a
count with whitespace in its braces as the count -- a{1, 3} would admit
"aa" -- where re reads a brace that is not exactly a count as text, and
admits "a{1, 3}"; in verbose mode it skips a comment there too. Each raises
a ValueError naming it. This engine has no possessive quantifier: write
(?:a?)+ to repeat the quantified item. Write a count without whitespace, or
\{ for the text.
A quantifier repeats an atom in re, and nothing else. This engine reads one
on a quantifier and one on an assertion too: a{2}{3} as a{6}, a** as
(?:a*)*, and ^* as the empty match, where re refuses each, as a
"multiple repeat" and "nothing to repeat". Each raises a ValueError in those
words, so a pattern means nothing here that re cannot check. A ? right
after a quantifier makes it lazy in both, and only right after: in verbose
mode, a* ? is a quantifier on a quantifier to re. Write a group,
(?:a{2}){3}, to repeat a repetition, and leave an assertion unquantified.
A pattern Python spells and this engine does not is refused, which is the
loud direction and the one to prefer. A lookaround ((?=...), (?<=...)), a
backreference (\1, (?P=name)), an atomic group ((?>...)), a conditional
((?(1)...|...)), an inline comment ((?#...)), a named character
(\N{BULLET}), the ASCII flag (?a), the \Z anchor and the open-ended
{,n} repetition all raise a ValueError naming the parse error, rather than
being read as something else.
Agreement with re is not ASCII¶
Checking a pattern against the list above tells you the two engines agree. It
does not tell you the pattern denotes what you meant, and the character classes
are where that bites: \d, \w and \s are Unicode-aware here, exactly as
they are in re. The two engines agree, and a reader who wanted digits gets
every decimal digit Unicode defines.
import re
from typing import Annotated
from valgebra import Regex, Validator
year = Validator(Annotated[str, Regex(r"\d{4}")])
assert year.is_valid("٢٠٢٦") # Arabic-Indic digits are decimal digits
assert re.fullmatch(r"\d{4}", "٢٠٢٦") is not None # `re` agrees
assert Validator(Annotated[str, Regex(r"\w")]).is_valid("é")
assert Validator(Annotated[str, Regex(r"\s")]).is_valid("\u00a0") # no-break space
Write the ASCII set when you mean the ASCII set. [0-9] is the portable
spelling; (?-u:\d) is the same thing with the Unicode flag turned off for that
group, and is this engine's syntax rather than re's:
from typing import Annotated
from valgebra import Regex, Validator
assert not Validator(Annotated[str, Regex(r"[0-9]{4}")]).is_valid("٢٠٢٦")
assert not Validator(Annotated[str, Regex(r"(?-u:\d){4}")]).is_valid("٢٠٢٦")
assert Validator(Annotated[str, Regex(r"[0-9]{4}")]).is_valid("2026")
This is the shape that reaches production: a timestamp, an identifier or a
version pattern written with \d admits strings the parser downstream rejects,
and every engine involved agrees the pattern is fine.
A bound against nan is rejected with a ValueError, and so is
MultipleOf(nan): every comparison with nan is false, so such a bound admits
no value at all. A Decimal not-a-number is rejected the same way, quiet or
signaling: every comparison and remainder with Decimal("sNaN") raises
InvalidOperation, which the walk reads as a refusal. That is the empty set written as a bound, which no caller
means, and dropping the marker instead would admit every value of the base. A
complex bound is rejected for the same reason, whatever its imaginary part:
Python orders no value against a complex number, and 1 >= 0j raises. A
bound that is empty because the order says so — Gt(inf) on a float — is kept,
because emptiness is then an answer rather than the absence of one.
A float under a Decimal bound is compared as the Decimal it equals,
Decimal.from_float, which is exact. Python compares a float with a Decimal
exactly too, but through the thread's decimal context, and a FloatOperation
trap there makes the comparison raise: read through it, 0.75 would be outside
Ge(Decimal("0.25")) in a thread that sets the trap and inside it in one that
does not. The set is the one the bound writes, whatever the context. A subclass
of float keeps its own comparison, which may be code of its own, so it meets
the bound as Python compares it, context included:
from decimal import Decimal, FloatOperation, localcontext
from typing import Annotated
import annotated_types as at
from valgebra import Validator
at_least = Validator(Annotated[float, at.Ge(Decimal("0.25"))])
with localcontext() as context:
context.traps[FloatOperation] = True
assert at_least.is_valid(0.75)
assert not at_least.is_valid(0.1)
MultipleOf(n) requires a nonzero number: no value is a multiple of zero,
so MultipleOf(0) is an unsatisfiable constraint, and a step that is not a
number is unsatisfiable for the same reason one step further on. The constraint
is value % n == 0, and a remainder that is not a number equals no zero —
timedelta(6) % timedelta(2) is timedelta(0), which is not the integer 0 —
so such a step names a schema no value belongs to. Both are rejected with a
ValueError when the validator is built, rather than rejecting every value at
check time, and so is a complex step, since Python takes no remainder against a
complex number. int, float and Fraction steps divide every number as they
read. A Decimal step divides an integer and not a float, because % refuses
to mix a float and a Decimal: Annotated[float, MultipleOf(Decimal("0.5"))]
is rejected with a NotImplementedError, and over int | float the step
narrows the integers.
typing caches Annotated by equality, and MultipleOf(2 + 0j) equals
MultipleOf(2) as MultipleOf(Decimal("0.5")) equals MultipleOf(0.5). In a
process that built the complex or Decimal spelling first, the real one arrives
as it and is rejected with it; the message says so. A whole float step equals
the integer one too, and is not the same set over the integers: % reads an
int against a float as the float nearest it, so MultipleOf(2.0) admits
2**53 + 1, whose nearest float is 2**53, and MultipleOf(2) does not. In a
process that wrote MultipleOf(2.0) first, Annotated[int, MultipleOf(2)] is
that marker and admits 2**53 + 1 as well, with nothing to say so, since both
are valid. Write a step over the integers as an int. The step is read as the
number it is, not by its value: reading 2.0 as 2 would refuse a value the
marker's own % admits.
A float step is exact. value % n == 0 is Python's own %, with no
tolerance added, and a binary float holds no 0.1: 0.3 % 0.1 is
0.09999999999999998. So Annotated[float, MultipleOf(0.1)] admits 0.1 and
0.2 and refuses 0.3 and 1.0. A Fraction step over a float is the same
%, which turns the fraction into the float nearest it: 3.0 % Fraction(1, 3)
is 1.6653345369377348e-16, so a step of a third admits 1 / 3 and refuses
3.0. A step meant in decimal places wants a decimal base, whose % is exact
in those places:
from decimal import Decimal
from fractions import Fraction
from typing import Annotated
import annotated_types as at
from valgebra import Validator
tenths = Validator(Annotated[float, at.MultipleOf(0.1)])
assert tenths.is_valid(0.2)
assert not tenths.is_valid(0.3) # 0.3 % 0.1 is not 0.0
thirds = Validator(Annotated[float, at.MultipleOf(Fraction(1, 3))])
assert thirds.is_valid(1 / 3)
assert not thirds.is_valid(3.0) # 3.0 % Fraction(1, 3) is not 0.0
decimal_tenths = Validator(Annotated[Decimal, at.MultipleOf(Decimal("0.1"))])
assert decimal_tenths.is_valid(Decimal("0.3"))
assert not decimal_tenths.is_valid(Decimal("0.15"))
The compound markers Interval and Len expand to the bounds they carry, so
Interval(ge=0, le=10) contributes Ge(0) and Le(10), and Len(2, 4)
contributes MinLen(2) and MaxLen(4). A Len whose minimum is below zero is
refused as a MinLen of it is, although annotated_types unpacks it to no
lower bound at all. A length is a whole number, and Python compares one with a
float, a Fraction or a Decimal exactly, so a length bound equal to a whole
number is that length: MaxLen(2.0), MaxLen(Fraction(2)) and MaxLen(True)
are MaxLen(2) and MaxLen(1), which typing's cache would hand them anyway,
and a bound equal to no whole length, MaxLen(2.5), is refused. So is a number
of any other type, a subclass of those three included, whose int() and ==
are its own code rather than the arithmetic len is compared by. So is a length
past sys.maxsize: CPython counts a length in a Py_ssize_t and len() raises
past it, so no value is longer, and MinLen(2**63) would be the empty set
written as a bound. They are read through the grouping
protocol annotated_types documents — a marker that stands for several
constraints answers for them — so a marker of your own written that way is
read the same way:
from typing import Annotated
import annotated_types as at
from valgebra import Validator
assert Validator(Annotated[int, at.Interval(ge=0, le=10)]).is_valid(5)
assert not Validator(Annotated[int, at.Interval(ge=0, le=10)]).is_valid(11)
assert Validator(Annotated[str, at.Len(2, 4)]).is_valid("abc")
assert not Validator(Annotated[str, at.Len(2, 4)]).is_valid("a")
class Percent(at.GroupedMetadata):
"""A marker of your own, standing for the two bounds it yields."""
def __iter__(self):
yield at.Ge(0)
yield at.Le(100)
assert Validator(Annotated[int, Percent()]).is_valid(50)
assert not Validator(Annotated[int, Percent()]).is_valid(101)
assert Validator(Annotated[int, at.MultipleOf(3)]).is_valid(9)
assert not Validator(Annotated[int, at.MultipleOf(3)]).is_valid(5)
Predicates: the slow path¶
A Predicate runs an arbitrary Python callable. It is the one refinement
constraint that leaves Rust for a caller's own code — literals, instance and
attribute checks, and comparison bounds also compare against Python objects, but
against fixed operators, not arbitrary callables — so it is a documented slow
path, never a silent fallback. is_valid runs it once per value it reaches,
and stops at the first value that fails. validate explains as it decides, in
one walk, except at a union: a refused union walks each branch it has to once
to decide whether any branch admits the value and again to explain it, so a
predicate inside runs twice there, and a union left with one branch to walk
explains it once. A dict schema -- a record, a TypedDict, a mapping -- that
refuses a value is read twice: a deciding pass, then an explaining pass. A
record -- closed, a TypedDict, or open under a catch-all clause that reads
its values -- is read by its keys, and resumes at the field the first pass
stopped at, so a predicate on or under that field runs a second time, and past
its fields where an undeclared key refused it. A mapping is read by a scan, and
its explaining pass starts at the first entry: a predicate on every entry up to
the refused one runs a second time. A predicate on the clause an undeclared key
falls under runs once to find whether the clause covers the entry and again to
report it (docs/dev/walk.md). A dict schema
that is a branch of a union is read the second time only where no branch admits
the value. Without
fail_fast, validate also reads past the first failure, running the
predicate on values is_valid never reached. A
predicate is user code, and every occurrence the walk reaches is a call --
two equal occurrences in one meet or union are one occurrence, deduplicated
when the schema is built, so Annotated[int, f, f] calls f once a value. The
walk keeps no cache over it, because a cache would change what a predicate that
does not answer from its value alone observes, and the same reading is what
keeps A & ~A from folding when A carries one. A predicate that must run once
per value memoizes on its own side. Use it for checks the markers cannot
express:
from typing import Annotated
import annotated_types as at
from valgebra import Validator
even = Validator(Annotated[int, at.Predicate(lambda x: x % 2 == 0)])
assert even.is_valid(4)
assert not even.is_valid(3)
A predicate that raises is reported distinctly, as predicate_error rather
than an ordinary failure, so a buggy predicate is not mistaken for a rejected
value.
A bare callable is a predicate too, without the wrapper:
from typing import Annotated
from valgebra import Validator
positive = Validator(Annotated[int, lambda value: value > 0])
assert positive.is_valid(1)
assert not positive.is_valid(-1)
Predicate is the portable spelling — pydantic reads it — so prefer it in an
annotation other tools also consume. The bare form is
valgebra's own convenience, and it excludes a class, which is never read as a
marker: calling one constructs rather than asks.
A bare callable is metadata only¶
The bare form is read only in Annotated metadata position, and reading it
there at all is valgebra's own convenience: the libraries that share this
metadata channel require a wrapper — pydantic AfterValidator, beartype
Is[...], annotated_types Predicate — and msgspec reads no callable at all,
since its Meta carries none. The typing spec
leaves each consumer to say what its own metadata means, so this arm reaches
exactly as far as the metadata position and no further.
A schema language that reads a top-level callable as a predicate is expressing a
different rule for a position Annotated metadata does not cover; valgebra's
rule for that position is the one below.
A typing form in metadata is not a predicate either, although every one is
callable: calling list[int], a NewType or an Annotated alias with a value
builds something rather than answering whether the value belongs. A typing form
is metadata this frontend does not recognize and is ignored, as a class is. An
Annotated alias carrying a marker this page reads -- at.IsFinite, or an
alias of your own over at.Gt(0), written as an assignment or with the type
statement -- is refused rather than ignored: it carries its markers for the
type it annotates, and in metadata it annotates nothing, so it was written
where its type was meant:
import math
from typing import Annotated
import annotated_types as at
import pytest
from valgebra import Validator
with pytest.raises(NotImplementedError, match="is an Annotated alias"):
Validator(Annotated[float, at.IsFinite[float]])
finite = Validator(at.IsFinite[float]) # the alias, as the type
assert finite.is_valid(0.0)
assert not finite.is_valid(math.inf)
PEP 702's deprecated is ignored the same way. typing_extensions.deprecated
is that module's own below 3.13 and the standard library's warnings.deprecated
from it, and on every release Annotated[int, deprecated("...")] is int.
A decorator function is called. What this frontend tells apart from a
predicate is an object whose class is a typing form or the standard library's
deprecated. A function is a function, and nothing about one says whether it
asks or decorates: math.isfinite and typing.final are both the standard
library's. So a decorator in metadata is called with the value.
dataclasses.dataclass, abc.abstractmethod and typing.no_type_check raise
for a value that is neither a class nor a function, and refuse every value;
typing.final and typing.override return what they are given, and refuse
0:
import typing
from typing import Annotated
from typing_extensions import deprecated
from valgebra import Validator
assert Validator(Annotated[int, deprecated("use the account id")]) == Validator(int)
final = Validator(Annotated[int, typing.final]) # called, as a predicate is
assert final.is_valid(5)
assert not final.is_valid(0)
Passed as a schema on its own, a callable is refused. Read as the
fallback literal
reads an object, it would denote the one function object, and
intersection(Record, kind_is_known) would be a schema admitting nothing that
nothing reports: is_empty answering False is "no, or not yet proven" (see
the decidability boundary), and the meet of a
record with a function object's literal is one it does not decide. The refusal names the two
spellings that were meant, the refinement as Annotated and the constant as
Literal:
from typing import Annotated, TypedDict
import pytest
from valgebra import Validator, intersection
class Record(TypedDict):
kind: str
def kind_is_known(value):
return value["kind"] in {"a", "b"}
with pytest.raises(NotImplementedError, match="a callable is not a schema"):
intersection(Record, kind_is_known)
refined = Validator(Annotated[Record, kind_is_known]) # the refinement
assert refined.is_valid({"kind": "a"})
assert not refined.is_valid({"kind": "z"})
annotated_types.Not wraps a predicate and denotes the values it rejects:
from typing import Annotated
import annotated_types as at
from valgebra import Validator
odd = Validator(Annotated[int, at.Not(lambda value: value % 2 == 0)])
assert odd.is_valid(3)
assert not odd.is_valid(2)
A marker that is itself callable is called, which is what applies Not's
negation and what keeps a functools.partial's bound arguments. Only a marker
that is not callable is taken apart by its .func, which is the shape
Predicate has.
On classes¶
Refinements declared on a TypedDict, dataclass, or NamedTuple field are
enforced — the constraint travels with the field:
from typing import Annotated, TypedDict
import annotated_types as at
from valgebra import Validator
class Account(TypedDict):
balance: Annotated[int, at.Ge(0)]
assert Validator(Account).is_valid({"balance": 100})
assert not Validator(Account).is_valid({"balance": -1})
A validator narrows by its set¶
A compiled validator in the metadata is a set, and Annotated[T, v] denotes the
values of T that v admits: the meet of T and v, the set
intersection(T, v) builds. Beside other markers it is met with the refined
base, and a grouped marker that yields one is read the same way. This is the
spelling for a class field whose set typing cannot express: a static checker
reads the field as T, and the validator refuses what v refuses, in the Rust
walk and not through a Python callback.
from dataclasses import dataclass
from typing import Annotated
from valgebra import Validator, complement, intersection
non_bool_int = intersection(int, complement(bool))
@dataclass
class Row:
count: Annotated[int, non_bool_int]
rows = Validator(Row)
assert rows.is_valid(Row(3))
assert not rows.is_valid(Row(True)) # a bool is an int, and not a member
assert Validator(Annotated[int, non_bool_int]) == intersection(int, non_bool_int)
Unrecognized markers¶
Per the typing spec, metadata valgebra does not recognize as a constraint is
ignored — so non-constraint Annotated metadata, such as a documentation string,
is harmless and carries no membership meaning. The carve-out is the
annotated_types vocabulary itself: a marker from there was written to narrow
this schema, so one valgebra does not check is refused rather than ignored. A
validator is not unrecognized metadata at all: it narrows by its set, above.
A constraint is read off the vocabulary's markers, Regex and a compiled
re.Pattern, and off nothing else, whatever its attributes are called.
msgspec's Meta, pydantic's Field and AfterValidator, and a class of your
own may carry a ge or a pattern and mean what their own library means by
it — msgspec and pydantic search with a pattern where valgebra matches it
whole — so each is ignored, as pydantic ignores msgspec's metadata and msgspec
pydantic's. A library joins the vocabulary the way its README says, as a
GroupedMetadata yielding the vocabulary's markers, and those are read; a
class derived from one of the vocabulary's is read as the class it derives
from. Its bounds are read as its attributes are, through a __getattribute__
of its own where it has one, and once a class: which names a marker class
answers is read the first time a marker of it is built into a validator and
kept for the process, so a class given a bound after that keeps the reading
it was first given.
from typing import Annotated
from valgebra import Validator
class Minimum:
"""Another library's marker, carrying a name the vocabulary also uses."""
def __init__(self, ge: int) -> None:
self.ge = ge
assert Validator(Annotated[int, Minimum(0)]) == Validator(int)
assert Validator(Annotated[int, Minimum(0)]).is_valid(-1)
A class is never read as a marker. A marker carries its values on an
instance — Ge(0) holds ge = 0 — so the class itself holds no value to read
and calling it constructs rather than asks. A marker class written where its
instance was meant is refused with the spelling that was meant, since ignoring
it would leave the schema its base; any other class, such as a unit or an
enumeration, is ignored: