Schema language¶
A schema denotes a set of Python values. This page lists every form valgebra
reads and the set it denotes. The primary notation is standard typing; compact
native forms and the combinators are alternatives for the same sets. A form
spelled from typing_extensions reads as its typing spelling does, on every
supported release, including those where the two are different objects.
A form this page does not list¶
The list is exhaustive, so a typing form absent from it is one valgebra does not
read. It raises NotImplementedError when the validator is built, not when a
value arrives — a schema that cannot be read never becomes one that quietly
admits everything.
from collections.abc import Mapping
from valgebra import Validator
raised = False
try:
Validator(Mapping[str, int])
except NotImplementedError as error:
raised = True
assert "unsupported typing form" in str(error)
assert raised
The abstract-collection generics are the ones a reader is most likely to
reach for: Mapping[K, V], Sequence[T], Set[T] and Iterable[T], all
four from collections.abc, and the subscripted concrete classes such as
deque[T] and OrderedDict[K, V]. typing.Set[T] is not among them: it is
the builtin set under another name, and builds as set[T] does. None of the
others builds, and not for one reason:
Mapping[K, V]andSequence[T]are refused. The record and mapping node denotes dicts, and giving it a carrier so these build is a change the algebra does not admit, recorded with the test it fails (the refused changes indocs/dev/schema-ir.md, in the repository).Iterable[T]cannot be checked: reading its elements consumes an iterator, and a check that consumed one would change the value it was asked about.Set[T],deque[T]andOrderedDict[K, V]are unbuilt: nothing refuses them and nothing builds them. Writing one costs aNotImplementedErrorat build, and the paragraphs below say what to write instead.
type[T] is refused as well, and is a different question: it constrains a value
that is itself a class, where every form above constrains a container's
contents.
Where the value really is a dict or a list, the builtin form denotes the set
you want: dict[K, V] for Mapping[K, V], list[T] for Sequence[T]. Where it
is not — a UserDict, a deque — no form here denotes it, and a
predicate refinement over the class is the available
expression.
Scalars¶
| Schema | Denotes |
|---|---|
int |
every int instance |
float |
every float instance |
str |
every str instance |
bytes |
every bytes instance |
bool |
{True, False} |
None |
{None} |
The set relationships follow Python's own, exactly:
from valgebra import Validator
# bool is a subclass of int, so True and False are ints
assert Validator(int).is_valid(True)
# int does not subclass float, so an int is not a float
assert not Validator(float).is_valid(1)
assert Validator(float).is_valid(1.0)
assert Validator(float | int).is_valid(1) # what a checker accepts there
float departs from the typing spec here. The spec gives a checker a
shortcut -- "when an argument is annotated as having type float, an argument of
type int is acceptable" -- so a checker reads a float annotation as float |
int. valgebra checks the object a caller holds, and an int is not a float
instance: write float | int for the set a checker accepts there. complex
departs the same way: the same shortcut reads it as complex | float | int, and
Validator(complex) admits a complex instance alone, refusing 1.0 and 1,
so write complex | float | int for that set.
Any versus object¶
Both are the top of the lattice (anything): every value. They build the
same schema, and the only difference is the one repr gives back: Any prints
as Any, and object as anything (the
algebra).
from typing import Any
from valgebra import Validator
assert Validator(object).is_valid(["anything", 1, None])
assert Validator(Any).is_valid(object())
assert Validator(Any) == Validator(object)
assert repr(Validator(Any)) == "Any"
Never and NoReturn¶
Both are the bottom of the lattice (nothing): the empty set, which no
value belongs to. They build the schema nothing is, and print as nothing
(lattice bounds). Both are in typing on every
supported release.
Collections¶
| Schema | Denotes |
|---|---|
list[T] |
lists whose every element is in T |
set[T] |
sets whose every element is in T |
frozenset[T] |
frozensets whose every element is in T |
dict[K, V] |
dicts whose keys are in K and values in V |
tuple[A, B] |
length-2 tuples with A then B |
tuple[T, ...] |
tuples of any length, every element in T |
tuple[A, *tuple[B, ...]] |
a fixed prefix A, then zero or more B (see below) |
tuple[A, B, ...] |
the same, in a shorter spelling a static checker refuses |
typing.List, typing.Tuple, typing.Dict, typing.Set, typing.FrozenSet |
the class each aliases, bare or parametrized |
typing.Deque, DefaultDict, OrderedDict, Counter, ChainMap, Type, and every other typing alias of a class -- typing.Sequence, typing.Mapping, typing.Text among them |
the class each aliases when bare; parametrized, refused, as deque[int] is, but Callable[...], which reads as the class (below) |
from valgebra import Validator
assert Validator(list[int]).is_valid([1, 2, 3])
assert Validator(dict[str, int]).is_valid({"a": 1})
assert Validator(tuple[int, str]).is_valid((1, "a"))
assert Validator(tuple[int, ...]).is_valid((1, 2, 3))
assert Validator(tuple[str, *tuple[int, ...]]).is_valid(("x", 1, 2))
Native forms¶
A native form exists only where standard typing cannot spell the set: the
list literal carries the sequence shapes typing has no syntax for. Everything a
typing annotation already expresses is written that way — set[T], not {T};
tuple[A, B], not (A, B); both literals are rejected with a message pointing
to the typing spelling.
| Native form | Denotes |
|---|---|
[T] |
list[T] — a homogeneous list (the single-element idiom) |
[T, ...] |
list[T] — homogeneous, written with the tail marker |
[A, B] |
a fixed-length list, matched positionally (list[A, B] is illegal typing) |
[A, B, ...] |
a fixed prefix, then a repeated tail (see below) |
{K: V} |
dict[K, V] |
{"key": T, "key2?": T} |
a record (see below) |
any other constant c, not a tuple, a set or a callable |
Literal[c] |
from valgebra import Validator
assert Validator([int]).is_valid([1, 2]) # homogeneous list[int]
assert Validator([int, str]).is_valid([1, "a"]) # fixed-length list
assert not Validator([int, str]).is_valid([1]) # wrong length
assert Validator({str: int}).is_valid({"a": 1}) # dict[str, int]
assert Validator("active").is_valid("active") # the literal "active"
A fixed-length list is matched positionally: element i must satisfy the
ith schema and the length must match. typing cannot spell it (list[A, B] is
illegal), which is the reason the list literal carries the shape; a fixed-length
tuple is the typing tuple[A, B], and the container is part of the type, so a
list is never a member of the tuple form and vice versa.
Prefix and repeated tail¶
A sequence schema is, in general, a regular expression over element types: a
fixed positional prefix followed by an optional repeated tail. A trailing ...
repeats the element just before it, so [T, ...] (any number of T) is the
prefix-free case. The same shape is available for tuples, spelled
tuple[A, *tuple[B, ...]]; the container is part of the type, so a tuple is never a member of the list form
and vice versa.
| Form | Denotes |
|---|---|
[A, B, ...] |
a list: an A, then zero or more B |
[T, T, ...] |
a non-empty list of T (at least one) |
tuple[A, *tuple[B, ...]] |
a tuple: an A, then zero or more B |
tuple[A, B, ...] |
the same tuple, in the shorter spelling a static checker refuses |
An unpacked variadic tuple says the prefix-and-tail shape the way PEP 646
spells it, and Unpack[tuple[B, ...]] is the same thing written out. An unpacked
fixed tuple splices its elements in, so tuple[A, *tuple[B, C]] is
tuple[A, B, C], and what it splices is read as if written in its place: a
star inside it unpacks too, so tuple[A, *tuple[B, *tuple[C, ...]]] is
tuple[A, B, *tuple[C, ...]]. What a sequence cannot carry is an element
after the repeating tail — tuple[*tuple[int, ...], str] names a set this
algebra does not spell — so that form is rejected rather than read as something
else. So is a ... after a splice, tuple[*tuple[A, B], ...], because ...
repeats the one element written before it and a splice is not one element, and
an unpacked tuple anywhere but a tuple's arguments, such as list[*tuple[int]].
A type alias stands for the form it names, so *Tail under type Tail =
tuple[int, ...] is the tuple's unpack, an applied generic alias's with its
arguments in place; an alias that unpacks itself, type Rec = tuple[int, *Rec],
holds itself and has no finite length, and is refused.
from valgebra import Validator
prefixed = Validator([str, int, ...]) # a str, then zero or more ints
assert prefixed.is_valid(["x"])
assert prefixed.is_valid(["x", 1, 2])
assert not prefixed.is_valid([1]) # the prefix must be a str
non_empty = Validator([int, int, ...]) # at least one int
assert non_empty.is_valid([1])
assert not non_empty.is_valid([])
tup = Validator(tuple[str, *tuple[int, ...]]) # the same shape, as a tuple
assert tup.is_valid(("x", 1, 2))
assert not tup.is_valid(["x", 1, 2]) # a list is not a member of the tuple form
short = Validator(tuple[str, int, ...]) # the shorter spelling
assert short == tup
assert repr(tup) == "tuple[str, int, ...]"
Literals¶
Literal[...] denotes the values equal to its constant and of the same type:
a value is a member iff it has the same type as the literal and is equal to
it. For an int, a str or an enum member that is the constant alone; for a
float zero it is both zeros, so Literal[0.0] admits -0.0. The same-type
rule keeps the constants 1, True and 1.0 distinct, even though Python's
== conflates them:
from typing import Literal
from valgebra import Validator
assert Validator(Literal[1]).is_valid(1)
assert not Validator(Literal[1]).is_valid(True)
assert not Validator(Literal[1]).is_valid(1.0)
Literal takes the values the typing spec lists -- an int, str, bytes or
bool value, an enum member, None -- and a static checker refuses anything
else there. Any other constant, a float or an instance, is written bare:
Validator(1.0) is the schema Literal[1.0] names, and a checker accepts it
(static checkers). A tuple is one exception: a bare
tuple, a named tuple's value included, is refused as a tuple literal, which
(int, str) would read as a shape and a constant alike, and the constant
(1, 2) is written tuple[Literal[1], Literal[2]]. A set and a frozen set are
refused the same way, as the container literal each would also read as. A
callable is the last:
written bare it is refused (below), and
the constant is Literal[fn], which reads at run time and which a static
checker refuses.
The constant nan denotes nothing. Membership is equality, and a nan is
equal to nothing at all — itself included — so the set has no member. That is not a
gap in the check: the set is decided empty, and float admits nan. Write
float and a predicate if what you mean is "the not-a-number value".
from valgebra import Validator
nan = float("nan")
assert Validator(nan).is_empty() # equality admits nothing
assert not Validator(nan).is_valid(nan)
assert Validator(float).is_valid(nan) # the kind still holds it
A string inside a generic is a forward reference, and is refused¶
The fallback reads a bare value as a literal, and for a string that reading
stops at the argument of a typing form; list[5] is list[Literal[5]].
list["Account"] is a forward reference to a type
named Account, which the typing spec resolves against the namespace the
annotation was written in — a namespace valgebra does not have, because it is
handed the runtime object rather than the source. Reading the string as a literal
instead would build a list of the word "Account", a schema that refuses what
the annotation admits, so the position is refused:
from valgebra import Validator
raised = False
try:
Validator(list["Account"])
except NotImplementedError as error:
raised = True
assert "forward reference" in str(error)
assert raised
# Say the type, or say the value.
assert Validator(list[int]).is_valid([1])
assert Validator(["active"]).is_valid(["active"]) # a list of that literal
A class's own annotations are a different matter: Validator(SomeClass) resolves
their strings for you (see classes), and a TypedDict's
extra_items given as a string alike, in the module that defines the class:
PEP 728 takes a type expression there. A type parameter's default is
not one of them: PEP 696 makes it a type expression, and the runtime keeps a
string default as written, so a field that is the parameter alone is refused
as a forward reference where its default is a string, as the same string
written as the argument is.
Anything unrecognized is a literal¶
The literal form is also the fallback: an object the frontend does not read
as one of the forms above becomes Literal[that object], so Validator(x)
denotes {x} for any x valgebra has no other reading for. That is what makes
Validator("active") mean the string rather than an error, and it applies to a
module or an instance just the same.
A string at the top is a departure from the typing spec. The spec's grammar
reads a string where a type goes as a string_annotation -- the annotation
written as text, to be parsed and evaluated -- and a checker reads
x: "Account" as the class Account. A schema is built at run time from a
value in hand, so valgebra reads a string at the top as that value, which is
the one reading a caller writing Validator("active") means. Inside a typing
form the string stays the forward reference the spec makes it, and is refused
(above).
A callable that is not a class is refused instead. Written where a schema
goes it is a predicate one position too far out, or a constant without its
Literal, and read as the fallback reads an object it would be the set holding
the one function object, which no value a caller checks is: a meet with it
admits nothing, and nothing says so. The typing spec's grammar for a type has
no production for a function either. Each meaning has its spelling, and the
refusal names both: as Annotated metadata the callable is a predicate
narrowing a base (see
refinements), and in a
Literal it is the object itself. A class is callable too, and is read as the
class it is.
from typing import Annotated, Literal
import pytest
from valgebra import Validator
def positive(value):
return value > 0
with pytest.raises(NotImplementedError, match="a callable is not a schema"):
Validator(positive)
checked = Validator(Annotated[int, positive]) # the values it accepts
assert checked.is_valid(1)
assert not checked.is_valid(-1)
itself = Validator(Literal[positive]) # the function object
assert itself.is_valid(positive)
assert not itself.is_valid(1)
The forms that are refused rather than read as a literal¶
The fallback is for an object with no other reading. A typing form the spec does give a meaning to is a different case: read as a literal it would denote the form object itself, which no value a caller has belongs to, so the schema would refuse everything without saying why. Each of these is refused at build with a message instead:
| Form | Why it has no set |
|---|---|
Self |
names the enclosing class, which a schema is built without |
LiteralString |
a property of where a string came from, which a value does not carry |
TypeVar, ParamSpec, TypeVarTuple |
a variable stands for a type and is not one |
Final, ClassVar |
a declaration about a name, not about a value; a dataclass field's Final[T] is read as T (classes) |
Unpack[X] |
binds element types into a tuple[...], so it has no meaning alone, and nor has *tuple[...] outside one |
a user Generic[T] class read by isinstance alone, applied |
the parameter is erased at runtime and the class declares no field it reaches, so the type argument narrows nothing |
a class generic in a TypeVarTuple or a ParamSpec, applied |
the parameter stands for a list of types or a signature, which no form here binds a field to |
Required, NotRequired or ReadOnly as a class's type argument or a type parameter's default |
a qualifier qualifies a TypedDict item where it is declared and nowhere else (PEP 655, PEP 705), and the runtime keeps the key it reaches as declared |
Required or NotRequired as a TypedDict's extra_items |
PEP 728 makes it an error: an extra key is never required, so the qualifier says nothing true of it; ReadOnly is read through there |
bare Protocol, and a generic Protocol[T] |
the base a protocol is declared from declares no member, and a protocol over a type parameter names one set per argument |
a Protocol member declared ClassVar or Final |
the qualifier says where the value lives rather than what it is |
| a set or frozen set literal | {int} and frozenset({int}) name containers, which are set[T] and frozenset[T] |
| a tuple literal | (A, B) is tuple[A, B]; the list literal [A, B] is the fixed-length list |
| a frozen dict literal | frozendict(a=int) names a record of frozen dicts, which no node reads: collections.abc.Mapping or instance_of(frozendict) admits a frozen dict whole, and {"a": int} admits dicts alone (Python 3.15+) |
Validator[T] |
annotates a validator for a static checker; the schema, or a compiled Validator, is what builds |
bare TypedDict |
the base a TypedDict class is declared from; the class is what has a set |
bare NamedTuple |
names every named tuple class, and at runtime it is a function isinstance cannot ask |
bare Generic |
the base a generic class is declared from; the class deriving from it is what has a set |
Required[NotRequired[X]] and NotRequired[Required[X]] |
the typing spec makes the pair an error, and the two qualifiers name two records: the runtime's key sets read the key as required and pydantic as optional; refused as a TypedDict's item, and outside a record unwrapped as each qualifier is |
A qualifier is the exception that is read rather than refused:
Required[X], NotRequired[X] and ReadOnly[X] survive hint resolution
because field metadata is kept, so the frontend unwraps them and compiles the
type they qualify. Written outside a record they are unwrapped just the same,
which makes Validator(Required[int]) the same schema as Validator(int).
Besides the places the table above names, a dict literal's field refuses
Required and NotRequired, because the literal says whether a key is
required by its ? alone and the qualifier would say it a second time
(records).
Unions and Optional¶
X | Y and Optional[X] denote the union of the member sets:
from typing import Optional
from valgebra import Validator
assert Validator(int | str).is_valid("x")
assert Validator(Optional[int]).is_valid(None)
Records¶
A record is a dict with named fields: a required field's key must be present
with a matching value, and a closed record admits no key outside the declared
names. The typing spelling is a TypedDict, open as the typing spec defines one
or closed with closed=True (PEP 728; in typing from 3.15 and in
typing_extensions before). A static checker reads it as its own type, so
is_valid narrows to it (static checkers). close()
does not close one: its openness is the clause str: anything its reading
carries, a region that clause claims, and close shuts only the keys no clause
claims (below). The closed record is closed=True.
from typing import NotRequired
from typing_extensions import TypedDict
from valgebra import Validator
class User(TypedDict, closed=True):
name: str
age: NotRequired[int]
user = Validator(User)
assert user.is_valid({"name": "Ada"}) # optional key absent
assert user.is_valid({"name": "Ada", "age": 36})
assert not user.is_valid({"name": "Ada", "x": 1}) # closed: no extra keys
A dict literal with all-string keys is the same record written as a shape:
closed by default, with a trailing ? on a key name marking it optional.
from valgebra import Validator
user = Validator({"name": str, "age?": int})
assert user.is_valid({"name": "Ada"})
assert not user.is_valid({"name": "Ada", "x": 1})
A key name that ends in a question mark¶
The suffix is stripped once, so a name that ends in ? is written with one
more: {"page??": int} is the optional key page?. The required key
page? has no dict-literal spelling, and a TypedDict spells it
(below).
from valgebra import Validator
assert Validator({"page??": int}).is_valid({"page?": 1})
assert Validator({"page??": int}).is_valid({}) # still optional
A field is required by its key¶
A field is named once. {"a": int, "a?": str} is two dict keys and one field
name, which asks the key to be required and optional at once, so it is refused
rather than built into a record admitting nothing:
from valgebra import Validator
try:
Validator({"a": int, "a?": str})
raise AssertionError("expected a rejection")
except ValueError as error:
assert "declared twice" in str(error)
The ? is the literal's one marker for it. A TypedDict qualifier on a field
-- {"a": NotRequired[int]}, or Required on a key with a ? -- is refused,
naming the key's spelling, because unwrapped to its type it would build a
record that requires a where the qualifier says the opposite. ReadOnly
says nothing about presence and is read as the type it wraps.
A required key ending in ? has no dict-literal spelling: every trailing
? is the marker, so there is no string that reads back as one. Write it as a
TypedDict through the functional syntax, where the key is taken literally and
required-ness comes from the class:
from typing import TypedDict
from valgebra import Validator
Query = TypedDict("Query", {"page?": int})
assert Validator(Query).is_valid({"page?": 1})
assert not Validator(Query).is_valid({}) # required
It is one of the forms whose repr does not read back as Python: it renders as
{<required: 'page?'>: int, str: anything}, the mark that says the key named
page? is required, where {'page?': int} would read back as the optional key
page (boundaries, and stable repr below).
Open the record with open (undeclared keys admitted) or re-close it with
close:
from valgebra import Validator
closed = Validator({"name": str})
assert not closed.is_valid({"name": "Ada", "extra": 1})
assert closed.open().is_valid({"name": "Ada", "extra": 1})
Openness is the default of the keys no clause claims. A clause names a
region of the key space — str: int claims the str keys — and the two
transforms decide the keys the clauses leave over and nothing else. The clauses
admitting every value are one clause over the union of their keys, however they
are written, so {str: anything, complement(str): anything} closes to {} as
the top does, and beside bytes: int it closes to the bytes keys alone. A record
claims no region, which is why opening one frees every key and closing one
refuses every key; that is the common case, not the general rule. A mapping
claims one, so opening dict[str, int] keeps what a str key maps to and frees
only the key-types beside it.
They apply at any depth -- a record a mapping maps its keys to, and inside a
recursive definition. They are projections rather than inverses: applying
either twice gives the schema it gave the first time, equal by ==: a name
saying what the new catch-all says is dropped by the first rewrite. And close
after open is at most close -- it returns the regions open freed
everywhere but the two shapes below.
Both rewrite the records a schema writes, not the set it denotes, so two spellings of one set can close apart. Close the records, not their meet: a meet of two open records admits what the open record of their fields admits, and closing it closes each record on its own, which no dict satisfies at once.
from valgebra import Validator, anything, intersection
meet = intersection(
Validator({"a": int, anything: anything}),
Validator({"b": str, anything: anything}),
)
record = Validator({"a": int, "b": str, anything: anything})
assert meet.is_equivalent(record)
assert meet.close().is_empty() # each record closed on its own
assert record.close().is_valid({"a": 1, "b": "x"})
from valgebra import Validator
# A mapping: the region its clause claims is not the transform's to touch.
mapping = Validator(dict[str, int])
assert not mapping.is_valid({1: "x"}) # no clause claims an int key
assert mapping.open().is_valid({1: "x"}) # opened, that region is free
assert not mapping.open().is_valid({"a": "x"}) # and a str key still maps to int
assert mapping.open().close() == mapping
# `{}` declares no field and claims no region: closed it admits the empty dict
# alone, opened it admits every dict.
assert Validator({}).is_valid({}) and not Validator({}).is_valid({"x": 1})
assert Validator({}).open().is_valid({"x": 1})
# And a clause reads the same whether or not a field is declared beside it.
beside_a_field = Validator({"name": str, str: int})
assert beside_a_field.close().is_equivalent(beside_a_field)
assert beside_a_field.open().is_valid({"name": "Ada", 7: "free"})
assert not beside_a_field.open().is_valid({"name": "Ada", "count": "not an int"})
Where it is not a round trip, the reason is that the transforms normalize,
and both normalizations are forced. Each drops a declared optional name that
says what its record says of every key it does not name, where that is one of
the two bounds -- absent beside a closed record, anything beside an open one,
the cases dom reads in the construction -- and an optional name typed like a
narrower catch-all, {"a?": int, str: int}, is kept: open writes the
catch-all beside {"a?": anything}, and the close after it drops the name --
and it has to, or {"a?": anything} and {}, one set once opened, would close
to two. Two
clauses over one key schema are one clause over the join of their values, as
closing {str: anything, bytes: int, complement(str): anything} gives {bytes:
anything}, and two clauses carrying one value are one clause over the union of
their keys, so opening dict[str, anything] gives a single clause over every key -- and that
has to be so too: where the keys claimed hold a literal, the long spelling is
keyed by the literal's complement, a shape the set representation
declines. The one join close leaves out is the one that
would write the top clause, which closes to {}: {anything: int, int: anything,
complement(int): anything} claims every key in its first clause, and closes to
itself. Neither fold is recoverable, so closing afterwards
lands on the smaller term:
from valgebra import Validator
record = Validator({"a?": object})
assert record.close().is_valid({"a": 1}) # closing it alone keeps the name
assert record.open().is_valid({"a": 1})
assert not record.open().close().is_valid({"a": 1}) # the name is gone
assert record.open().close() == Validator({})
mapping = Validator(dict[str, object])
assert repr(mapping.open()) == "dict[anything, anything]" # two clauses, folded
assert not mapping.open().close().is_valid({"a": 1})
The name the closed record drops is the one it says of every key it does not
name: an optional field admitting nothing is the closed record's default, so
{"a?": nothing} is {} and opens as {} does, to every dict. A key that
stays refused once opened is written beside a clause that frees the rest:
from valgebra import Validator, anything, nothing
assert Validator({"a?": nothing}).open() == Validator({}).open()
assert Validator({"a?": nothing}).open().is_valid({"a": 1})
refused = Validator({"a?": nothing, str: anything}).open()
assert not refused.is_valid({"a": 1})
assert refused.is_valid({"b": 1, 2: None})
dict[str, int] above is the ordinary case: its clause carries a value that is
not the top, nothing folds, and the round trip is exact.
The merged clause is recoverable while the freed region still stands in it:
{int: Any, str: int} opens to a clause over int beside the complement of int
| str, and close takes out that complement and nothing else. It is the folded
key above -- every key, with the claim inside it not spelled -- that closes to
the smaller term. A class is read the same way. Construction keeps
complement(C) | C as written, since isinstance may run a hook, but open and
close read it as every key: it is every key wherever isinstance answers one way
for a key. dict[C, Any] and dict[complement(C), Any] open to every key, and no
term tells them apart, so each closes to {}, as dict[int, Any] does:
from typing import Any
from valgebra import Validator, complement
class Key:
pass
mapping = Validator(dict[Key, Any])
assert repr(mapping.open()) == "dict[anything, anything]"
assert mapping.open().close() == Validator({})
assert Validator({complement(Key): Any}).open().close() == Validator({})
mixed = Validator({Key: Any, str: int})
assert mixed.open().close() == mixed.close()
assert not mixed.open().close().is_valid({None: 1})
To free some key-types and constrain others without open, write the
permissive clause yourself as the complement of the keys you constrained —
which is what open writes when the regions it frees are not the whole of what
is left:
from valgebra import Validator, anything, complement
partly_open = Validator({"name": str, str: int, complement(Validator(str)): anything})
assert partly_open.is_valid({"name": "a", "count": 1})
assert partly_open.is_valid({"name": "a", 7: object()}) # no clause claims it
assert not partly_open.is_valid({"name": "a", "count": "not an int"})
Under a complement, the direction reverses¶
A record reached through a complement is rewritten like any other. Opening it
makes the complemented set larger, which makes the complement — and so the whole
schema — smaller:
from valgebra import Validator, anything, complement
not_a_k_record = complement(Validator({"k": anything}))
schema = Validator({"x": not_a_k_record})
value = {"x": {"k": 1, "z": 2}}
# `{"k": 1, "z": 2}` is not a closed `{k: …}` record, so it is in the complement.
assert schema.is_valid(value)
# Opening admits it to the inner record, so it leaves the complement.
assert not schema.open().is_valid(value)
assert repr(schema.open()) == (
"{'x': complement({'k': anything, anything: anything}), anything: anything}"
)
close reverses under a complement for the same reason: it narrows the inner
record, which widens the schema.
So "open admits more" holds of the record the transform rewrites, and of the
whole schema only where no complement stands between them. Under a negation the
two swap. The practical consequence is that adding or removing an .open()
inside a complement changes the schema in the direction opposite to the one
the name suggests, and the change is invisible from outside unless you check —
repr shows it, as above.
Heterogeneous maps and catch-alls¶
A dict schema's string keys are named fields; any other key is a schema that keys a default clause for the rest. One form therefore expresses records, mappings, and their combination: several schema keys give a heterogeneous map whose value type depends on which key schema matches, and named fields plus a schema key give a record with a typed catch-all. Named fields take precedence over the catch-all.
Clauses are a disjunction, not a precedence list: a key that is not a named field
is admitted when some clause matches both it and its value. So overlapping
key schemas widen what the map admits rather than the earlier one winning, and
writing them in a different order does not change the schema's meaning. Two
keys that build one key schema, {Validator(int): str, int: bytes}, say one
clause twice, and are refused: write the key once, with the union of the
values.
from valgebra import Validator
# str keys map to ints, int keys map to strs
hetero = Validator({str: int, int: str})
assert hetero.is_valid({"a": 1, 2: "b"})
assert not hetero.is_valid({"a": "x"}) # a str key needs an int value
# a record whose every other key must be an int
extensible = Validator({"name": str, str: int})
assert extensible.is_valid({"name": "Ada", "age": 36})
assert not extensible.is_valid({"name": "Ada", "age": "old"})
A constant key that is not a str -- 1, None, b"x" -- is read as the
literal it spells (anything unrecognized is a literal),
so it keys a clause like any other schema: {"a": int, 1: str} is
{"a": int, Literal[1]: str}. A clause governs the keys it matches rather than
requiring them, so the key 1 is optional, and a literal reads its constant's
exact type, so True and 1.0, which a dict stores as the same key as 1, are
not it. The string keys are the fields, and the only keys a dict literal
requires.
from typing import Literal
from valgebra import Validator
coded = Validator({"a": int, 1: str})
assert coded.is_equivalent({"a": int, Literal[1]: str})
assert coded.is_valid({"a": 0}) # the clause does not require its key
assert coded.is_valid({"a": 0, 1: "x"})
assert not coded.is_valid({"a": 0, True: "x"}) # `True` is not the literal `1`
The one shape that separates a clause from a field is a literal-keyed clause
beside a kind clause that covers the same key. Both clauses read "k", so
"k" may carry what either admits, and the schema is below the record whose
optional field takes both. It is not the same set: a field reads every key equal
to its name, a str subclass's included, and a literal reads the exact str
alone. The field spelling is a different set again, since a field is read
instead of the clauses:
from typing import Literal
from valgebra import Validator
class Key(str):
__slots__ = ()
both = Validator({Literal["k"]: str, str: int})
assert both.is_valid({"k": "x"}) # the literal clause reads it
assert both.is_valid({"k": 1}) # and so does the str clause
assert not both.is_valid({"k": 1.5}) # neither does
assert both.is_subtype_of({"k?": str | int, str: int})
assert not both.is_valid({Key("k"): "x"}) # the literal reads the exact `str`
field = Validator({"k": str, str: int})
assert not field.is_valid({"k": 1}) # the field is read, not the clauses
assert field.is_valid({Key("k"): "x"}) # the field reads any key equal to it
A key schema names a type, a constant, or words¶
A clause's key says which keys it governs. It is a type -- str, int, a
union of them, Any -- or a Literal, which names the keys one by one, or a
str narrowed by a pattern or a length, which names the words of a regular
language:
from typing import Annotated
from valgebra import Regex, Validator
headers = Validator(dict[Annotated[str, Regex("x-.*")], str])
assert headers.is_valid({"x-trace": "abc"})
assert not headers.is_valid({"trace": "abc"}) # no clause governs this key
assert not headers.is_valid({"x-trace": 1})
# Relations read the words: every header is a `dict[str, str]`, and not the reverse.
assert headers.relation_to(dict[str, str]) == "subset"
assert Validator(dict[str, str]).relation_to(headers) == "not_subset"
Clauses are a disjunction, so two whose keys overlap read the join of their values on the keys they share, and each on its own elsewhere. A key takes one answer, however the clauses are written:
from typing import Annotated
from valgebra import Regex, Validator
overlapping = Validator(
{Annotated[str, Regex("a.*")]: int, Annotated[str, Regex(".*z")]: str}
)
assert overlapping.is_valid({"az": 1}) # both clauses hold `az`, and each admits
assert overlapping.is_valid({"az": "s"}) # a value of its own
assert not overlapping.is_valid({"ab": "s"}) # only the first holds `ab`
assert overlapping.relation_to(dict[str, int | str]) == "subset"
JSON Schema's patternProperties reads overlap the other way, as a conjunction.
To write it, give the keys both patterns hold a clause of their own, over the meet
of the two schemas, so no two clauses overlap and the two readings agree.
A length or a pattern written over a wider base and met with str is a key of the
words it leaves: intersection(str, Annotated[object, MinLen(2)]) is the key
Annotated[str, MinLen(2)] is, and builds as it does.
A key narrowed any other way -- a number's bound, a word's order, a callback -- names part of a kind no language holds, and is refused where it is written, and so is a recursion with such an arm, which keys a map as its definition. One keying a map inside its own body builds, since its arms are not built where the key is read: the walk reads it exactly, and a relation the key partition would decide declines. One level down, a narrowed position of a tuple key, it builds:
from typing import Annotated
import annotated_types as at
import pytest
from valgebra import Validator
with pytest.raises(NotImplementedError):
Validator({Annotated[int, at.Gt(0)]: int})
# Check such keys beside the mapping rather than inside it. The predicate is
# opaque, as every predicate is, so the keys are checked on membership and say
# nothing about inclusion.
positive_keys = at.Predicate(lambda d: all(k > 0 for k in d))
counts = Validator(Annotated[dict[int, int], positive_keys])
assert counts.is_valid({1: 1})
assert not counts.is_valid({0: 1})
Constraining some keys and freeing the rest¶
Because the clauses are a disjunction, a clause that matches every key subsumes every narrower one. So freeing the keys beside a typed clause means a clause over the complement of the keys that one claims: disjoint clauses cannot widen each other, and the constrained keys stay constrained.
That is what open writes. Written by hand it is how a caller frees
some of what is left rather than all of it:
from valgebra import Validator, anything, complement
# str keys must be ints; any key that is not a str is unconstrained.
partly_open = Validator({"name": str, str: int, complement(Validator(str)): anything})
assert partly_open.is_valid({"name": "Ada", "age": 36})
assert partly_open.is_valid({"name": "Ada", 7: object()}) # no clause claims it
assert not partly_open.is_valid({"name": "Ada", "age": "old"})
# And `open` frees the same region, so it writes the same schema.
assert Validator({"name": str, str: int}).open().is_equivalent(partly_open)
A complement of whole kinds stays in the decided
fragment: the set representation reads complement(str)
as every other kind. A complement of a literal holds part of a kind, which it
declines, so relations about such a map fall back to the rules. Membership is
unaffected either way — the walk reads the value.
Classes¶
| Form | Denotes |
|---|---|
TypedDict |
the dicts a record admits, open as the typing spec defines one; Required/NotRequired/ReadOnly honored, closed=True/extra_items obeyed and inherited by a subclass that gives neither |
| dataclass | the instances of the class whose every declared field holds a value of its type |
NamedTuple |
the instances of the class whose fields, by position, hold values of their types |
instance_of(C) |
the instances of the class, whatever their fields hold (below) |
any other class (a pydantic BaseModel, a msgspec Struct, your own) |
the instances of the class, as isinstance answers: what its fields hold is not read, whatever the class's own library validates (below) |
Enum |
the members of the enumeration |
Protocol |
the values carrying every member it declares, each holding what the member declares (below) |
NewType |
the set of the supertype it wraps |
PEP 695 type alias |
the set of the aliased type, and ties the fixpoint where the alias names itself (recursion) |
a generic alias applied, Pair[int] |
the set of its body with the arguments in place of its parameters, a missing one taking its default (recursion) |
a generic TypedDict, dataclass or NamedTuple applied, Box[int] |
the class's set with the arguments in place of its parameters in every field, a parameter left out taking its default (below) |
import enum
from dataclasses import dataclass
from valgebra import Validator
class Color(enum.Enum):
RED = 1
GREEN = 2
@dataclass
class Point:
x: int
y: int
assert Validator(Color).is_valid(Color.RED)
assert Validator(Point).is_valid(Point(1, 2))
assert not Validator(Point).is_valid(Point(1, "y"))
What a class declares is not every annotation on it. A ClassVar annotates
the class and an InitVar names a constructor parameter, so neither is an
attribute of an instance and neither is checked; a field declared
init=False is on the instance and is. A field declared Final[int] is a field
holding an int, as the typing spec says of a dataclass: Final says the
attribute is not assigned to after __init__, which is not a question about
the value. Annotated wraps the qualifier without changing it, so
Annotated[Final[int], Ge(0)] is a field holding Annotated[int, Ge(0)]. A
bare Final names no type -- a checker infers one from the default -- and the
class is refused. So is a field declared Annotated[ClassVar[int], ...]: it is
a class variable to the typing spec and a field to dataclasses, which reads
only an outermost ClassVar, and the message names the spelling of each.
That last one has an edge, and it is the check-only semantics showing through:
a field with init=False and no default is not set by the constructor, so
unless __post_init__ assigns it the attribute is absent from the instance —
and an absent attribute is not a value of any type. valgebra reads the object it
is given rather than the declaration, so such an instance is not a member until
something assigns the field.
from dataclasses import dataclass, field
from valgebra import Validator
@dataclass
class Row:
key: int
seen: bool = field(init=False) # no default: the constructor sets nothing
def touch(self) -> None:
self.seen = True
row = Row(1)
assert not Validator(Row).is_valid(row) # `seen` is not there yet
row.touch()
assert Validator(Row).is_valid(row)
On a TypedDict, Required,
NotRequired and ReadOnly qualify the key rather than narrowing its type:
required-ness is read from the qualifier where the field carries one, inside an
Annotated too, and from the class otherwise, and read-only-ness is about writing the key back rather
than about which values belong. Reading the qualifier is what makes a class mean
the same thing under from __future__ import annotations, where the class's own
key sets are computed from strings and cannot see it.
A generic class applied¶
A generic TypedDict, dataclass or NamedTuple names a field's type with its
parameter, and applied to arguments it is the class with each in its
parameter's place. A class deriving from one applied is read the same way,
however deep the derivation, and a parameter neither applied nor bound takes
its default, as PEP 696 reads a bare generic class:
import json
from dataclasses import dataclass
from valgebra import Validator
@dataclass
class Box[T]:
item: T
@dataclass
class IntBox(Box[int]):
label: str = ""
wrong = json.loads('{"item": "one"}') # nothing checks what `item` holds
assert Validator(Box[int]).is_valid(Box(1))
assert not Validator(Box[int]).is_valid(Box("one"))
assert not Validator(IntBox).is_valid(IntBox(**wrong))
A type variable is its class's own: a class that binds a base's parameter and
declares its own with the same TypeVar object reads each field in the scope of
the class that declares it, as the type checkers do. A default may name a
parameter before it, which the same application binds, as PEP 696 reads it: with
U = TypeVar("U", default=list[T]), Pairs[int] reads U as list[int], and
so does a base applied short, which Python hands back with the default as
written. A parameter with no default is refused where the class is written bare
and a field names it, and so is a class two bases reach under two arguments,
since a field it declares would name two types. So is a class applied to more
arguments than it declares parameters: Point[str], for a NamedTuple
declaring none, is an alias tuple's own subscription makes whatever the class
declares, and no field of Point has a place for str. A generic class whose fields are
not read -- one isinstance alone answers -- gains nothing from an argument,
and is refused applied
(above).
A TypedDict is open, a dict literal is closed¶
They denote different sets, and each denotes what its own author's spec says.
from typing import TypedDict
from valgebra import Validator
class User(TypedDict):
name: str
assert Validator(User).is_valid({"name": "Ada", "note": "extra"})
assert not Validator({"name": str}).is_valid({"name": "Ada", "note": "extra"})
Validator(TD) reads an annotation whose meaning is fixed by the typing spec,
and the spec makes a TypedDict open — reading it as a narrower set would be a
deviation the class carries no mark of. The dict-literal form is this library's
own spelling, and a schema written as a shape means that shape. Both sets are
spellable both ways: write closed=True (PEP 728) for a closed TypedDict, and
{"name": str, anything: anything} for an open shape -- or .open(), which
does the same to every record in a schema at once.
A TypedDict admits an instance of a dict subclass -- an OrderedDict, a
defaultdict, a class of your own -- read by its storage as every container
schema reads a subclass instance. The typing spec says values inhabiting a
TypedDict "must be instances of dict itself, not a subclass", and the type
checkers refuse OrderedDict[str, int] there; a record and a mapping are one
node here, and the exact type of a dict is a set no node names.
A protocol is the record of its members¶
A Protocol denotes the values that carry every member it declares, each read
the way an attribute access reads it. An annotated member holds a value of its
type, a method or special method holds a callable, and a property -- or a
functools.cached_property -- holds what its function's return annotation
names, or anything where it names nothing.
@runtime_checkable changes nothing here. It is what lets isinstance answer,
and a schema does not ask isinstance, so a protocol reads the same with or
without it, and the same on every Python release.
from typing import Protocol
from valgebra import Validator
class Named(Protocol):
name: str
def greet(self) -> str: ...
class Person:
def __init__(self, name: str) -> None:
self.name = name
def greet(self) -> str:
return f"hello, {self.name}"
class Robot:
name = 7
def greet(self) -> str:
return "beep"
assert Validator(Named).is_valid(Person("Ada"))
assert not Validator(Named).is_valid(Robot()) # `name` is present, not a str
assert not Validator(Named).is_valid("Ada") # a str carries no `name`
Each member is checked for what it holds, so a name that is present and
not a string is refused, where isinstance against the protocol would admit
it. A method is read as Callable[...] is, as callable with its parameters and
return unchecked (what a validator cannot do).
An instance of a class that names the protocol among its bases belongs only
where it carries the members.
test_a_protocol_admits_the_same_values_on_every_release in
tests/test_frontend_forms.py holds the values isinstance answers
differently on 3.11 than from 3.12 -- a member a __getattr__ hook
serves, a getter that raises, an unassigned slot -- to one answer on every
interpreter the test matrix runs.
Pass the class, not its annotations¶
Give Validator the class itself. It reads the annotations, resolves the string
forms, and keeps the Annotated metadata — which is where every refinement
lives.
Extracting the hints yourself and passing the mapping is the path that goes
wrong, because typing.get_type_hints drops Annotated metadata by default.
The schema still builds and still validates; it has quietly lost its
constraints, and nothing raises:
from typing import Annotated, TypedDict, get_type_hints
import annotated_types as at
from valgebra import Validator
class Account(TypedDict):
balance: Annotated[int, at.Ge(0)]
assert not Validator(Account).is_valid({"balance": -5}) # the class: constrained
stripped = get_type_hints(Account) # {'balance': <class 'int'>}
assert Validator(stripped).is_valid({"balance": -5}) # the bound is gone
kept = get_type_hints(Account, include_extras=True)
assert not Validator(kept).is_valid({"balance": -5}) # include_extras keeps it
If you must derive the hints — building a schema for a class chosen at runtime,
say — pass include_extras=True. Passing the class is the supported path and has
no such footgun.
Recursive classes
A class whose own type appears in a field (a tree node, a linked list) is
recursive and cannot compile directly unless the field names it through a
type alias, which ties the fixpoint
(recursion); otherwise express it
with recursive, which ties the fixpoint explicitly.
Bare classes, callables, and the runtime boundary
A class the tables above do not read by its declaration is an isinstance
check: Validator(complex) admits any complex, and a plain user class
admits its instances. Callable (and
Callable[...]) checks only that the value is callable — the argument and
return types cannot be inspected at runtime, so they are not enforced. Any
is admitted unchecked. Everything else is decided structurally: a list[int]
schema does check each element.
The class alone¶
A dataclass or a NamedTuple reads as its class and the fields it declares,
so an instance whose field holds another type is not a member. The class alone
-- every instance, whatever its fields hold, which is the set isinstance
answers -- is instance_of, since the annotation Box already names the
narrower set:
import json
from dataclasses import dataclass
from valgebra import Validator, instance_of
@dataclass
class Box:
a: int
ill_typed = Box(**json.loads('{"a": "x"}')) # nothing checks what `a` holds
assert not Validator(Box).is_valid(ill_typed)
assert instance_of(Box).is_valid(ill_typed)
assert Validator(Box).is_subtype_of(instance_of(Box))
It reads nothing the class declares, so a dataclass whose declaration has no
reading -- a bare Final field, a type parameter, a field naming the class
itself -- has this one. For any other class it is what Validator builds. A
TypedDict and a Protocol have no instances of their own, and are refused.
Refinements¶
Annotated[T, ...markers] denotes the values of T that satisfy every marker —
bounds, lengths, multiples, patterns and predicates. See the refinements guide.
Stable repr¶
A compiled validator prints back as an expression that builds it, which makes schemas inspectable — and makes what is printed something you can paste into a session and get the same schema from:
from valgebra import Validator, anything, recursive
assert repr(Validator(list[dict[str, int]])) == "list[dict[str, int]]"
# A record's fields print in name order, whatever order they were written in:
# they are a *map*, so the order is not part of the schema, and two spellings of
# one record are one schema.
assert repr(Validator({"name": str, "age?": int})) == "{'age?': int, 'name': str}"
assert Validator({"name": str, "age?": int}) == Validator({"age?": int, "name": str})
# A recursive schema prints as the call that builds it, the back edge as the
# lambda's own parameter.
tree = recursive(lambda t: {"value": int, "left?": t})
assert repr(tree) == "recursive(lambda X: {'left?': X, 'value': int})"
assert recursive(lambda X: {"value": int, "left?": X}) == tree
# An open record prints the catch-all it carries, so it reads back as itself.
opened = Validator({"name": str}).open()
assert repr(opened) == "{'name': str, anything: anything}"
assert Validator({"name": str, anything: anything}) == opened
It is a rendering, not a serialization. A few forms cannot be written as an
expression and do not read back, and the API page owns the list,
each with the reason. A schema deeper than the renderer's own bound prints
<...>, a mark chosen to be a syntax error, so a render that lost something
cannot be read back as a schema that kept it. A required key whose name ends in
?, which TypedDict("TD", {"a?": int}) declares and no dict display spells
(above), prints marked for the same
reason: {<required: 'a?'>: int, str: anything}. A class prints as its name
only where the name builds what the schema holds, so a class met with a shape or
a record it does not declare prints as the meet it is. Do not parse a repr to
recover structure — see inspection for asking a schema
questions instead.