Skip to content

Conformance

What the compiler promises against JSON Schema, how it is checked, and where a difference is allowed to live.

The contract

For every keyword valgebra_jsonschema/schema.py accepts and every JSON value, compiled membership gives the verdict the specification gives. A keyword it cannot translate raises Unsupported; it is never skipped. A difference not in the ledger below is a defect.

The oracles

  • The specification, as the official JSON-Schema-Test-Suite encodes it, vendored at commit 89e83a740146 with its MIT license: draft 2020-12 with the optional files on regular expressions and on large numbers, and drafts 2019-09, 07, 06 and 04 through upgrade.
uv run python tools/suite.py --check docs/studies/json-schema-test-suite.json
uv run python tools/suite.py --draft 2019-09 --check docs/studies/json-schema-test-suite-draft2019-09.json
uv run python tools/suite.py --draft 7 --check docs/studies/json-schema-test-suite-draft7.json
uv run python tools/suite.py --draft 6 --check docs/studies/json-schema-test-suite-draft6.json
uv run python tools/suite.py --draft 4 --check docs/studies/json-schema-test-suite-draft4.json
  • Real-world schemas. JSONSchemaBench's 9558, fetched at a pinned commit and never vendored, each checked on drawn and generic values against the jsonschema package's validator for its draft:
uv run python tools/corpus.py --check docs/studies/jsonschemabench.json
  • Inclusions, as the JSON Schema containment test suite labels them: pairs of schemas fetched at a pinned commit and never vendored, each labeled direction asked of valgebra. No proof may contradict a label, and a refutation that does is a wrong label the results page names:
uv run python tools/containment.py --check docs/studies/containment.json
  • The jsonschema package, pinned in the development group, as a second implementation: tests/test_properties.py compares membership on 400 random schemas and 25 random values each, over the keywords whose meaning the two share exactly.

Where the specification and the package disagree, the specification wins, and the disagreement is a ledger entry against the package.

The inventory

Every keyword of the draft 2020-12 vocabularies is in exactly one class, and tests/test_boundaries.py holds this table against the code in both directions.

class keywords
translated exactly type, enum, const, maximum, exclusiveMaximum, minimum, exclusiveMinimum, an integral multipleOf, minLength, maxLength, pattern, prefixItems, items, contains, minItems, maxItems, properties, required, additionalProperties, minProperties, maxProperties, patternProperties, propertyNames, dependentRequired, dependentSchemas, allOf, anyOf, oneOf, not, if, then, else, $ref, $id, $anchor, $dynamicAnchor, $dynamicRef, unevaluatedProperties, unevaluatedItems
translated as an opaque predicate uniqueItems, minContains, maxContains, a fractional multipleOf, patternProperties past four patterns or in a schema past valgebra's limit, a propertyNames reading more than words that is neither a boolean schema nor a list of names, and oneOf and a $ref to a definition that does not reach itself in a schema past valgebra's limit with keys read by predicate too
annotations title, description, default, examples, deprecated, readOnly, writeOnly, format, contentEncoding, contentMediaType, contentSchema, $schema, $vocabulary, $comment, $defs, definitions
refused $recursiveRef, $recursiveAnchor, additionalItems, dependencies

$ref resolves against the base URI it sits under, as draft 2020-12's section 8.2 states: $id sets the base of its resource, and the reference names a resource of the document by its URI, then a schema in it by a JSON pointer or by an $anchor. A document the caller supplies in compile_schema(schema, documents={uri: document}) is a resource at its URI, read in its own dialect, or in the compiled document's where it declares none. The compiled document names its own URI, and a document supplied at that URI is not read. A URI two schema objects of one document claim, and an anchor two of one resource set, name neither: a reference to one is refused, and a document that names neither compiles. Of two documents, the one supplied at a URI names it, then an embedded $id of the document supplied last. A reference to any other document is refused and never fetched, and so is an embedded resource declaring another $schema. An older draft's $id fragment that is a plain name is the $anchor it names. $dynamicRef reads the dynamic scope across every document, as draft 2020-12's sections 7.1 and 8.2.3.2 state it, and a draft 2019-09 $recursiveRef: "#" is the $dynamicRef: "#" that reads the dynamic anchor with no name upgrade writes for $recursiveAnchor: true on a resource's root, the correspondence the draft 2020-12 release notes give.

A keyword outside the vocabularies is an annotation, as draft 2020-12 says to treat it, and Compiler.ignored records it: x-label or OpenAPI's nullable assert nothing. Two draft-07 keywords that 2020-12 removed, additionalItems and dependencies, are refused in a 2020-12 schema rather than ignored: a schema carrying them was written to a draft where they assert, and ignoring them would admit values its author meant to refuse. In a draft-07 schema upgrade translates them.

A pattern is read by translating it into valgebra's dialect, and one the translation cannot write, a lookaround past the pattern's start or a backreference, refuses its schema: 5 of JSONSchemaBench's 9558. A second engine reading ECMA-262 for those alone, as a predicate, would split this package's reading of a pattern in two, the translation's and the engine's, whose agreement nothing tests past the patterns it takes, and would make a package of Python over valgebra carry a native dependency for them. Refusing a schema it cannot evaluate is what the specification's next release requires.

Differences ledger

what the specification jsonschema 4.26.0 this package why
pattern classes \d, \w, \s, ., boundaries \b, \B ECMA-262: ASCII digits and word characters, ECMA-262 white space, any character but four line terminators, boundaries of ASCII word characters Python's re: Unicode classes ECMA-262 the specification; on optional/ecmascript-regex jsonschema passes 42 of 74, this package 74 of 74 (tools/suite.py --reference)
$ before a final newline no match match no match the specification
an escape of punctuation ECMA-262 leaves undefined under the u flag (\:, \- outside a class), a lone ] or }, a { that opens no count a pattern with one is not valid under u; Annex B reads the character the character the character, the rest of the pattern as under u every dialect reads the character: Annex B, Python's re and valgebra's engine; Annex B reads the rest of such a pattern by UTF-16 code units, where u reads code points. 34 of the corpus's 2471 patterns hold one; any other construct invalid under u is refused
fractional multipleOf the decimal numbers written binary floats: 0.0075 is not a multiple of 0.0001 exact rationals: a value's shortest decimal, an integral value its integer the specification; an integral value read as its integer agrees with an integral step's reading, as it must, since a multiple of 3 is one of 1.5
a number written with a fraction or an exponent: a bound, a constant the decimal number written the double json.loads returns the double json.loads returns the universe is what json.loads returns: {"maximum": 1e23} refuses 10**23, since 1e23 parses to 99999999999999991611392
NaN, which json.loads returns for the token NaN no JSON value: RFC 8259 §6 permits no NaN a number a value of no schema the specification: a relation is about JSON values, and a NaN outside every bound would refute {"type": "number"} below {"anyOf": [{"maximum": 10}, {"minimum": 5}]}, which every number satisfies
a string holding a lone surrogate, which json.loads returns for "\udc00" a pattern searches its code units Python's re searches its code points no pattern matches it, and a property name holding one is refused valgebra's engine reads Unicode text, and such a string is none; RFC 8259 calls its behavior unpredictable
integer in a draft-04 schema a number written without a fraction or an exponent: 1.0 and 1e300 are none the same an integral number, as draft 2020-12 reads integer: 1.0 and 1e300 are integers draft-04's integer is a property of a number's spelling, while its own equality makes 1 and 1.0 one value, so it is no set of JSON values; every leaf here holds every spelling of one value, which is what a relation between two sets reads, and the reading by value is the one later drafts give integer
integral constants above 2**53 one number each one number each one number each, its float spelling only when the float is the same number 9007199254740993 and 9007199254740992.0 are two numbers
format: hostname under assert_format, on an xn-- label the label is an IDNA A-label, so its Punycode and what it decodes to are read not read the label as it is spelled: 23 of the suite's 66 cases, each of them invalid and admitted reading a label takes IDNA2008's tables, a package of their own and no dependency here
additionalItems, dependencies in a 2020-12 schema not keywords: annotations annotations refused see the inventory
a document supplied at the compiled document's own URI no URI identifies two schemas, and a validator SHOULD raise an error where two claim one (section 9.1.2) the compiled document: referencing registers the root at its $id over the registry's resource the compiled document; the supplied one is not read section 9.1.1 makes the root's $id its canonical URI, and a registry a caller passes may hold the compiled document; JSONSchemaBench's CloudFormation schema declares the draft-04 meta-schema's URI as its $id, the corpus study supplies that meta-schema, and the schema's 9092 references name its own definitions
a URI two schema objects of one document claim, or an anchor two of one resource set no URI identifies two schemas, and a validator SHOULD raise an error where two claim one (section 9.1.2); two anchors of one name are undefined, and an implementation MAY raise one (section 8.2.2) one of the two: the first written of two siblings, and of two under different keywords, one that changes with Python's string hashing (PYTHONHASHSEED) neither: a $ref or $dynamicRef naming it is refused the error the specification asks for, raised where a reference meets the ambiguity: 15 JSONSchemaBench documents hold such a URI, none names it, and each compiles; the semantics of Attouche et al. (POPL 2024, section 3) reads a URI as naming one schema
a draft 2019-09 $recursiveRef whose dynamic scope enters a resource without $recursiveAnchor between two with it the outermost schema in the scope setting it (section 8.2.4.2.2) the outermost of the resources setting it entered since the last one without it the outermost schema in the scope setting it the specification, and the draft 2020-12 release notes, whose $dynamicRef works the same way

The valgebra boundary

valgebra_jsonschema/_valgebra.py is the only module that imports valgebra, and the derived forms the compiler needs — the conditional, key presence, exactly-one-of, a keyword's scope, a fixed-length list, the callback path — are written there once, as the recipes of valgebra's documentation. tests/test_boundaries.py fails if any other module imports valgebra.