Skip to content

Loading schemas

The three sources

A schema can come from a path, from text you already hold, or from bytes whose encoding you would rather not guess at.

import pathlib
import xsdkit

xsd_text = pathlib.Path("report.xsd").read_text()
raw = pathlib.Path("report.xsd").read_bytes()

xsdkit.SchemaSet.from_file("report.xsd")
# The `uri` is what the document is *treated* as having, which is what a
# relative `schemaLocation` inside it resolves against. Without one,
# relative locations resolve against the working directory.
xsdkit.SchemaSet.from_string(xsd_text, uri="report.xsd")
xsdkit.SchemaSet.from_bytes(raw, uri="report.xsd")

use xsdkit::{Diagnostics, SchemaSetBuilder};

fn load() -> Result<xsdkit::Schemas, Diagnostics> {
    SchemaSetBuilder::new()
        .file("report.xsd")
        .compile()
        .into_result()
}

use xsdkit::SchemaSetBuilder;

fn from_memory(xsd_text: String, raw: Vec<u8>) {
    // The second argument is the URI the document is *treated* as having,
    // which is what a relative `schemaLocation` inside it resolves against.
    SchemaSetBuilder::new().text(xsd_text, "report.xsd");
    SchemaSetBuilder::new().bytes(raw, "report.xsd");
}

The uri is not decoration. It is what diagnostics point at and what relative schemaLocation hints resolve against, so giving a real one to a string you loaded from somewhere else makes every later error message useful.

Several documents can go into one set — which is the normal case when a schema family has no single root:

# use xsdkit::SchemaSetBuilder;
let schemas = SchemaSetBuilder::new()
    .file("common.xsd")
    .file("orders.xsd")
    .file("shipping.xsd")
    .compile()
    .into_result()?;
# Ok::<_, xsdkit::Diagnostics>(())

Encodings are detected, not assumed

from_bytes reads the byte-order mark, then the XML declaration, then falls back to UTF-8. Bytes that contradict the encoding they claim are an error, not a document quietly full of U+FFFD replacement characters — a schema that silently loses a character in a pattern facet is worse than one that fails to load.

from pathlib import Path
schemas = xsdkit.SchemaSet.from_bytes(Path("report.xsd").read_bytes())

In Python, validate and iter_typed take str or bytes for the same reason: hand them the bytes and let the library read the declaration.

Finding the other documents

xs:include, xs:import, xs:redefine and xs:override all name a schemaLocation. The specification is explicit that this is a hint — a processor is free to ignore it and use its own copy — which is why every resolution strategy here is yours to choose.

Search paths

schemas = xsdkit.SchemaSet.from_file(
    "report.xsd",
    search_paths=["schemas/", "vendor/schemas/"],
)

Locations are tried relative to the referring document first, then against each search path in order.

A resolver

When the documents are not on disk at all — in a zip, in a database, behind an HTTP client you control, pinned to versions you vendored — supply a resolver. It is a function of (location, base).

```python,ignore import zipfile

with zipfile.ZipFile("schemas.zip") as z: schemas = xsdkit.SchemaSet.from_string( main_xsd, resolver=lambda location, base: z.read(location), ) 

Return `bytes` (best — the encoding is then detected), or `str`, or a
`(uri, document)` pair to record where it was actually found so diagnostics
name the right file. Raise to say it could not be resolved: your exception
message becomes the diagnostic, and the first exception raised becomes the
`SchemaError`'s `__cause__`, type and traceback intact. `KeyboardInterrupt` and
`SystemExit` are not reports of a missing document, so they stop the build and
propagate as themselves — Ctrl-C works in a slow resolver.

!!! warning "A resolver replaces the filesystem"

    It is an alternative to `search_paths`, not a layer on top of it. Once you
    supply one, it is asked for everything, and nothing falls back to disk, so
    passing both is refused with `ValueError`.

=== "Rust"

    ```rust
    use xsdkit::{Resolver, SchemaSetBuilder};

    struct Vendored;

    impl Resolver for Vendored {
        fn resolve(&self, location: &str, _base: Option<&str>) -> Result<(String, Vec<u8>), String> {
            let path = format!("vendor/{location}");
            std::fs::read(&path)
                .map(|bytes| (path.clone(), bytes))
                .map_err(|e| format!("{path}: {e}"))
        }
    }

    let schemas = SchemaSetBuilder::new()
        .resolver(Vendored)
        .file("report.xsd")
        .compile()
        .into_result();
    ```

### The network is off

The built-in `FileResolver` refuses `http://` and `https://` outright:

```text
error[XSD1101]: refusing to fetch `http://www.w3.org/2001/xml.xsd` over the network;
                supply a resolver or a local copy
  --> report.xsd:3
  help: `schemaLocation` is a hint; add a search path or a custom Resolver

Fetching a schema over the network at load time makes your build depend on someone else's uptime and turns a schema reference into a remote code path. If you want it, write four lines of resolver and own the decision. See Security.

One special case needs no fetching at all: the xml: namespace. xml:lang, xml:space, xml:base and xml:id are predeclared, so a schema that imports xml.xsd works without it being present.

Composition

All four composition mechanisms are implemented, including the awkward ones.

Directive What it does
xs:include Adds components from a document with the same target namespace
xs:import Makes another namespace's components referenceable
xs:redefine Includes a document and replaces some of its definitions (XSD 1.0)
xs:override The same idea, redesigned to be comprehensible (XSD 1.1)

Chameleon includes work: a document with no targetNamespace of its own is absorbed into its includer's namespace. The same file included by two different namespaces yields two distinct sets of components, which is the behaviour the specification requires and a classic source of bugs.

The namespace rules around that are enforced as well. An xs:include of a document that declares a different target namespace is an error (XSD1102) — pulling in another namespace is what xs:import is for — and an xs:import whose document does not declare the namespace it named is another (XSD1103), which is usually a schemaLocation pointing at the wrong file.

for d in schemas.documents:
    print(d.uri, d.target_namespace, "chameleon" if d.chameleon else "")

Circular include graphs terminate. So do circular type derivations, and every other graph walk in the library — see Security.

redefine has a rule that catches everyone

Inside xs:redefine, a reference to the name being redefined means the original. <complexType name="T"><extension base="T"> extends the included T, not the one being declared. xs:override deliberately has no such rule: there, references mean the new components.

Several root documents

A schema with no single root — a vendor bundle, or a directory of XSDs that import one another — loads into one set in one call.

```python,ignore schemas = xsdkit.SchemaSet.from_files(["orders.xsd", "invoices.xsd"]) schemas, diagnostics = xsdkit.load_files(sorted(pathlib.Path("vendor").glob("*.xsd"))) 

## Options

Every loader takes the same set.

| Option | Default | Meaning |
|---|---|---|
| `search_paths` | none | A list of directories, `str` or `Path`, to try for `schemaLocation` hints |
| `resolver` | filesystem | Replaces resolution entirely |
| `conformance` | `strict` | `lax` downgrades some errors — see below |
| `version` | `"1.0"` | `"1.1"` turns on XSD 1.1 — see [XSD 1.1](xsd11.md) |
| `nodes_limit` | 10,000,000 | Cap on XML nodes per document |
| `max_depth` | 256 | Cap on element nesting per document — see [Security](project/security.md#bounded-work) |

### Strict and lax

`strict` refuses to hand back a schema that had any error. `lax` downgrades the
violations that still permit building usable components — a dangling `import`
being the common one — so you get the model *and* the list of what was wrong.

Real schemas ship with broken references often enough that the mode earns its
keep.

=== "Python"

    ```python
    schemas, diagnostics = xsdkit.load("vendor/partial.xsd", conformance="lax")

    for d in diagnostics:
        print(d)
    # error[XSD1201]: no type named `{urn:vendor}Missing`
    #   --> vendor/partial.xsd:12
    ```

    `load`, `load_files`, `load_string` and `load_bytes` return the diagnostics
    instead of raising.
    `SchemaSet.from_file` raises `SchemaError` — which carries the full list on
    its `.diagnostics` — so use whichever matches whether imperfection is
    expected.

=== "Rust"

    ```rust
    use xsdkit::{Compilation, Conformance, SchemaSetBuilder};

    let Compilation { schemas, diagnostics } = SchemaSetBuilder::new()
        .conformance(Conformance::Lax)
        .file("vendor/partial.xsd")
        .compile();
    ```

    `compile()` always returns both halves — a schema with errors still
    compiles to components, and one without them can still have warnings worth
    reading. `Compilation::into_result()` is where the `Result` shape is asked
    for, and where the choice to discard warnings on success is made
    explicitly.

## Cost

Loading is linear in the size of the documents and is the expensive half;
querying afterwards is not. A 3,000-declaration schema compiles in about 15 ms.
Compile once and keep the result — see [Performance](project/performance.md).

## Compiling once and loading thereafter

When "keep the result" has to survive the process — a CLI run repeatedly, a
Python interpreter started per request — the `serde` feature makes a compiled
`Schemas` serializable, so compilation happens once and everything afterwards
is a load.

The Python wheel is built with it. A `SchemaSet` pickles, so a process pool
shares one without compiling it again in every worker, and `serialize()` and
`SchemaSet.deserialize()` give the bytes for a cache of your own:

```python
schemas = xsdkit.SchemaSet.from_file("report.xsd")
cached = schemas.serialize()
xsdkit.SchemaSet.deserialize(cached).validate(Path("report.xml")).is_valid   # True

In Rust, turn the feature on:

xsdkit = { version = "0.3", features = ["serde"] }

use xsdkit::Schemas;

fn cache(schemas: &Schemas) -> Result<Schemas, postcard::Error> {
    let bytes = postcard::to_allocvec(schemas)?;
    postcard::from_bytes(&bytes)
}

Any serde format works, self-describing ones included: names are stored as a table and every component refers to them by index, and that table is written as a sequence so a format with string-only keys can carry it too.

On a 900 KB schema of 2,000 types this is about 7× — 31 ms to compile against 4.5 ms to load — at the cost of a cache several times the size of the source XSD. Whether that trade is worth taking depends entirely on the schema, so measure it on yours:

cargo run --release --features serde --example cache -- main.xsd [search/path ...]

The format is not stable across versions

A name is an index into an interner, so a cache only means anything alongside the code that wrote it. Key the cache on the crate version and rebuild on a miss; do not ship one as a data file. SchemaSet.deserialize refuses bytes another xsdkit version wrote, with ValueError, and like pickle it is only for bytes you trust.

Next