nuthatch/docs/spec/testing.md

# Testing

## Host unit tests

The recorded host toolchain is GHC 9.10.3, cabal-install 3.16.1.0, and libpq
18beta1 or a later compatible libpq release. `make test` runs the
`nuthatch-unit` Tasty suite entirely on the host. It requires no network access
and no PostgreSQL, Redis-compatible, OIDC, S3-compatible, container, or
Kubernetes service.

The suite uses Tasty as its single explicit test tree, Tasty HUnit for named
examples, and Hedgehog through Tasty Hedgehog for generated properties.
Hedgehog failures include the seed needed to replay the shrunk counterexample.
Generators represent valid and invalid domains with distinct types.

## Model correspondence

`make check-model-correspondence` runs before the kind protocol-conformance
gate. It uses the typed Haskell protocol evidence executable, Node 24.18.0,
Java 25.0.3, and the TLA+ Tools 2.19 jar whose checksum is pinned in
`tests/model-correspondence/tools.env`. The jar defaults to
`/usr/local/lib/tla2tools.jar`; `TLA2TOOLS_JAR` may name an equivalent file
with the pinned checksum.

The checker discovers executable leaf actions below `Next` from SANY's
semantic graph, executes a minimal TLC configuration, resolves checks named by
every behavioral configuration, and compares them with typed implementation
inventories. Concrete snapshot fixtures are projected into the model
vocabulary without implementing an abstract transition relation. Generated
graphs and TLC state are created in a temporary directory and are not retained.

## Kind dependency fixture

Backend integration tests execute inside an isolated kind cluster so they use
the same DNS and service routing boundaries as deployed Nuthatch components.
The fixture provides PostgreSQL, Redis and DragonflyDB, Keycloak and ZITADEL,
and Garage and VersityGW. Images and fixture tools are checksum or
digest-pinned.

`make check-dependencies` creates the complete fixture, verifies each released
dependency image through its product interface, and destroys the fixture.
`make check-kind-matrix` runs a fixture-consumer Job sequentially across the
Redis/DragonflyDB × Keycloak/ZITADEL × Garage/VersityGW matrix without
deploying Nuthatch. `make check-app` deploys Nuthatch and runs its smoke Job
across the Redis/DragonflyDB × Garage/VersityGW matrix. `make app-up` retains
the first app lane for interactive work, and `make app-down` removes it.
`tests/kind/setup.sh` creates the dependency fixture without Nuthatch for
interactive test work; `tests/kind/destroy.sh` removes either form.
`NUTHATCH_TESTBED_RETAIN=1` retains a failed all-in-one or matrix run for
inspection.

The fixture uses `tests/kind/.runtime/kubeconfig` and the
`kind-nuthatch-testbed` context. Every Kubernetes operation supplies both
explicitly. Fixture lifecycle operations are serialized by a host-runtime lock
shared across worktrees, and teardown is idempotent.

All fixture credentials and data are synthetic. Test workloads use ClusterIP
service names and do not require host port forwarding. The fixture does not
read or change the user's Kubernetes context.

Test runners select the `base`, `durable`, `coordination`, `oidc`, and
`target` fixture families; `all` selects every dependency family. The selected
families expand into provider lanes, which run sequentially and stop at the
first failure. Each lane receives an isolated PostgreSQL database,
Redis-compatible logical database, and target bucket as applicable.

Lane configuration is available both as environment variables from a
lane-specific ConfigMap and as `/run/nuthatch-fixture/profile.env`. Successful
lanes delete their Job, profile, database, coordination state, and bucket
before the next lane. Failed lanes remain intact when the enclosing runner is
retained. Locally built application or test images are loaded with
`load_local_image`; released dependencies always use the digest-pinned images
from `tests/kind/versions.env`.

The host provides rootless nerdctl and containerd at the versions recorded in
`tests/kind/versions.env`, together with `ctr`, `curl`, `flock`, `jq`, and
`sha256sum`. The fixture downloads its pinned kind and kubectl binaries.