nuthatch/docs/spec/testing.md

Testing

Host unit tests

The recorded host toolchain is GHC 9.10.3, cabal-install 3.16.1.0, and libpq 18beta1 or a later compatible libpq release. make test runs the nuthatch-unit Tasty suite entirely on the host. It requires no network access and no PostgreSQL, Redis-compatible, OIDC, S3-compatible, container, or Kubernetes service.

The suite uses Tasty as its single explicit test tree, Tasty HUnit for named examples, and Hedgehog through Tasty Hedgehog for generated properties. Hedgehog failures include the seed needed to replay the shrunk counterexample. Generators represent valid and invalid domains with distinct types.

Kind dependency fixture

Backend integration tests execute inside an isolated kind cluster so they use the same DNS and service routing boundaries as deployed Nuthatch components. The fixture provides PostgreSQL, Redis and DragonflyDB, Keycloak and ZITADEL, and Garage and VersityGW. Images and fixture tools are checksum or digest-pinned.

make check-dependencies creates the complete fixture, verifies each released dependency image through its product interface, and destroys the fixture. make check-kind-matrix runs a fixture-consumer Job sequentially across the Redis/DragonflyDB × Keycloak/ZITADEL × Garage/VersityGW matrix without deploying Nuthatch. make check-app deploys Nuthatch and runs its smoke Job across the Redis/DragonflyDB × Garage/VersityGW matrix. make app-up retains the first app lane for interactive work, and make app-down removes it. tests/kind/setup.sh creates the dependency fixture without Nuthatch for interactive test work; tests/kind/destroy.sh removes either form. NUTHATCH_TESTBED_RETAIN=1 retains a failed all-in-one or matrix run for inspection.

The fixture uses tests/kind/.runtime/kubeconfig and the kind-nuthatch-testbed context. Every Kubernetes operation supplies both explicitly. Fixture lifecycle operations are serialized by a host-runtime lock shared across worktrees, and teardown is idempotent.

All fixture credentials and data are synthetic. Test workloads use ClusterIP service names and do not require host port forwarding. The fixture does not read or change the user's Kubernetes context.

Package runners select the base, durable, coordination, oidc, and target fixture families; all selects every dependency family. The selected families expand into provider lanes, which run sequentially and stop at the first failure. Each lane receives an isolated PostgreSQL database, Redis-compatible logical database, and target bucket as applicable.

Lane configuration is available both as environment variables from a lane-specific ConfigMap and as /run/nuthatch-fixture/profile.env. Successful lanes delete their Job, profile, database, coordination state, and bucket before the next lane. Failed lanes remain intact when the enclosing runner is retained. Package-owned application or test images are loaded with load_local_image; released dependencies always use the digest-pinned images from tests/kind/versions.env.

The host provides rootless nerdctl and containerd at the versions recorded in tests/kind/versions.env, together with ctr, curl, flock, jq, and sha256sum. The fixture downloads its pinned kind and kubectl binaries.