ms2k
20ea9557fd
Run package test jobs across every selected provider combination with isolated lane state, released-image canaries, controlled OIDC DNS, and deterministic cleanup.
37 lines
4.7 KiB
Markdown
37 lines
4.7 KiB
Markdown
# Implementation Testing
|
||
|
||
Backend integration and fault tests execute as Kubernetes `Job` resources inside the kind dependency fixture. Cluster-local execution gives test code the same DNS, service routing, and failure boundaries as the deployed services without exposing fixture dependencies on the host.
|
||
|
||
## Fixture lifecycle
|
||
|
||
`make check-dependencies` remains the dependency compatibility gate: it creates the full fixture, runs the released-image probes, and tears the fixture down. `make check-kind-sample` builds and loads a Remora-owned cargo-nextest runner, then exercises the full provider matrix around [`fixture-consumer-sample`](../../tests/kind/sample-job.yaml.in). Its init containers access the lane database, coordination namespace, OIDC discovery and signing keys, target bucket, and Vault path before cargo-nextest runs. That Job is the reference structure for a work-package test Job.
|
||
|
||
For interactive work, `tests/kind/setup.sh` creates a ready fixture and leaves it running. `tests/kind/destroy.sh` removes it. Setup replaces any fixture left by an earlier run, teardown is idempotent, and the harness serializes lifecycle mutations around the single `remora-testbed` cluster. `REMORA_TESTBED_RETAIN=1` retains a failed all-in-one run for inspection.
|
||
|
||
The fixture owns an isolated kubeconfig at `tests/kind/.runtime/kubeconfig` and the `kind-remora-testbed` context. Every Kubernetes operation uses both explicitly. Package runners use the `kube` helper from `tests/kind/lib.sh`; they do not read or change the user's kubeconfig or current context.
|
||
|
||
## Package test Jobs
|
||
|
||
Each work package owns its test code, test-runner image, Job manifest, acceptance command, and required compatibility families. Its all-in-one runner follows [`run-sample-job.sh`](../../tests/kind/run-sample-job.sh): acquire the fixture lock, install a cleanup trap, call `setup_fixture <families...>`, load any package-owned image with `load_local_image <image> [containerd-namespace]`, and call `run_job_matrix <job-name> <manifest> <image> <families...>`. The image namespace defaults to `default`. A retained failure prints the isolated kubeconfig and context.
|
||
|
||
The selectable families are `durable`, `coordination`, `oidc`, `target`, `secret`, and `all`. Individual implementations are not selectors. A suite that requests `coordination`, `oidc`, and `target` runs the unchanged cargo-nextest archive against the full Redis/DragonflyDB × Keycloak/ZITADEL × Garage/VersityGW Cartesian product. Lanes run sequentially and stop on the first failure.
|
||
|
||
Before each lane, the runner recreates a lane database, allocates unique coordination keys, provisions an OIDC tenant/client/user/role grant, creates a target bucket and prefix, and allocates a Vault path as required by the selected families. The resulting environment is exposed both through environment variables and a ConfigMap-mounted `/run/remora-fixture/profile.env`. Package manifests use `@@JOB_NAME@@`, `@@PROFILE_NAME@@`, and `@@TEST_IMAGE@@` placeholders so only the profile and lane identity vary between Jobs.
|
||
|
||
Released third-party dependencies are always the digest-pinned images loaded by the fixture. Work packages may build Remora test-runner images locally; they do not rebuild or wrap dependency images. The Job manifest names every loaded image exactly and sets `imagePullPolicy: Never`.
|
||
|
||
A test Job:
|
||
|
||
- runs in `remora-testbed` with `restartPolicy: Never`, `backoffLimit: 0`, and a finite `activeDeadlineSeconds`;
|
||
- disables service-account token mounting and Kubernetes service-link environment variables unless the test explicitly requires them;
|
||
- declares CPU and memory requests and limits;
|
||
- exits nonzero for every failed assertion and writes diagnostic output to its container logs; and
|
||
- uses ClusterIP service names for dependencies and synthetic fixture credentials only.
|
||
|
||
Package runners may create additional namespaced fixtures and fault injectors needed by their acceptance matrix. Process-wide coordination fault tests run exclusively. A failed lane is retained when `REMORA_TESTBED_RETAIN=1`; successful lanes delete their Job and profile before the next lane. Tests never depend on state left by an earlier lane or package.
|
||
|
||
## Network access
|
||
|
||
Automated backend tests use cluster-local service discovery. Tests of Keycloak or ZITADEL issuer behavior use the `dnsPolicy` and `dnsConfig` pattern in [`fixture-consumer-sample`](../../tests/kind/sample-job.yaml.in) so discovery URLs and token issuers have one identity from every participating pod.
|
||
|
||
Port forwarding is for interactive diagnostics only and is not an acceptance-test transport. Ingress is introduced by deployment and system qualification, where routing is itself under test; substrate work packages do not install an ingress controller merely to reach fixture services.
|