ms2k/remora
SHA256
Watch
0
Code Releases Activity

Replace transient test labels with stable identifiers

Browse source
This commit is contained in:
ms2k 2026-07-19 14:27:29 +09:00
commit d0b745950b
1 changed files with 63 additions and 38 deletions
Download patch file Download diff file Expand all files Collapse all files

10
docs/spec/testing.md
View file

@ -4,21 +4,21 @@ Backend integration and fault tests execute as Kubernetes `Job` resources inside
## Fixture lifecycle
`make check-dependencies` remains the dependency compatibility gate: it creates the full fixture, runs the released-image probes, and tears the fixture down. `make check-kind-sample` builds and loads a Remora-owned cargo-nextest runner, then exercises the full provider matrix around [`fixture-consumer-sample`](../../tests/kind/sample-job.yaml.in). Its init containers access the lane database, coordination namespace, OIDC discovery and signing keys, target bucket, and Vault path before cargo-nextest runs. That Job is the reference structure for a work-package test Job.
`make check-dependencies` remains the dependency compatibility gate: it creates the full fixture, runs the released-image probes, and tears the fixture down. `make check-kind-sample` builds and loads a Remora-owned cargo-nextest runner, then exercises the full provider matrix around [`fixture-consumer-sample`](../../tests/kind/sample-job.yaml.in). Its init containers access the lane database, coordination namespace, OIDC discovery and signing keys, target bucket, and Vault path before cargo-nextest runs. That Job is the reference structure for a `TestComponent` test Job.
For interactive work, `tests/kind/setup.sh` creates a ready fixture and leaves it running. `tests/kind/destroy.sh` removes it. Setup replaces any fixture left by an earlier run, teardown is idempotent, and a host-runtime lock shared across worktrees serializes lifecycle mutations around the single `remora-testbed` cluster and `remora-wp02` containerd namespace. `REMORA_TESTBED_RETAIN=1` retains a failed all-in-one run for inspection.
For interactive work, `tests/kind/setup.sh` creates a ready fixture and leaves it running. `tests/kind/destroy.sh` removes it. Setup replaces any fixture left by an earlier run, teardown is idempotent, and a host-runtime lock shared across worktrees serializes lifecycle mutations around the single `remora-testbed` cluster and `remora-dependency-testbed` containerd namespace. `REMORA_TESTBED_RETAIN=1` retains a failed all-in-one run for inspection.
The fixture owns an isolated kubeconfig at `tests/kind/.runtime/kubeconfig` and the `kind-remora-testbed` context. Every Kubernetes operation uses both explicitly. Package runners use the `kube` helper from `tests/kind/lib.sh`; they do not read or change the user's kubeconfig or current context.
## Package test Jobs
Each work package owns its test code, test-runner image, Job manifest, acceptance command, and required compatibility families. Its all-in-one runner follows [`run-sample-job.sh`](../../tests/kind/run-sample-job.sh): acquire the fixture lock, install a cleanup trap, call `setup_fixture <families...>`, load any package-owned image with `load_local_image <image> [containerd-namespace]`, and call `run_job_matrix <job-name> <manifest> <image> <families...>`. The image namespace defaults to `default`. A retained failure prints the isolated kubeconfig and context.
Each `TestComponent` owns its test code, test-runner image, Job manifest, acceptance command, and required compatibility families. Its all-in-one runner follows [`run-sample-job.sh`](../../tests/kind/run-sample-job.sh): acquire the fixture lock, install a cleanup trap, call `setup_fixture <families...>`, load any component-owned image with `load_local_image <image> [containerd-namespace]`, and call `run_job_matrix <job-name> <manifest> <image> <families...>`. The image namespace defaults to `default`. A retained failure prints the isolated kubeconfig and context.
The selectable families are `base`, `durable`, `coordination`, `oidc`, `target`, `secret`, and `all`. `base` provisions only the cluster and common namespace for suites whose controlled fixtures run inside their test pod. Individual implementations are not selectors. A suite that requests `coordination`, `oidc`, and `target` runs the unchanged cargo-nextest archive against the full Redis/DragonflyDB × Keycloak/ZITADEL × Garage/VersityGW Cartesian product. Lanes run sequentially and stop on the first failure.
Before each lane, the runner recreates a lane database, allocates unique coordination keys, provisions an OIDC tenant/client/user/role grant, creates a target bucket and prefix, and allocates a Vault path as required by the selected families. The resulting environment is exposed both through environment variables and a ConfigMap-mounted `/run/remora-fixture/profile.env`. Package manifests use `@@JOB_NAME@@`, `@@PROFILE_NAME@@`, and `@@TEST_IMAGE@@` placeholders so only the profile and lane identity vary between Jobs.
Released third-party dependencies are always the digest-pinned images loaded by the fixture. Work packages may build Remora test-runner images locally; they do not rebuild or wrap dependency images. The Job manifest names every loaded image exactly and sets `imagePullPolicy: Never`.
Released third-party dependencies are always the digest-pinned images loaded by the fixture. Test components may build Remora test-runner images locally; they do not rebuild or wrap dependency images. The Job manifest names every loaded image exactly and sets `imagePullPolicy: Never`.
A test Job:
@ -34,4 +34,4 @@ Package runners may create additional namespaced fixtures and fault injectors ne
Automated backend tests use cluster-local service discovery. Tests of Keycloak or ZITADEL issuer behavior use the `dnsPolicy` and `dnsConfig` pattern in [`fixture-consumer-sample`](../../tests/kind/sample-job.yaml.in) so discovery URLs and token issuers have one identity from every participating pod.
Port forwarding is for interactive diagnostics only and is not an acceptance-test transport. Ingress is introduced by deployment and system qualification, where routing is itself under test; substrate work packages do not install an ingress controller merely to reach fixture services.
Port forwarding is for interactive diagnostics only and is not an acceptance-test transport. Ingress is introduced by deployment and system qualification, where routing is itself under test; substrate test components do not install an ingress controller merely to reach fixture services.