149 lines
8 KiB
Markdown
149 lines
8 KiB
Markdown
# Remora API implementation contract
|
|
|
|
These files are mutable implementation artifacts. They describe the current
|
|
HTTP operations and WebSocket frames but are not authoritative over
|
|
[`Remora.tla`](../protocol/Remora.tla) or
|
|
[`architecture.md`](../spec/architecture.md). Conflicts are corrected here.
|
|
|
|
OpenAPI `3.2.0` describes HTTP operations and JSON Schema Draft 2020-12
|
|
describes WebSocket frames. The API contract version is `1.0.0`; additive
|
|
fields require clients to tolerate unknown response fields only after a
|
|
contract version that permits them, while incompatible changes use a new major
|
|
path or media type.
|
|
|
|
## Idempotency
|
|
|
|
Every public domain mutation requires `Idempotency-Key`. The identity is caller
|
|
subject, operation, and key. The server canonicalizes the semantic request,
|
|
including the authorization target, with RFC 8785 and stores its SHA-256
|
|
fingerprint and complete response atomically with the mutation. The default
|
|
retention is 24 hours and is never shorter than the advertised
|
|
`Idempotency-Expires-At`.
|
|
|
|
Event-ticket issuance is an ephemeral authentication exchange rather than a
|
|
domain mutation. Each retry independently mints a short-lived credential; it
|
|
does not refine a `JobProtocol` effect or enter the public mutation ledger.
|
|
|
|
The same identity and fingerprint replays the original status, resource, and
|
|
body after duplicate or ambiguous delivery. A different fingerprint returns
|
|
`idempotency_conflict` and cannot affect either request. Naturally idempotent
|
|
deletion and cancellation semantics still use this request identity.
|
|
|
|
Internal claim uses job ID plus `operation_id` because no attempt exists yet.
|
|
Renew, progress, and outcome use attempt ID, fencing token, worker incarnation,
|
|
and `operation_id`. Each operation identity retains a request fingerprint. An
|
|
exact replay has the original effect; a changed worker, fence, or payload
|
|
returns `precondition_failed` without mutation. First observation is accepted
|
|
only when the corresponding `JobProtocol` action is enabled. Later eligibility
|
|
changes do not alter a stored response, while a fresh operation is evaluated
|
|
against current eligibility.
|
|
|
|
Renewal follows `RenewLease`: it extends an existing live coordination lease only
|
|
while the attempt remains current durable authority, `AttemptExecuting`, and
|
|
non-terminal. It never creates or restores a claim. After authority loss the
|
|
lease ages out or is released; it cannot be extended. Progress and outcome
|
|
operations require current fenced authority and return `precondition_failed`
|
|
for stale or non-authoritative tuples.
|
|
|
|
Activation returns the live lease expiry and a recommended renewal time. Renewal
|
|
returns the corresponding updated projection. These timestamps schedule worker
|
|
behavior; the coordination store remains authoritative for whether the lease is
|
|
live.
|
|
|
|
A successful claim returns the attempt authority and the attempt's
|
|
`ExecutionEnvelope`. The worker verifies that their job, attempt, fence, and
|
|
worker incarnation agree before source or target I/O. Resolved-source and
|
|
checkpoint descriptors identify durable streamable state; they are not
|
|
reconstructed from coordination data.
|
|
|
|
Resolved-source and checkpoint references carry their descriptor kind, schema
|
|
version, byte length, and SHA-256. The service accepts them only when their kind,
|
|
ownership, integrity, and fenced attempt authority match the mutation.
|
|
|
|
[`RemoraApiMutation`](../protocol/RemoraApiMutation.tla) models this durable
|
|
operation ledger around an abstract successful `JobProtocol` effect. It does
|
|
not redefine job phases or attempt-authority transitions. Its `Rejected`
|
|
response abstracts the endpoint-specific `idempotency_conflict` and
|
|
`precondition_failed` codes enforced by the executable contract.
|
|
|
|
## Authorization and errors
|
|
|
|
Public operations apply the authorization rules in the architecture. List is
|
|
filtered before pagination. A missing and an unauthorized resource both return
|
|
`not_found`; callers cannot probe job existence. Cursors and event replay are
|
|
bound to the authenticated subject and filter.
|
|
|
|
`ApiError` fields are an allowlist. Messages and field diagnostics never carry
|
|
credential values, bearer material, URL userinfo, raw upstream response bodies,
|
|
internal stack traces, or unredacted configured endpoints.
|
|
|
|
Credential creation uses a closed material shape selected by `kind`; rotation
|
|
must use the existing kind's shape. Custom HTTP headers use lowercase canonical
|
|
names and reject connection-nominated, routing, framing, cookie, proxy, and
|
|
authentication headers.
|
|
|
|
## Jobs, pages, and results
|
|
|
|
`Job.phase` uses the exact `JobProtocol` phase vocabulary; this contract does
|
|
not define phase transitions. `lifecycle`, current authoritative attempt,
|
|
current matching progress, reusable-checkpoint projection, result, and failure
|
|
are separate fields. Progress is present only when its attempt ID and fence
|
|
match the authoritative attempt in the same projection.
|
|
|
|
Job pages order by `(created_at DESC, id DESC)`. The first request fixes a
|
|
membership snapshot, authorization/filter fingerprint, and high-water mark.
|
|
`next_cursor` is an opaque integrity-protected keyset cursor for that snapshot;
|
|
new jobs do not enter later pages. Invalid, altered, cross-subject, or
|
|
cross-filter cursors return the corresponding structured cursor error.
|
|
|
|
A result endpoint returns only a visible `CommittedResult`. Tombstoned,
|
|
unauthorized, absent, and never-committed results are not exposed. The hard
|
|
delete endpoint returns the durable deletion resource established by the
|
|
architecture.
|
|
|
|
Before hard deletion, job reads return the complete `job` projection. Once only
|
|
the permanent minimal record remains, authorized reads and deleted-job pages use
|
|
the distinct `tombstone` projection rather than inventing discarded job fields.
|
|
|
|
A deletion whose exact remaining work cannot progress because target access or
|
|
configuration is unavailable enters `blocked` with a redacted problem. It
|
|
retains the deletion obligation and can return to active processing when the
|
|
dependency is restored. Owner access remains authorized through the concrete
|
|
target retained in the tombstone.
|
|
|
|
Worker failure outcomes accept only attempt-reportable runtime classes and do
|
|
not accept a caller-supplied retryable flag. The service derives the durable
|
|
disposition from the class mapping in the architecture. Validation,
|
|
cancellation, and lost-attempt projection use their dedicated boundaries.
|
|
|
|
## WebSocket replay
|
|
|
|
An HTTP job or list snapshot carries `event_cursor` at its consistency
|
|
watermark. The client connects to `/v1/events` with that cursor; there is no
|
|
subscription frame. Server frames are ordered by increasing `sequence` for a
|
|
connection and carry an opaque next cursor. Delivery is at least once, and
|
|
clients deduplicate event IDs. Each connection starts at sequence one and
|
|
increments it by one for every frame, including heartbeats and replayed event
|
|
IDs; a client resnapshots when it observes a gap.
|
|
|
|
Browser clients first create a short-lived, single-use event ticket through an
|
|
OIDC bearer-authenticated HTTP request. The server stores only its digest and
|
|
binds it to the authenticated principal and exact allowed origin. Redemption
|
|
atomically consumes it during the WebSocket handshake. Native clients may
|
|
authenticate that handshake directly with the OIDC bearer header.
|
|
|
|
`job_changed` is an invalidation with an attempt-aware display hint, not a
|
|
resource replacement or patch. A client compares `job_revision` with the last
|
|
HTTP snapshot revision and fetches a new authorized snapshot when it is newer.
|
|
It advances its applied revision only from that snapshot. Matching progress may
|
|
be displayed provisionally while the fetch is pending.
|
|
|
|
`deletion_changed` is likewise an invalidation. Its state is a display hint;
|
|
the deletion resource remains the authoritative HTTP projection.
|
|
|
|
Replay applies current authorization before delivery. If a cursor is absent,
|
|
invalid, expired, authorization-incompatible, or no longer available after
|
|
coordination loss, the server sends `resync_required`; the client obtains a new
|
|
HTTP snapshot and reconnects from its cursor. Progress frames for a stale
|
|
attempt/fence are suppressed before emission and ignored defensively by the
|
|
client.
|