Separate lease-loss release and finalize session terminals
This commit is contained in:
parent
1698296ecd
commit
eded0faa7a
1 changed files with 316 additions and 22 deletions
|
|
@ -9,7 +9,7 @@ This document is the normative product specification for Lazaret.
|
|||
| This specification | Product north star, trust model, stable vocabulary and identifiers, below-abstraction architecture, and product requirements |
|
||||
| [TLA+ application protocol model](protocol/Lazaret.tla) and [TLC config](protocol/Lazaret.cfg) | Protocol lifecycle, communication semantics, publication, registration, escalation, recovery, and checked invariants |
|
||||
| [Prepared-lower TLA+ model](protocol/PreparedLower.tla) and [TLC config](protocol/PreparedLower.cfg) | Prepared-artifact publication, generation leases, mounts, reclamation, stage recovery, and checked invariants |
|
||||
| [Design schemas](schema.md) | Logical durable records, network-policy documents, readiness evidence, and their relationships |
|
||||
| [Design schemas](schema.md) | Logical records, protocol data shapes, network-policy documents, readiness evidence, and their relationships |
|
||||
|
||||
This specification does **not** restate modeled actions or invariants. The TLA+
|
||||
models are authoritative above their abstraction boundaries. This specification
|
||||
|
|
@ -207,20 +207,105 @@ session identity needed for routing and cleanup, including Session ID, PTY maste
|
|||
|
||||
Interactive terminal control requires:
|
||||
|
||||
- Window resize applied to the host PTY master and guest `SIGWINCH` delivered to
|
||||
the exact session process or process group through runsc.
|
||||
- Client control events (for example interrupt) translated into runsc signal
|
||||
operations for that exact session process or process group; raw host PTY
|
||||
control characters alone are not a sufficient contract.
|
||||
- Per-session exit status collected from the non-detached `runsc exec` client as
|
||||
the primary source.
|
||||
- On session close, PTY master loss, or exec-client loss: apply the modeled
|
||||
session-close transition, stop the exact guest session process or process
|
||||
group through runsc, reap the exec client, and close the PTY.
|
||||
- Each interactive command starts as the leader of a generation-bound guest
|
||||
process group. The Host Coordinator records both the leader PID and process
|
||||
start time and the process group ID relative to the exact runsc container
|
||||
before admitting terminal events.
|
||||
- Window resize applies `TIOCSWINSZ` to the host PTY master and then delivers
|
||||
guest `SIGWINCH` to that process group through runsc. Rows and columns are
|
||||
nonzero unsigned 16-bit values; pixel dimensions may be zero when unknown.
|
||||
- Client `Interrupt`, `Quit`, and `Suspend` events translate respectively to
|
||||
`SIGINT`, `SIGQUIT`, and `SIGTSTP` for that process group. Raw control bytes do
|
||||
not substitute for these events. Host CLI suspension is local: the CLI
|
||||
restores its terminal before stopping itself and sends the current window size
|
||||
after resuming; it does not synthesize a guest signal.
|
||||
- Per-session exit status comes from the non-detached `runsc exec` client. An
|
||||
interactive PTY produces one merged output byte stream, so the protocol does
|
||||
not label its bytes as separate stdout and stderr streams.
|
||||
|
||||
Disconnected Client Sessions close rather than reattach to an unbounded scrollback
|
||||
owned by the coordinator.
|
||||
|
||||
The session connection carries the terminal events and messages defined by the
|
||||
normative design schema. One Host Coordinator event loop serializes input,
|
||||
resize, signal, close, prompt-routing, PTY output, and exec-client completion for
|
||||
each Session ID. It uses bounded queues and PTY backpressure; terminal data for
|
||||
one session cannot consume another session's control capacity. Input, resize,
|
||||
and signal events are accepted only while the modeled terminal route is the
|
||||
sandbox route. An invalid Session ID, inactive route, stale generation, or
|
||||
terminal in release refuses the event.
|
||||
|
||||
PTY output is forwarded only while the modeled terminal route is the sandbox
|
||||
route. During `Prompt` or `HostProcess` routing the Host Coordinator stops reading
|
||||
the PTY master and relies on bounded kernel PTY buffering and backpressure; it
|
||||
does not accumulate an unbounded user-space queue. Forwarding resumes only after
|
||||
the model restores the sandbox route.
|
||||
|
||||
Normal guest completion begins the modeled session close and physical release,
|
||||
so no new terminal or escalation event is admitted. The Host Coordinator
|
||||
first cancels any active prompt or exact host process and serializes that
|
||||
cancellation before terminal output. It then collects and reaps the exec client,
|
||||
drains and forwards PTY bytes in order until EOF or a bounded drain deadline,
|
||||
and emits at most one `SessionExit` after the last forwarded byte on the closing
|
||||
connection. This close handshake does not restore session routing. If the
|
||||
connection is already lost, the outcome is not retained for reattachment.
|
||||
|
||||
Physical Session Terminal acquisition is ordered as follows:
|
||||
|
||||
1. The Host Coordinator records `SessionTerminal=Intent` in the Domain Store.
|
||||
2. It allocates the PTY and forks a wrapper with the expected HC pid/start
|
||||
identity. The wrapper arms its HC parent-death edge, re-reads its parent pid
|
||||
and start identity, and aborts before invoking runsc if either differs. It may
|
||||
create the guest process only after that recheck succeeds.
|
||||
3. It establishes the guest process group and obtains the guest leader PID,
|
||||
start time, and group ID.
|
||||
4. One Domain Store transaction records the exact Host Coordinator, PTY, exec
|
||||
client, container, guest leader, and guest process-group identities and sets
|
||||
`SessionTerminal=Running`.
|
||||
5. Only then may the Host Coordinator accept terminal events for the session.
|
||||
|
||||
Failure before `Running` never admits the terminal. A live Host Coordinator uses
|
||||
its in-memory exact handles to terminate any partial exec and guest group and
|
||||
records release intent. If HC itself is lost, the armed exec edge stops the host
|
||||
client and the generation lease-loss path stops the guest sandbox; bootstrap
|
||||
recovery cleans the retained Intent only after runtime absence.
|
||||
|
||||
Session close, Client connection loss, PTY loss, or exec-client loss starts the
|
||||
same physical release:
|
||||
|
||||
1. Apply the modeled session close, remove terminal and escalation routing, and
|
||||
record `SessionTerminal=ReleaseIntent` while retaining all exact identities.
|
||||
2. If the exact generation is still serviceable and the guest leader identity
|
||||
and group still match, send `SIGHUP`, wait for a bounded grace period, then
|
||||
send `SIGTERM`, wait again, and send `SIGKILL` if the group remains. A leader
|
||||
mismatch refuses the numeric process-group signal. If the leader is absent,
|
||||
an empty recorded group needs no signal; a nonempty group is left in release
|
||||
intent for Domain teardown.
|
||||
3. Terminate and reap the exact exec client when it is still a Host Coordinator
|
||||
child and close the exact PTY master. Record `SessionTerminal=Clean` only
|
||||
after the guest group, exec client, and PTY are absent; otherwise retain
|
||||
release intent.
|
||||
|
||||
The process-group boundary is the per-session cleanup contract, not a security
|
||||
boundary. A sandbox process may daemonize or move into another process group;
|
||||
the final Client Session leaving the Domain triggers Domain teardown, which is
|
||||
the containment boundary for such processes.
|
||||
|
||||
Hard Host Coordinator loss closes its PTY descriptors and fails modeled open
|
||||
sessions; the exec-client lifetime edge prevents an unrecorded or recorded exec
|
||||
client from surviving that loss. The surviving Runtime Supervisor follows the
|
||||
generation lease-loss teardown: it stops the complete Agent Sandbox, unmounts
|
||||
the exact State Anchor while retaining the prepared-lower lease, and exits.
|
||||
Bootstrap recovery waits until the Runtime Supervisor and attached runsc are
|
||||
absent, never recreates a PTY or reattaches a Client Session, releases the
|
||||
retained prepared lower under the durable generation refinement, verifies that
|
||||
the exact generation container and guest processes are gone, terminates any
|
||||
still-live exact exec-client host process, and then records each retained Session
|
||||
Terminal `Clean`. An `Intent` without physical identities becomes `Clean` only
|
||||
after that runtime-absence check. A process, container, PTY, or generation
|
||||
mismatch fails closed and retains the record; it is never adopted or signaled as
|
||||
the recorded session.
|
||||
|
||||
### 3.4 Filesystem and Git Model
|
||||
|
||||
The Agent Workspace is mounted read-write. Tracked paths and untracked paths that
|
||||
|
|
@ -338,6 +423,23 @@ The physical unmount and lease-release refinement is:
|
|||
catalog transaction.
|
||||
5. Commit `prepared_lower = Clean` in the Domain Store.
|
||||
|
||||
Host Coordinator hard loss uses the model's split lease-loss refinement. The
|
||||
surviving exact Runtime Supervisor stops the sandbox and sidecar, verifies and
|
||||
unmounts its exact State Anchor, and exits without opening the Domain Store or
|
||||
Storage Catalog. The original durable anchor intent and exact Prepared ownership
|
||||
authorize rollback of the resource the supervisor created; they do not authorize
|
||||
catalog lease removal. The catalog lease remains held after unmount.
|
||||
|
||||
Bootstrap recovery begins only after the Runtime Supervisor and attached runsc
|
||||
are absent. If the exact mount remains, the bootstrap Host Coordinator records
|
||||
`state_anchor = ReleaseIntent` and completes normal exact unmount. If the
|
||||
surviving Runtime Supervisor already unmounted it, bootstrap independently
|
||||
verifies exact mount absence. It then atomically commits `state_anchor = Clean`
|
||||
and `prepared_lower = ReleaseIntent`, removes the exact catalog lease under the
|
||||
catalog lock, and commits `prepared_lower = Clean`. Thus lease-loss unmount and
|
||||
lease release are distinct: only a lock-holding Host Coordinator releases the
|
||||
prepared-lower lease.
|
||||
|
||||
Recovery resumes only from matching durable identity. `state_anchor =
|
||||
ReleaseIntent` retains the catalog lease while recovery verifies and completes
|
||||
the unmount. `prepared_lower = ReleaseIntent` permits removal of a still-present
|
||||
|
|
@ -346,8 +448,11 @@ permits the Domain participant to become `Clean`. A catalog lease with missing,
|
|||
unreadable, non-release, or mismatched Domain evidence is retained and reported.
|
||||
A Domain `LeaseHeld` record whose exact catalog lease is absent is inconsistent
|
||||
and fails closed; recovery neither recreates the lease nor mounts the State
|
||||
Anchor. These rules make every interrupted release conservative and prevent a
|
||||
mounted State Anchor from becoming unleased.
|
||||
Anchor. A failed generation with a Prepared anchor, an absent exact mount, and a
|
||||
still-present exact catalog lease is the conservative HC-loss state above; only
|
||||
bootstrap recovery may advance it to release intent. These rules make every
|
||||
interrupted release conservative and prevent a mounted State Anchor from
|
||||
becoming unleased.
|
||||
|
||||
Store version acceptance occurs before physical resource inspection. The first
|
||||
product version has no product predecessor. Future migration steps create and
|
||||
|
|
@ -487,7 +592,7 @@ not duplicated here.
|
|||
| **FR-9** | Trusted host configuration shall allow the developer to disable Git-ignored path isolation for an Agent Profile. |
|
||||
| **FR-11** | The host CLI shall list, inspect, structurally verify, and reset all or selected State Anchors without requiring direct manipulation of Lazaret's storage layout. |
|
||||
| **FR-14** | The Host Coordinator shall own one host PTY master per interactive Client Session; the PTY slave shall attach through non-detached `runsc exec` standard streams. The root attached `runsc run` client shall remain terminal-neutral. Production Session PTYs shall not depend on `runsc exec -console-socket`. |
|
||||
| **FR-15** | For each interactive Client Session, Lazaret shall apply window resize to the host PTY master and deliver guest `SIGWINCH` to the exact session process or process group through runsc; translate Client control events into runsc signal operations; collect exit status primarily from the non-detached `runsc exec` client; and after the modeled session-end transition stop the exact guest process or process group, reap the exec client, and close the PTY. |
|
||||
| **FR-15** | For each interactive Client Session, Lazaret shall preserve the Session Terminal record and protocol in the normative design schema; serialize bounded terminal events and merged output by Session ID; map resize and the `Interrupt`, `Quit`, and `Suspend` events to the specified exact process-group operations; collect exit status from the non-detached `runsc exec` client; and follow the specified release and hard-loss recovery ordering without reattachment or mismatched cleanup. |
|
||||
| **FR-16** | Domain network connectivity shall use `slirp4netns` as the production connectivity helper. The Domain Store shall record independent `helper` intent before launch and exact helper and supervising Runtime Supervisor pid/start identities before network readiness. |
|
||||
| **FR-17** | The Domain Store shall be opened only by the Host Coordinator process holding the Domain `flock`; mutations shall use whole redb transactions. Storage Catalog opens shall be bounded by `storage-catalog.lock` and one transaction. Cross-store lease release shall follow the durable release-intent, exact-unmount, catalog-delete, Domain-clean ordering defined by the durable generation refinement. |
|
||||
| **FR-18** | Product network-readiness evidence shall conform to the normative design schema and include the readiness fingerprint plus the generation-bound deny/permit/DNS probe result. |
|
||||
|
|
|
|||
Loading…
Reference in a new issue