okami.mom/albedo

git clone https://git.nkp.pet/okami.mom/albedo

Aly Raffauffix(daemon): remove stopped session running marks8d97901

main
11.5 KiB195 linesraw

session attachment

Session creation and user turn admission use durable operation receipts. An accepted submission survives a daemon restart while it waits for transcript consumption. A retry with the same operation ID returns the original admission.

The CLI app package owns session ID resolution. Full lowercase UUIDv7 IDs address roots and children directly. Shortened IDs match only root sessions, with an exact match taking precedence over prefixes. A missing full ID returns the same no-match error as a missing prefix; resume never creates a replacement. Opening reads the root list and settings once, then captures the selected session once and adds a child to its initial list if needed.

opening a session attaches the cli to its event stream and independently reads GET /sessions/{session_id}?tail=0. replayed history does not establish the session's live phase; status and live events do.

an idle status is enough to enable the composer, including a persisted interrupted session after a daemon restart. a missing or unrecognized phase is display metadata, not a failed connection. before live status is known, enter preserves the draft; superseded status replies cannot unblock it. starting a new turn does not require /compact to change the old phase.

Running observations

daemon/albedo_bus.erl owns the ephemeral running cache. Only active turns retain a {SessionId, OwnerPid} record. An idle publication removes the calling actor's exact record, and both registry owner-death handling and the progress owner monitor remove the stopped actor's record. Delayed cleanup cannot remove a replacement actor's active mark. Missing records and dead owners read as idle, so the registry can check activity without calling a busy session. The table lasts for the daemon lifetime; it retains no idle or deleted session IDs after owner cleanup. Changes to this cache require a fresh daemon lifetime rather than hot-loading code.

Stream failures and recovery

Internal checkpoint and provider-started signals are suppressed before stream encoding and do not allocate SSE sequence numbers.

Session streams require the text/event-stream content type, zstd-coded when the request accepts it (encoding): http_stream flushes the coder after every frame and ends the frame before a deliberate stop. A zstd context works only in the process that created it, so the stream process creates its own writer. Each batch carries a nonempty generation, a nonnegative integer cursor, and an events array. The generation identifies one lifetime of the session actor. Reconnect requests send both after_generation and after_seq; an initial request omits both. A missing or different generation, or a sequence outside the replay window, returns a durable transcript reset. Empty keepalive batches carry the current pair too.

Attachment requires session_replay capability version 2. Reset snapshots include active_output: unfinished text and thinking captured with the same actor cursor as the durable history watermark. The session actor owns this projection. Do not reconstruct it from the replay ring or bounded activity previews, which may have already discarded the beginning of a response.

The projection retains at most 64 KiB of raw inline text and emits at most 64 KiB of encoded inline content across its segments. Larger output spills to temporary files with 16 KiB buffered writes. Capture flushes pending writes before returning references. Each reference fixes the byte cutoff at capture; later deltas cannot extend that prefix. Full-content pages use /sessions/{session_id}/active-output/{content_id} and retain the original snapshot query while following next tokens.

The CLI stages complete prefix hydration before replacing its transcript, then restores history and active output before processing later events. Provisional chunks retain message identity even after the live display buffer settles them. Canonical assistant and thinking publication precedes committed.replaces_live_ids, which identifies matching provisional chunks. Model commits set replaces_all_live to retire all provisional output, including fragments absent from the bounded projection after a storage failure. Retry identities never reuse a failed attempt's content. Cancellation and failure retire remaining provisional output without changing durable history.

Issued spill references live for 15 minutes across commit and retry. Existing maintenance removes expired and orphaned files; session deletion revokes them. Spill storage is bounded to 64 MiB per run, 1 GiB across retained files, and 4096 retained files. A projection has at most 256 segments. Spill directories use mode 0700; content and metadata use 0600. Owner identity includes the VM incarnation so reused PIDs cannot retain orphaned files after restart. Storage failure makes backfill explicitly unavailable rather than returning a truncated successful snapshot. It must not abort generation or discard a durable commit. Expired references return 410 active_output_expired and cause the client to capture a new snapshot. Do not add compatibility readers for snapshots without active output.

Text and thinking deltas reach subscribers at most once a frame (16 ms): session_state.emit publishes a delta after a quiet frame at once and holds later ones until the frame ends, merging neighbours of one message into one event. Any other event, a commit, a retry, and the end of a turn publish the held deltas first, so the stream keeps its order and active output never lags a commit. The replay window keeps at most 256 events and less than one response's bytes, so any replay fits a single batch.

Session subscribers receive coalesced wake notifications. Event payloads stay in the session's existing replay buffer; a successful frame acknowledges its cursor before another wake is issued. A subscriber's death removes its watch without requiring another event. A session that stops (closed at shutdown or deleted) wakes its subscribers, and a stream whose session stopped or did not answer ends with a clean EOF rather than a failure batch: the daemon closes sessions before it stops listening, so that is a restart or a deletion, and the client's reconnect learns which.

The CLI validates the whole batch before consuming it and saves both cursor values only after every callback succeeds. Initial attachment and a generation change require a leading reset. Within one generation, the sequence cannot decrease without a reset. A generation change with a reset is normal recovery. A change without a reset is a protocol failure. The reset replaces visible history and clears live tool progress. Reset batches include current_progress in the captured session snapshot. The CLI delivers that snapshot after history as live progress, so attaching during a tool call restores its current tool name, phase, and bounded code preview. Older-history navigation uses transcript row IDs. Unknown event kinds are ignored. Missing kinds and malformed known events are protocol failures.

Transport failures, ordinary EOF, HTTP 408/429, and server errors reconnect with a delay starting at 500 milliseconds and increasing to at most five seconds. EOF and transient failures retain the saved pair. Cancellation ends the subscription quietly and clears the pair, as does explicit protocol recovery. Other HTTP refusals, explicit SSE failures, and consumer callback failures end the subscription with a visible notice, except an SSE failure coded daemon_unavailable or daemon_stopping, which older daemons send while they shut down and which reconnects like a server error. A terminal failure also stops status polling for that attachment.

The TUI attempts one durable transcript reset per attachment after a protocol failure. It clears live tool progress and requires the recovery stream to begin with a reset. A second protocol failure ends automatic recovery; reopening the session starts a new attachment. Pending submissions and drafts remain available, and a stream failure does not claim that model execution finished.

Notes in the transcript

The TUI renders daemon notes as faint plain text, wrapping each line before styling it. Never style a whole multi-line note as one Lip Gloss block: that pads every short line to the longest line, which can span thousands of columns in a capability update. Long notes use the same click-to-expand fold as user messages. Folding changes only the display, not the stored or model-facing text.

Loaded sessions and unloading

A session is loaded while its actor runs; only a loaded session has a stream cursor. Opening it, watching its stream, or submitting to it loads it. Reading GET /sessions/{session_id}/history never does, so the session list previews stored history instead of the session resource.

The maintenance sweep releases an idle kernel after ALBEDO_IDLE_SECONDS (default 10 minutes) and unloads a session whose last turn (activity_at) is older than ALBEDO_UNLOAD_SECONDS (default one hour): the provider's prompt cache has expired by then. The session refuses while a turn runs or waits, a client watches its stream, a background job is live, or one of its children works — runs a turn, or sits between turns on a job of its own, or has a child that works: that child's report will wake it, and its prompt cache may be kept warm meanwhile (robot-docs/cache-warming.md). Unloading saves the kernel's variables like a release. The next request loads it again.

Listing sessions reads loaded sessions' summaries but does not count as attention, so an open session list never keeps a kernel alive.

The same sweep collects every waiting process whose heap passed 1 MB (albedo_daemon:collect_idle): a long-lived actor otherwise keeps the heap a busy turn grew it to. The launcher's VM defaults (priv/bin/albedo-daemon) then hand the freed carriers back to the OS instead of caching them.

The session list marks a session active (green) while it is loaded and its last turn is under an hour old.

asynchronous persistence failures

The dispatcher logs failures to advance delivered schedule occurrences or save mail delivery errors. Undelivered mail and unadvanced schedules remain durable and retry under the existing dispatch policy. A schedule whose submission succeeded but whose advance failed can be delivered again.

Prompt ownership and cancellation

Inputs use PUT /sessions/{session_id}/inputs/{input_id} with a client-generated UUIDv7. The same identity follows admission, transcript consumption, turn membership, and cancellation. Accepted inputs survive restart. The input contract defines message, continuation, skill, and command variants.

POST /sessions/{session_id}/inputs/{input_id}/cancel returns the current input and cancelled, interrupt_requested, shared_running, or not_pending. A waiting cancellation commits its receipt and pending removal together. A running turn shared with other inputs continues. Cancellation never falls back to a session-wide interruption or affects a later unrelated turn.

The CLI uses a separate five-second cleanup deadline and reports unconfirmed cancellation when cleanup fails. An interruption acknowledgement does not claim that work stopped.

Live turn membership precedes worker output and expands when queued input steers a turn. Completion follows its final output or failure. The durable input read retains membership and terminal outcome across reconnects and restarts. Prompt clients follow their input identity instead of inferring completion from echo metadata or idle status. Drafts and unresolved input handles survive resets.

1# session attachment
2
3Session creation and user turn admission use durable [operation receipts](operations.md).
4An accepted submission survives a daemon restart while it waits for transcript
5consumption. A retry with the same operation ID returns the original admission.
6
7The CLI app package owns session ID resolution. Full lowercase UUIDv7 IDs
8address roots and children directly. Shortened IDs match only root sessions,
9with an exact match taking precedence over prefixes. A missing full ID returns
10the same no-match error as a missing prefix; resume never creates a replacement.
11Opening reads the root list and settings once, then captures the selected session
12once and adds a child to its initial list if needed.
13
14opening a session attaches the cli to its event stream and independently
15reads `GET /sessions/{session_id}?tail=0`. replayed history does not establish the
16session's live phase; status and live events do.
17
18an idle status is enough to enable the composer, including a persisted
19`interrupted` session after a daemon restart. a missing or unrecognized
20phase is display metadata, not a failed connection. before live status is
21known, enter preserves the draft; superseded status replies cannot unblock
22it. starting a new turn does not require `/compact` to change the old phase.
23
24## Running observations
25
26`daemon/albedo_bus.erl` owns the ephemeral running cache. Only active turns
27retain a `{SessionId, OwnerPid}` record. An idle publication removes the
28calling actor's exact record, and both registry owner-death handling and the
29progress owner monitor remove the stopped actor's record. Delayed cleanup
30cannot remove a replacement actor's active mark. Missing records and dead
31owners read as idle, so the registry can check activity without calling a busy
32session. The table lasts for the daemon lifetime; it retains no idle or deleted
33session IDs after owner cleanup. Changes to this cache require a fresh daemon
34lifetime rather than hot-loading code.
35
36## Stream failures and recovery
37
38Internal checkpoint and provider-started signals are suppressed before stream
39encoding and do not allocate SSE sequence numbers.
40
41Session streams require the `text/event-stream` content type, zstd-coded when
42the request accepts it ([encoding](../docs/http-api-design.md)): `http_stream`
43flushes the coder after every frame and ends the frame before a deliberate
44stop. A zstd context works only in the process that created it, so the stream
45process creates its own writer. Each batch carries
46a nonempty `generation`, a nonnegative integer `cursor`, and an `events` array.
47The generation identifies one lifetime of the session actor. Reconnect requests
48send both `after_generation` and `after_seq`; an initial request omits both.
49A missing or different generation, or a sequence outside the replay window,
50returns a durable transcript reset. Empty keepalive batches carry the current
51pair too.
52
53Attachment requires `session_replay` capability version 2. Reset snapshots
54include `active_output`: unfinished text and thinking captured with the same
55actor cursor as the durable history watermark. The session actor owns this
56projection. Do not reconstruct it from the replay ring or bounded activity
57previews, which may have already discarded the beginning of a response.
58
59The projection retains at most 64 KiB of raw inline text and emits at most
6064 KiB of encoded inline content across its segments. Larger output spills to
61temporary files with 16 KiB buffered writes. Capture flushes pending writes
62before returning references. Each reference fixes the byte cutoff at capture;
63later deltas cannot extend that prefix. Full-content pages use
64`/sessions/{session_id}/active-output/{content_id}` and retain the original
65`snapshot` query while following `next` tokens.
66
67The CLI stages complete prefix hydration before replacing its transcript,
68then restores history and active output before processing later events.
69Provisional chunks retain message identity even after the live display buffer
70settles them. Canonical assistant and thinking publication precedes
71`committed.replaces_live_ids`, which identifies matching provisional chunks.
72Model commits set `replaces_all_live` to retire all provisional output, including
73fragments absent from the bounded projection after a storage failure.
74Retry identities never reuse a failed attempt's content. Cancellation and
75failure retire remaining provisional output without changing durable history.
76
77Issued spill references live for 15 minutes across commit and retry. Existing
78maintenance removes expired and orphaned files; session deletion revokes them.
79Spill storage is bounded to 64 MiB per run, 1 GiB across retained files, and
804096 retained files. A projection has at most 256 segments. Spill directories
81use mode `0700`; content and metadata use `0600`. Owner identity includes the
82VM incarnation so reused PIDs cannot retain orphaned files after restart.
83Storage failure makes backfill explicitly unavailable rather than returning a
84truncated successful snapshot. It must not abort generation or discard a
85durable commit. Expired references return `410 active_output_expired` and cause
86the client to capture a new snapshot. Do not add compatibility readers for
87snapshots without active output.
88
89Text and thinking deltas reach subscribers at most once a frame (16 ms):
90`session_state.emit` publishes a delta after a quiet frame at once and holds
91later ones until the frame ends, merging neighbours of one message into one
92event. Any other event, a commit, a retry, and the end of a turn publish the
93held deltas first, so the stream keeps its order and active output never lags
94a commit. The replay window keeps at most 256 events and less than one
95response's bytes, so any replay fits a single batch.
96
97Session subscribers receive coalesced wake notifications. Event payloads stay
98in the session's existing replay buffer; a successful frame acknowledges its
99cursor before another wake is issued. A subscriber's death removes its watch
100without requiring another event. A session that stops (closed at shutdown or
101deleted) wakes its subscribers, and a stream whose session stopped or did not
102answer ends with a clean EOF rather than a failure batch: the daemon closes
103sessions before it stops listening, so that is a restart or a deletion, and the
104client's reconnect learns which.
105
106The CLI validates the whole batch before consuming it and saves both cursor
107values only after every callback succeeds. Initial attachment and a generation
108change require a leading reset. Within one generation, the sequence cannot
109decrease without a reset. A generation change with a reset is normal recovery.
110A change without a reset is a protocol failure. The reset replaces visible
111history and clears live tool progress. Reset batches include `current_progress` in the captured session snapshot. The CLI delivers that snapshot after history as
112live progress, so attaching during a tool call restores its current tool name,
113phase, and bounded code preview. Older-history navigation uses transcript row IDs.
114Unknown event kinds are ignored.
115Missing kinds and malformed known events are protocol failures.
116
117Transport failures, ordinary EOF, HTTP 408/429, and server errors reconnect with
118a delay starting at 500 milliseconds and increasing to at most five seconds.
119EOF and transient failures retain the saved pair. Cancellation ends the
120subscription quietly and clears the pair, as does explicit protocol recovery.
121Other HTTP refusals, explicit SSE failures, and consumer callback failures end
122the subscription with a visible notice, except an SSE failure coded
123`daemon_unavailable` or `daemon_stopping`, which older daemons send while they
124shut down and which reconnects like a server error. A terminal failure also stops status
125polling for that attachment.
126
127The TUI attempts one durable transcript reset per attachment after a protocol
128failure. It clears live tool progress and requires the recovery stream to
129begin with a reset. A second protocol failure ends automatic recovery; reopening
130the session starts a new attachment. Pending submissions and drafts remain
131available, and a stream failure does not claim that model execution finished.
132
133## Notes in the transcript
134
135The TUI renders daemon notes as faint plain text, wrapping each line before
136styling it. Never style a whole multi-line note as one Lip Gloss block: that
137pads every short line to the longest line, which can span thousands of columns
138in a capability update. Long notes use the same click-to-expand fold as user
139messages. Folding changes only the display, not the stored or model-facing text.
140
141## Loaded sessions and unloading
142
143A session is loaded while its actor runs; only a loaded session has a stream
144cursor. Opening it, watching its stream, or submitting to it loads it. Reading
145`GET /sessions/{session_id}/history` never does, so the session list previews
146stored history instead of the session resource.
147
148The maintenance sweep releases an idle kernel after `ALBEDO_IDLE_SECONDS`
149(default 10 minutes) and unloads a session whose last turn (`activity_at`) is
150older than `ALBEDO_UNLOAD_SECONDS` (default one hour): the provider's prompt
151cache has expired by then. The session refuses while a turn runs or waits, a
152client watches its stream, a background job is live, or one of its children
153works — runs a turn, or sits between turns on a job of its own, or has a child
154that works: that child's report will wake it, and its prompt cache may be kept
155warm meanwhile (robot-docs/cache-warming.md). Unloading saves the kernel's
156variables like a release. The next request loads it again.
157
158Listing sessions reads loaded sessions' summaries but does not count as
159attention, so an open session list never keeps a kernel alive.
160
161The same sweep collects every waiting process whose heap passed 1 MB
162(`albedo_daemon:collect_idle`): a long-lived actor otherwise keeps the heap a
163busy turn grew it to. The launcher's VM defaults (`priv/bin/albedo-daemon`)
164then hand the freed carriers back to the OS instead of caching them.
165
166The session list marks a session active (green) while it is loaded and its
167last turn is under an hour old.
168
169## asynchronous persistence failures
170
171The dispatcher logs failures to advance delivered schedule occurrences or save mail delivery errors. Undelivered mail and unadvanced schedules remain durable and retry under the existing dispatch policy. A schedule whose submission succeeded but whose advance failed can be delivered again.
172
173## Prompt ownership and cancellation
174
175Inputs use `PUT /sessions/{session_id}/inputs/{input_id}` with a client-generated
176UUIDv7. The same identity follows admission, transcript consumption, turn
177membership, and cancellation. Accepted inputs survive restart. The
178[input contract](../docs/http-api-design.md#input-admission-and-outcomes)
179defines message, continuation, skill, and command variants.
180
181`POST /sessions/{session_id}/inputs/{input_id}/cancel` returns the current input
182and `cancelled`, `interrupt_requested`, `shared_running`, or `not_pending`.
183A waiting cancellation commits its receipt and pending removal together. A
184running turn shared with other inputs continues. Cancellation never falls back
185to a session-wide interruption or affects a later unrelated turn.
186
187The CLI uses a separate five-second cleanup deadline and reports unconfirmed
188cancellation when cleanup fails. An interruption acknowledgement does not claim
189that work stopped.
190
191Live turn membership precedes worker output and expands when queued input
192steers a turn. Completion follows its final output or failure. The durable input
193read retains membership and terminal outcome across reconnects and restarts.
194Prompt clients follow their input identity instead of inferring completion from
195echo metadata or idle status. Drafts and unresolved input handles survive resets.