Add a repository-owned rustc wrapper that enforces execution ownership and dependency boundaries during ordinary Cargo commands, with compile-pass and compile-fail policy contracts.
Move scheduling, timers, provider polling, provisioning, recovery, supervision, and shutdown decisions behind engine and actor APIs. Add deterministic component properties, stateful Myelin lifecycle coverage, persisted regression cases, and the bounded CI workflow.
Tighten resource ownership by cancelling telemetry collectors, terminating reply observers, bounding dashboard projections, and releasing process file descriptors, child observers, and inode-verified Unix socket paths on every exit path.
Replace the single-purpose chat runtime with a persistent fleet daemon that provisions, adopts, and controls nodes through the dashboard.
Add distributed job-runner actors and provider-backed deployment so jobs can materialize workspaces, execute remotely, and return outputs over iroh.
The crate is the per-node metrics/logging pipe with a universal
subscriber endpoint, but "datastream" kept getting misread as a general
messaging plane. Rename crate, module paths, and public API
(`DatastreamEndpoint` → `TelemetryEndpoint`, etc.) so misuse is visible
on sight.
Renamed contracts (all in-repo producers/consumers migrated):
- env vars `MYELIN_DATASTREAM_*` → `MYELIN_TELEMETRY_*`
- artifact `datastream.ndjson` → `telemetry.ndjson`
- actor names `telemetry-publisher` / `telemetry-sink`
- wire ALPN `swactor/telemetry/0`
- `DATASTREAM_SPEC.md` → `TELEMETRY_SPEC.md`
Also fixes two latent test breaks: `process` and `iroh-driver` tests
imported `DatastreamEvent` from the crate root, which was never
re-exported; they now use the observer path `telemetry::frame::`.
Replace pointwise scenario testing with a reusable conformance kit in
tests/common: a deterministic trace harness (input alphabet, seeded
generator, naive shrinker), an invariant oracle covering twenty black-box
guarantees (identity, correlation, dead-hold, attempt-fact ownership,
quiescence no-op, monotonic generation, fair convergence, bounded
replacement), and a fair-scheduler tail asserting eventual reconciliation.
Three conformance levels run the same battery:
- FakeBackend: the reference in-memory substrate (256 seeds x 2 modes)
- PluginBackendAdapter over FakePlugin: seam contracts plus the battery
- ProcessPlugin: real child processes, faults as real signals/errors;
"no double-create" and "converged leaks nothing" verified by counting
live PIDs (16 seeds)
Also documents two seam findings the battery surfaced: ProvisionPlugin
cannot express ambiguity (kit convention: AMBIGUOUS_FAULT_MARKER error
reclassified by the adapter; definite classification leaks provider
resources) and spawn_effect closures form a spawner Arc cycle that leaks
backends under queue-based spawners (kit breaks it at harness drop).
Introduce a pure, level-triggered reconciler in `crates/provisioning`
that drives a declared cluster shape toward convergence over the
existing node lifecycle, replacing the edge-triggered imperative node
orchestration in `apps/myelin`.
- `reconcile`/`reconcile_node`/`observe`: pure decider and observation
folder with stable logical-node identity, per-attempt operation
identity, and deterministic retry backoff; `ClusterDriver` is the sole
writer of observed state, coalescing triggers, recording operations as
pending before dispatch, and scheduling timed requeues.
- `IdempotentEffectExecutor`: deduplicates submissions by
`(run_id, logical_node_id, attempt)` and runs provider work on the
engine-hosted blocking substrate, never blocking a reconcile pass.
- Myelin integration: `MyelinEffectBackend` bridges `ProvisionPlugin` to
the executor contract; `LocalProcessPlugin`/`LocalDockerPlugin`
provider adapters; `ProvisionedClusterGuard` pumps triggers,
observations, and due operations.
- Retire the imperative acquire/bootstrap/teardown sequencing across
`apps/myelin` orchestration, staging, observability, and provider
adapters in favor of the declarative driver.
- Move the reconciler specification to `docs/specs/archive`.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
