Add durable namespace control, Iroh-backed source transfer, session-independent publication lifetime, and wire managed Myelin jobs and bindings through the reusable data-plane API.
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
Architecture enforcement:
- Install a repository-owned rustc wrapper for ordinary cargo check,
build, and test commands. Resolve compiler item identities so renamed
imports and helper wrappers cannot hide spawning, timing, blocking,
polling, thread, or runtime-driving capabilities.
- Define the execution-owner crates and reject dependencies from those
substrates back into Myelin policy. Add compile-pass and compile-fail
contracts for actor helpers, execution owners, test waits, forbidden
capabilities, suppression attempts, and owner dependency inversions.
Execution ownership:
- Add engine-owned actor timers with cancellation and generation identity,
then migrate lifecycle deadlines and protocol ticks off application
tasks. Keep networking, process output, telemetry, and blocking provider
calls in their approved I/O substrates.
- Move process spawn, wait, signal, Unix listener, and output-following
mechanics into swactor-process. Isolate Vast.ai blocking HTTP mechanics
behind its adapter while actors retain retry, recovery, and provisioning
decisions.
Myelin control flow:
- Rework manual control, worker lifecycle, provisioning, provider recovery,
job deployment, distribution, edge orchestration, and shutdown as actor
state transitions and typed effects. Preserve durable provider adoption
and command outcomes across graceful and abrupt restarts.
- Replace controller loops and timer-forwarding tasks with actor messages;
leave substrate tasks as cancellable observation streams with no durable
policy state.
Properties and resource ownership:
- Add deterministic engine and component properties, a stateful mock-VastAI
lifecycle model, persisted regression cases, controlled fault injection,
and a bounded nightly workflow covering restart and teardown behavior.
- Terminate reply observers, cancel telemetry collectors, bound dashboard
projections, and release child observers, file descriptors, process
records, and inode-verified Unix sockets on every terminal path.
Verified with the compiler-policy contracts, 105 Myelin library tests, 32
swactor-process tests, telemetry cancellation contracts, randomized
stateful restart cases, cargo check, and formatting checks.
Add actor-backed manual node provisioning, control-plane endpoints, and fleet UI assets with durable provider lifecycle handling.
Simplify Myelin orchestration, node runtime, staging, and telemetry paths while removing obsolete engine-builder, dashboard-view, and local-mock implementations.
Align runtime delivery, data-plane, distribution, job-runner, process, telemetry, dashboard, Vast.ai integrations, and their tests with the revised actor and transport contracts.
Core no longer owns or drives OS threads. The runtime is now a single
tick-driven worker whose loop an external engine hosts and advances.
This is the cutover required before the engine seam is introduced.
Removed from core:
- Runtime::run() and its owned thread pool (spawn, park/unpark, join)
- notify_worker() and worker_threads: Vec<OnceLock<Thread>> plumbing
- Placement load-aware selector and its WorkerStats-driven next_worker()
- WorkerId newtype and the address->worker routing map; AddressMap is
now a plain AddrSet membership set
- num_threads from RuntimeConfig
Rewired for the single-worker tick API:
- python/wasm bindings, dashboard dummy node (deleted), myelin vastai
adapter, and the runtime/test suites
Cleanup folded in during review:
- prune three never-written WorkerStats counters (cross_sends,
messages_dropped, restarts)
- collapse the redundant tick_all params onto the WorkerContext handle
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Add project licensing and a full README, plus a minimal WebAssembly ping-pong example that exercises the actor runtime.
- `LICENSE`: add the full GNU AGPL-3.0 text and set `license = "AGPL-3.0-only"` on every package (`swactor`, each crate, `apps/myelin`, `tools/vastai`, `xtask`)
- `README.md`: rewrite from a stub into a full project overview, covering features (actor_id routing, WASM, iroh QUIC/SWIM, OTP-style std, process manager, zero-copy objects, datastream metrics, dashboard), architecture, examples, developing, status, and license
- `examples/ping-pong`: new standalone workspace (`pingpong` cdylib) where two actors volley on a single-threaded `wasm` runtime driven by a Node host via `tick()`, demonstrating spawning, message passing, and death monitoring (`watching`)
- `examples/ping-pong`: add a host/run harness -- `run.sh` (wasm-pack build + `run.mjs`), `serve.sh` (dashboard + static demo), `index.html`, and a pinned `Cargo.lock`
- `.gitignore`: stop ignoring `.loop/`, `.deployment-notes/`, and `.omp/`
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Remove the datastore crate, the top-level design/orchestration/ring specs, the
benches, and the ci config. Add the dashboard host telemetry sampler
(cpu/disk/net/gpu/mem). Localize the pipeline-parallel e2e stub/mock paths.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Reduce idle cpu usage on my main machine from 17% to 1%. Made SWIM gossip more lazy.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Remove co-dependencies for different modules found in `crates` and migrate the development history to a new repository. The docs were stale, and largely not getting used, so simply deleted for now. When code stabilizes more, they will become useful again.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
