Commit graph

24 commits

Author SHA1 Message Date
966497ec07 feat: proper README, LICENSE, and our first quick example
Add project licensing and a full README, plus a minimal WebAssembly ping-pong example that exercises the actor runtime.

- `LICENSE`: add the full GNU AGPL-3.0 text and set `license = "AGPL-3.0-only"` on every package (`swactor`, each crate, `apps/myelin`, `tools/vastai`, `xtask`)
- `README.md`: rewrite from a stub into a full project overview, covering features (actor_id routing, WASM, iroh QUIC/SWIM, OTP-style std, process manager, zero-copy objects, datastream metrics, dashboard), architecture, examples, developing, status, and license
- `examples/ping-pong`: new standalone workspace (`pingpong` cdylib) where two actors volley on a single-threaded `wasm` runtime driven by a Node host via `tick()`, demonstrating spawning, message passing, and death monitoring (`watching`)
- `examples/ping-pong`: add a host/run harness -- `run.sh` (wasm-pack build + `run.mjs`), `serve.sh` (dashboard + static demo), `index.html`, and a pinned `Cargo.lock`
- `.gitignore`: stop ignoring `.loop/`, `.deployment-notes/`, and `.omp/`

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-08-06 00:53:16 +04:00
c257dde02f refactor: fold driver pumps into iroh-driver and codecs into transport
Consolidate the duplicated JSON codec into `transport` and relocate the iroh edge-transport pieces into `iroh-driver`, dissolving the `mvp-system` transport shim.

- `transport`: add a canonical `json_codec::JsonCodec<M>` (re-exported from the crate root) as the single JSON codec for serde message types
- `distribution`/`datastream`: drop the per-crate `JsonCodec` copies and the `impl_json_codec!` macro; register SWIM/gossip and publisher messages against the shared `swactor_transport::JsonCodec`
- `iroh-driver`: move `driver_pumps` and `endpoint_advertisement` out of `mvp-system/src/transport/`, re-exporting `EndpointAddrMask`/`advertised_endpoint`/`MVP_IROH_ENDPOINT_ADDR_MASK_ENV`, and relocate the endpoint guarantee test to `iroh-driver/tests/endpoint_advertisement.rs`
- `mvp-system`: delete the `transport/` module and keep codec aggregation in a new `codecs.rs` (`register_mvp_actor_codecs`)
- `mvp-system/node`: shrink `worker_node_runtime.rs` (~260 lines) by adopting the relocated modules and collapsing verbose `emit_stdio_node_event` calls into local `boot()`/`worker_evt()` closures

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-30 12:18:02 +04:00
f66c5b0d2e feat: verification gate for N pipeline stage deployment
Lift the VastAI two-stage cap and let mvp-chat-check drive an arbitrary N-stage deployment through a new --pipeline-stages flag.

- orchestrator_app: remove the provider=vastai >2 stage cap and replace the two-stage plan test with an eight-stage plan test (8 specs, no mounts, remote GGUF, max-context).
- xtask: refactor scenario parsing into MvpChatCheckInvocation carrying an optional pipeline_stages, parse a `--pipeline-stages n` flag (rejecting 0/missing values), and default multinode/docker to 2 while VastAI uses the explicit count.
- xtask: thread the invocation through run_mvp_chat_check/run_mvp_chat_check_process and update the usage text and scenario tests (including `--vastai --pipeline-stages 8`).
- gitignore: ignore .deployment-notes/.

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-25 11:42:29 +04:00
6f1c048669 refactor: retire old pipeline-parallel app, restructure mvp-system
- Drop the standalone apps/old-pipeline-parallel-inference app and its
  fleet/orchestrator/tests (~24k lines).
- Add datastream::hardware (cpu/gpu/net) modules; add mvp-system config, arena_manager,
  and vastai_offer_preview.
- Rename mvp_orch_one_node->orchestrator and mvp_node->worker_node; expand VastAI
  provisioning; rework xtask runner.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-07 14:40:02 +04:00
a97864f7ec feat(mvp-system): expand one-node mvp binaries and provisioning
Flesh out mvp_node, mvp_one_node_chat, and mvp_orch_one_node binaries. Add node_image
and relay_provisioning; grow provisioning and vastai. Tune datastream mux/timing/emit.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-05 13:59:51 +04:00
0d1b95695d refactor: drop datastore crate, stale specs, and benches
Remove the datastore crate, the top-level design/orchestration/ring specs, the
benches, and the ci config. Add the dashboard host telemetry sampler
(cpu/disk/net/gpu/mem). Localize the pipeline-parallel e2e stub/mock paths.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-06-23 19:42:28 +04:00
c67ce181c2 feat(distribution): add n3 observability introspectors
Relay/subprocess/registry introspectors, build-time dep-version + git-sha, postproc
RTT/inference render, new swactor-iroh-relay binary; dashboard replay-viewer, sim SWIM
tuning report + repro tests, N3 postmortem docs.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-05-25 22:19:06 +04:00
db0ed79e20 feat: guarantees on runtime execution (#53)
Using a bounded model checker to provide absolute guarantees on runtime behavior.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-03-28 05:08:58 +00:00
599678e7c9 feat: cluster test improvements; datastore pools (#45)
Introduce a gossip-converged pooled datastore protocol layered on SWIM piggybacking, backed by a reusable gossip-channel abstraction, plus a Docker-free multi-process cluster test runner.

- crates/distribution/src/gossip_channel.rs: add the GossipChannel trait (piggyback on SWIM messages) and a budget-limited DisseminationBuffer<T> that replaces the four duplicated Lambda*ceil(log2(n)) dissemination copies
- crates/datastore/src/pool: add PoolDisseminator (CRDT state for membership/capacity/content-location/ACL with join/leave/announce) and PoolCoordinator (placement-aware CRUD actor delegating to the co-located DatastoreNode)
- crates/shared-types/src/pool.rs: add shared pool protocol types (PoolId plus member/capacity/content-location/ACL entries and PoolConfig) consumed by both distribution and datastore
- crates/dashboard/src/pool_html.rs: add a live pool dashboard page (membership, capacity, content locations) and add pool_tests integration coverage
- xtask/src/sim_cluster.rs: add the sim-cluster runner that spawns N swactor nodes over iroh through a local relay server, reusing the docker cluster scenarios without Docker
- crates/distribution/src/iroh_driver.rs: add relay-URL resolution (cache, then SWIM gossip, then home relay) with a 2s connect timeout to back the relay-based connections

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-20 17:30:37 +00:00
c6cb88e335 feat: stability for deployment and distribution (#44)
Make distribution and deployment more stable. Consolidate the logic for a generic swactor node.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-19 14:39:33 +00:00
9f4ccdc151 feat: MVP authorization layer for swactor datastores (#43)
Very barebones, untrustworthy, barely reviewed auth layer. LGTM.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-16 15:39:27 +00:00
9b83b773de feat: distribution simulation tests (#34)
Test cluster behavior in simulation.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-13 13:18:38 +00:00
c1feddcab4 feat: realize distribution crate (#33)
Stand up a runnable distribution stack on top of the core node logic.

- distribution: add NodeDriver bridging DistributedNode to real TCP I/O
  (TcpTransport/TcpAcceptor), translating NodeActions to/from wire messages;
  refine swim probe timing and transport wiring.
- node: new swactor-node binary (crates/node) hosting a real node over TCP.
- tests/docker: multi-host LAN cluster harness (compose, run-lan-cluster.sh,
  cluster + lan_cluster integration tests) exercising the full SWIM path.
- simulation: cluster_scenarios integration + distribution property coverage.
- docs: reorganize into distribution/, runtime/, diagrams/, connectome/; add
  DOCKER_REALIZATION + SIMULATION_TESTING realization notes.

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-13 07:55:12 +00:00
4e56590f05 feat: wasm runner actor skeleton (#32)
Lay down the wasm-actor host, a frontend-agnostic command layer, and a
distribution registry.

- command (new crate): CommandRouter dispatching to built-in inspection handlers
  (overview/workers/actors) plus user-registered handlers, with line and
  query-param parsers; built for REPL/REST/TUI/WebSocket frontends.
- wasm-actor (new crate): skeleton host — WasmActor, Builder, Engine, error
  types — with echo/double/silent guest fixtures and integration tests.
- distribution: add Registry (member catalog + lookups) and Snapshot, with tests.
- core: extend the worker watch API; add watch_api integration tests.

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-13 07:42:44 +00:00
9c866b51a6 feat: transport protocol (#27)
Address actors via ID, send messages over transport (TCP, QUIC, etc)


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-09 19:05:37 +00:00
ff9710d724 feat: gossip simulation (#21)
Simulate a simple push-pull epidemic broadcast.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-08 16:18:39 +00:00
01fc5b6b11 feat: fuzz-harness (#20)
Add a cargo-fuzz harness exercising the runtime under arbitrary actor spawn/send/inbox sequences in both single- and multi-threaded modes.

- fuzz/Cargo.toml: add an isolated `swactor-fuzz` package (own standalone workspace) wiring libfuzzer-sys + arbitrary against the `swactor` crate, with `fuzz_runtime` and `fuzz_mt` bin targets
- fuzz/fuzz_targets/fuzz_runtime.rs: single-threaded model target running randomized scenarios over Echo/Counter/Forwarder/Spawner/Bomber/Noop/WrongType actors with shared-inbox draining and invariant checks
- fuzz/fuzz_targets/fuzz_mt.rs: multi-threaded target driving a live `Runtime`/`RuntimeHandle` across 2-4 threads with spawn/send/burst/inbox actions and `check_invariants` (worker/actor counts) after each run
- fuzz/fuzz_targets: both targets derive a `FuzzInput`/`Action` model from `arbitrary`, cap the action count, and support opt-in run tracing via the `FUZZ_LOG` env var
- .gitignore: ignore fuzz corpus and artifact output

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-07 18:30:22 +00:00
4371642a27 refactor: repack external bindings into their own crates (#19)
Split the monolithic crate into a Cargo workspace with the Python and Wasm bindings as separate member crates.

- Cargo.toml: declare a `[workspace]` with members `.`/`crates/swactor-python`/`crates/swactor-wasm`, remove the `python` feature and pyo3 dependency, and change root crate-type from `["cdylib","rlib"]` to `["rlib"]`
- crates/swactor-python: new cdylib crate re-exporting the PyO3 bindings (Runtime/RuntimeConfig/RuntimeHandle/Inbox/Ctx/ActorAddress/RuntimeStats), depending on `swactor` + pyo3; pyproject.toml and uv.lock relocated here from the root
- crates/swactor-wasm: new cdylib crate moved from top-level `wasm/`, depending on `swactor` with `no_random` features
- src/actor.rs: widen `Actor::new`, `AnyActor`, `ContextInner`, and `Ctx::raw_inner` to `pub` so the separate binding crates can drive the runtime
- src/lib.rs: delete the in-tree `python` module and `#[pymodule]`, and gate the `no_random` RNG behind `all(feature = "no_random", not(feature = "getrandom"))`
- tools/: relocate package.json/package-lock.json; drop the now-duplicate `wasm/Cargo.lock`

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-07 17:39:02 +00:00
a12323c94f refactor: better tests (#18)
Still unsatisfied, but these are better than before.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-07 17:29:01 +00:00
c9d1383d1c feat: basic wasm (#12)
Unfeatured wasm module. Mostly to test that wasm builds


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-06 16:47:43 +00:00
a01c3eb5f9 feat: code DAG visualizer (#11)
Visualize the inter-module code dependency DAG


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-06 14:54:56 +00:00
65614e14d3 feat: python bindings (#7)
Python bindings allowing us to interact with the library in a python REPL


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-06 12:47:51 +00:00
e2c4f55941 refactor: major library changes (#5)
Refactoring to logically separate component modules in order to make it easier to develop tests, metrics, tracing, etc.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-02-06 11:25:37 +00:00
Zachery Aaron Shores-Chmielewski
88121780ce init
Skeletal actor framework. Somewhat unweildy, needs a message box, a better runtime, and different channels. However, hello world example works


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2025-11-24 19:39:17 -05:00