Move iroh_driver and the relay binary out of distribution into a dedicated
crates/iroh-driver (lib re-exports IrohDriver; relay bin renamed). Remove the node crate
and the single-gpu-inference example; drop the docker/datastream demo. Slim
pipeline-parallel vastai.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Promote pipeline-parallel-inference to a first-class app and consolidate observability on the datastream wire, decoupling the dashboard crate from `distribution`.
- apps/pipeline-parallel-inference: move the example out of `examples/` into `apps/` as its own workspace, rename binaries to `pp-worker`/`pp-orchestrator`, and strip release binaries
- cluster: add `ClusterNode`, a synchronous facade over the actorized distribution protocol (IrohDriver + per-node Runtime hosting Swim/Registry/Metadata/Directory actors with a `MembershipFanout`), replacing ad-hoc `driver.node()`/`tick()` call sites
- fleet: add per-node fleet telemetry that ships identity/resource records as `DatastreamFrame`s over the cluster transport to the orchestrator's `DatastreamSink`, folded into a `FleetView` on a 3s tick
- provision: add best-effort, opt-in SSH boot-phase telemetry (`PP_DEPLOY_KEY`) that streams rented-node boot logs onto the orchestrator's datastream as `proc.boot.<stage>.*`
- dashboard: rewire the crate dependency from `distribution` to `datastream`, drop the standalone `swactor-datastream-dashboard` binary, and rewrite `datastream_source.rs` to demux per-node frames into Overview/Distribution/Fleet views with live-node TTL filtering
- distribution: refresh dist/netmap plugin copy and README from "Kademlia routing" to gossip-directory terminology
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Strip the collector/aggregator/postproc/snapshot, vastai sampler+shipper,
host/iroh/subprocess/swim introspection, relay observability, sink/spool, and the diag
binaries; drop the t_diag_* tests. Remove DiagEvent emission from iroh_driver. Add
datastream emit/wire (mux + NoopSink/UdpFrameSink/ClusterFrameSink) and rewire the
dashboard onto datastream_source.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Upgrades iroh/iroh-relay across datastore, distribution, node, and integration,
adapts iroh_driver to the new Endpoint::builder(Minimal).relay_mode(...) API, and
removes the vendored ed25519-dalek patch now that 0.98 resolves the upstream compile
errors. Adds the SIM_SPEC.md simulator MVP spec under pipeline-parallel-inference.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Structured observability for the iroh/SWIM layer: Aggregator, typed Event/Snapshot
types, Sink (NoopSink default), ProbeScheduler, process stats, and host/iroh/swim
introspection, plus the swactor-diag-collector, -postproc, and -iroh-relay binaries
that assemble and render per-run bundles. Generalizes the pipeline-parallel-inference
example to N stages and adds the topology-planner spec.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Remove the unused TCP transport and dead CLI/simulation scaffolding and collapse scattered single-purpose modules into consolidated files across the dashboard, process, and simulation crates.
- crates/transport: drop the TCP transport (src/tcp.rs and its tcp feature), leaving only ed25519 identity and encoding utilities behind the iroh transport
- crates/dashboard: collapse actor_detail_html/actors_html/dashboard_html/topology_html into a single html.rs, drop command/parse.rs, and inline the trace types into lib.rs
- crates/process: fold driver, pipeline_types, queue, subscriber, waker, and local pipes/signal/wait into local/mod.rs, pipeline.rs, and a unified types.rs, consolidating the public re-exports
- crates/simulation: remove the ci subdirectory (local_sim, sim) and dead node/config/trace modules, and flatten the distribution subdirectory into top-level files
- crates/datastore: remove the unused store_cli, cli, and in-memory storage, and deduplicate crypto.rs across datastore and distribution (about 190 lines of shared code removed)
- crates/distribution: drop dead codec code and trim the messages module
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Reduce idle cpu usage on my main machine from 17% to 1%. Made SWIM gossip more lazy.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Remove co-dependencies for different modules found in `crates` and migrate the development history to a new repository. The docs were stale, and largely not getting used, so simply deleted for now. When code stabilizes more, they will become useful again.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Make distribution and deployment more stable. Consolidate the logic for a generic swactor node.
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
Major feature addition. For full details read `./docs/development_history/DISTRIBUTION.md`
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
