Zachery Aaron Shores-Chmielewski
ae9ca3bcf3
Promote pipeline-parallel-inference to a first-class app and consolidate observability on the datastream wire, decoupling the dashboard crate from `distribution`. - apps/pipeline-parallel-inference: move the example out of `examples/` into `apps/` as its own workspace, rename binaries to `pp-worker`/`pp-orchestrator`, and strip release binaries - cluster: add `ClusterNode`, a synchronous facade over the actorized distribution protocol (IrohDriver + per-node Runtime hosting Swim/Registry/Metadata/Directory actors with a `MembershipFanout`), replacing ad-hoc `driver.node()`/`tick()` call sites - fleet: add per-node fleet telemetry that ships identity/resource records as `DatastreamFrame`s over the cluster transport to the orchestrator's `DatastreamSink`, folded into a `FleetView` on a 3s tick - provision: add best-effort, opt-in SSH boot-phase telemetry (`PP_DEPLOY_KEY`) that streams rented-node boot logs onto the orchestrator's datastream as `proc.boot.<stage>.*` - dashboard: rewire the crate dependency from `distribution` to `datastream`, drop the standalone `swactor-datastream-dashboard` binary, and rewrite `datastream_source.rs` to demux per-node frames into Overview/Distribution/Fleet views with live-node TTL filtering - distribution: refresh dist/netmap plugin copy and README from "Kademlia routing" to gossip-directory terminology Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
39 lines
1.6 KiB
YAML
39 lines
1.6 KiB
YAML
# Diagnostics e2e stack.
|
|
#
|
|
# The collector service runs in its own container with the bundles dir
|
|
# bind-mounted from the host so the harness can read the finalized
|
|
# tarball. Both the collector and the pp processes use the host network
|
|
# namespace, so the stage children (spawned by `pp-orchestrator` via
|
|
# `docker-gpu-node.sh`) share localhost reachability with the
|
|
# collector — `SWACTOR_DIAG_COLLECTOR_URL=http://127.0.0.1:9080` works
|
|
# uniformly from every actor in the run.
|
|
#
|
|
# The host network choice mirrors the existing `docker-e2e.sh` shape:
|
|
# pp-orchestrator runs on the host (orchestrator) and each
|
|
# `pp-worker` runs in its own container under `--network host`. The
|
|
# collector container just adds one more service to that arrangement.
|
|
#
|
|
# Used by `scripts/docker-diag-e2e.sh`. Direct `docker compose up`
|
|
# without the wrapper is fine for manual collector-only operation, but
|
|
# the orchestrator-side wiring (env-var injection, bundle assertions)
|
|
# lives in the script.
|
|
|
|
services:
|
|
collector:
|
|
image: ${PP_DIAG_IMAGE:-swactor-pp-gpu:latest}
|
|
container_name: ${PP_DIAG_COLLECTOR_NAME:-pp-diag-collector}
|
|
network_mode: host
|
|
# Override the image's default entrypoint (pp_entrypoint.sh, which runs
|
|
# pp-worker) so this container runs the collector instead. The
|
|
# diagnostics binaries ship in the same unified code image.
|
|
entrypoint: /usr/local/bin/swactor-diag-collector
|
|
command:
|
|
- --bind
|
|
- 127.0.0.1:9080
|
|
- --root
|
|
- /var/lib/swactor-diag
|
|
- --udp
|
|
- 127.0.0.1:9081
|
|
volumes:
|
|
- ${PP_DIAG_BUNDLES_DIR:?PP_DIAG_BUNDLES_DIR must be set}:/var/lib/swactor-diag
|
|
restart: "no"
|