zacheryasc/swactor
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
139 commits 3 branches 0 tags 42 MiB
Commit graph

3 commits

Author SHA1 Message Date
Zachery Aaron Shores-Chmielewski
b3dbd7ce11 refactor: datastream crate is now telemetry 
The crate is the per-node metrics/logging pipe with a universal
subscriber endpoint, but "datastream" kept getting misread as a general
messaging plane. Rename crate, module paths, and public API
(`DatastreamEndpoint` → `TelemetryEndpoint`, etc.) so misuse is visible
on sight.

Renamed contracts (all in-repo producers/consumers migrated):
- env vars `MYELIN_DATASTREAM_*` → `MYELIN_TELEMETRY_*`
- artifact `datastream.ndjson` → `telemetry.ndjson`
- actor names `telemetry-publisher` / `telemetry-sink`
- wire ALPN `swactor/telemetry/0`
- `DATASTREAM_SPEC.md` → `TELEMETRY_SPEC.md`

Also fixes two latent test breaks: `process` and `iroh-driver` tests
imported `DatastreamEvent` from the crate root, which was never
re-exported; they now use the observer path `telemetry::frame::`.
 2026-08-15 12:18:56 +04:00
Zachery Aaron Shores-Chmielewski
b8aff00dc1 enforce datastream telemetry-only invariant: ban frame types from control code 
The datastream is metrics/logging only; control decisions must never branch
on a frame.  This was a recurring cultural problem with no structural
enforcement.  This change makes it a compile-time and CI-enforced fact.

datastream crate (lib.rs):
- Stop re-exporting Frame, DatastreamEvent, FrameDelivery at crate root.
   is now a compile error (E0425).  These types live
  only in datastream::frame::* and are documented as the observer surface.
- Safe identity types (ChannelId, StreamId, Position, Record, etc.) remain
  re-exported at root for producer-side callers.

orchestration/app.rs:
- Extracted all frame-touching code (CollectedDatastreamFrame,
  drain_datastream_connections, update_load_progress_from_frame,
  drain_frames, archive_collected_frame, pump, OrchDatastream,
  DashboardSupport) into two new observability modules:
  frame_collector.rs and orch_datastream.rs.
- The orchestrator now interacts through a FrameCollector whose
  drain/drain_with_progress methods take closures; it never names Frame,
  DatastreamEvent, or CollectedDatastreamFrame.
- StageLoadProgress (the one control-relevant signal previously scraped
  from frame payloads) is extracted inside FrameCollector and handed to
  the control loop as plain data.

xtask:
- New check-telemetry-isolation command scans control-plane modules
  (orchestration/, distribution/, data-plane/, provisioning/) for
  forbidden frame-type references and fails the build if any are found.

Verified: workspace builds (myelin + dashboard feature), datastream 29
tests pass, myelin 64 lib tests pass, check-telemetry-isolation passes
clean.

Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
 2026-08-12 16:14:57 +04:00
Zachery Aaron Shores-Chmielewski
ce57012e11 feat: rework datastream into catalog events, add gguf metadata 
- Rework the datastream endpoint into a catalog/event model (DatastreamEvent,
  SubscriptionRequest, channel/stream descriptors, DatastreamPublisherActor) across
  endpoint/frame/mux/wire/views.
- Add gguf_metadata planning reader, a dashboard hardware view, and iroh-driver
  datastream transport.
- Grow mvp-system orchestrator/worker_node bins and staging/provisioning; rename
  mvp_one_node_chat->mvp_chat; extend tinygrad worker.


Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
 2026-07-12 10:14:34 +04:00