2026-07-28 07:29:31 +00:00
|
|
|
[package]
|
|
|
|
|
name = "data-plane"
|
|
|
|
|
version = "0.1.0"
|
|
|
|
|
edition = "2024"
|
feat: proper `README`, `LICENSE`, and our first quick example
Add project licensing and a full README, plus a minimal WebAssembly ping-pong example that exercises the actor runtime.
- `LICENSE`: add the full GNU AGPL-3.0 text and set `license = "AGPL-3.0-only"` on every package (`swactor`, each crate, `apps/myelin`, `tools/vastai`, `xtask`)
- `README.md`: rewrite from a stub into a full project overview, covering features (actor_id routing, WASM, iroh QUIC/SWIM, OTP-style std, process manager, zero-copy objects, datastream metrics, dashboard), architecture, examples, developing, status, and license
- `examples/ping-pong`: new standalone workspace (`pingpong` cdylib) where two actors volley on a single-threaded `wasm` runtime driven by a Node host via `tick()`, demonstrating spawning, message passing, and death monitoring (`watching`)
- `examples/ping-pong`: add a host/run harness -- `run.sh` (wasm-pack build + `run.mjs`), `serve.sh` (dashboard + static demo), `index.html`, and a pinned `Cargo.lock`
- `.gitignore`: stop ignoring `.loop/`, `.deployment-notes/`, and `.omp/`
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-08-05 20:29:27 +00:00
|
|
|
license = "AGPL-3.0-only"
|
2026-07-28 07:29:31 +00:00
|
|
|
publish = false
|
|
|
|
|
|
|
|
|
|
[dependencies]
|
2026-08-15 08:17:48 +00:00
|
|
|
telemetry = { path = "../telemetry" }
|
refactor: data movement handled by an independant crate
Promote the data-plane into its own crate with a swactor-facing DataPlaneNodeActor that owns wire-edge lifecycle, and move the edge/ring/egress/ingress machinery out of mvp-system into crates/data-plane.
- data-plane/src/actor.rs: add the 769-line DataPlaneNodeActor implementing ActorInterface, owning WireEdgeEndpoint provisioning and the EdgeEstablisher lifecycle and draining commands to arena/worker/transport actors (DataPlaneArenaMsg/WorkerMsg/TransportMsg) with DataPlaneReportMsg back to a report sink
- data-plane/src/lib.rs: expand the crate surface to expose actor, arena, edge_actor, edge_lifecycle, egress, and ingress alongside object_record/ring, and reframe it as actor-oriented wire-edge / ring / arena / object-movement contracts
- data-plane: move edge_actor and edge_lifecycle (from node/), egress (from worker/), and ingress (from node_data/) into the crate, and add ArenaSample (serde Record, ARENA_SAMPLE_CHANNEL/INTERVAL) to arena.rs
- data-plane: add DATA_PLANE_ACTOR_SPEC.md and the data_plane_actor_guarantees/edge_lifecycle_guarantees/egress_guarantees tests
- mvp-system/node: add data_plane_bridge.rs wiring the node runtime to the DataPlaneNodeActor, drop the old node_data/arena.rs and node_data/mod.rs (now in data-plane), and remove the arena_manager_guarantees test
- mvp-system: drop node_data from the lib.rs pub surface and repoint node/mod.rs to consume the data-plane crate
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-28 17:30:56 +00:00
|
|
|
serde = { version = "1", features = ["derive"] }
|
2026-08-22 17:16:56 +00:00
|
|
|
serde_json = "1"
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
sha2 = "0.10"
|
|
|
|
|
swactor = { path = "../..", features = ["serde", "transport"] }
|
|
|
|
|
swactor-transport = { path = "../transport" }
|
|
|
|
|
swactor-engine = { path = "../engine" }
|
2026-07-28 07:29:31 +00:00
|
|
|
|
feat(data-plane): add routed SPSC stream transport
Implement namespace-owned stream rendezvous with incarnation isolation, cancellation, recovery, bounded local byte rings, and host/child actor protocols.
Add a replaceable stream transport boundary plus an Iroh ALPN adapter with framed records, backpressure, reconnect handling, and terminal propagation.
Wire stream sources and sinks through Myelin job deployment and Python bindings, with guarantee coverage across blob, namespace, job, byte-ring, and transport paths.
Sanitize nested Cargo build context in xtask so lint, Nextest, doctest, and maturin invocations retain stable fingerprints.
2026-08-23 08:47:31 +00:00
|
|
|
parking_lot = "0.12"
|
Move all edge logic into data-plane; reduce iroh-driver to a byte-transport port
Duty mixing between iroh-driver and data-plane is resolved: the transport
crate now owns only byte pumping, and the data-plane owns every edge
semantic.
data-plane:
- ids.rs: single EdgeId/RingId/StreamId/NodeId/RunId/LeaseRequestId/
ActorAddress definitions; arena, edge_lifecycle, and ring re-export them
(previously duplicated per module)
- edge_wire.rs: the whole transport contract — WireEvent, EdgeWriter, and
the EdgeTransport port (associated Writer/PeerAddr types)
- edge_runtime.rs: EdgeRuntime composition engine absorbing iroh-driver's
driver_pumps bookkeeping, the EdgeEstablisher lifecycle drive, arena
leasing, ingress stream buffering with object-record parsing, and ring
writes; effects go through a WorkerPort trait; progress surfaces as
structured Observations the application maps to telemetry/agent messages
- delete superseded test-only layers: actor.rs (DataPlaneNodeActor),
edge_actor.rs, ingress.rs, egress.rs and their guarantee tests
- fold ObjectIdAllocator into object_record (now edge-free, starts at 1)
iroh-driver:
- edge_transport speaks pure data_plane::edge_wire vocabulary; EdgeSendHandle
implements EdgeWriter; IrohDriver implements EdgeTransport (PeerAddr =
EndpointAddr) — the entire edge surface is open_writer + drain_events
- delete driver_pumps.rs; new dependency on data-plane (no cycle)
- IROH_DRIVER_SPEC §5 updated for the new module set and edge boundary
myelin:
- WorkerEdgeRuntime shrinks from ~830 lines of hand glue to an EdgeRuntime
holder plus a tinygrad WorkerPort impl and observation reporting; the
driver-event/edge-event translation layers and newtype re-wrapping are
gone
- orchestration/app.rs and job edge drains consume WireEvent
Tests: data-plane 31 (5 new EdgeRuntime contract tests), iroh-driver 13,
myelin 65 — all green.
2026-08-16 17:49:45 +00:00
|
|
|
|
|
|
|
|
[dev-dependencies]
|
|
|
|
|
parking_lot = "0.12"
|
2026-08-24 15:22:20 +00:00
|
|
|
criterion = { version = "0.5", default-features = false }
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
futures-lite = "2"
|
2026-08-22 17:16:56 +00:00
|
|
|
proptest = "1"
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
tokio.workspace = true
|
2026-08-24 15:22:20 +00:00
|
|
|
|
|
|
|
|
[[bench]]
|
|
|
|
|
name = "descriptor_performance"
|
|
|
|
|
harness = false
|
|
|
|
|
|
2026-07-28 07:29:31 +00:00
|
|
|
[target.'cfg(target_os = "linux")'.dependencies]
|
|
|
|
|
libc = "0.2"
|
2026-08-22 17:16:56 +00:00
|
|
|
|
|
|
|
|
[lints]
|
|
|
|
|
workspace = true
|