feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
#![cfg(target_os = "linux")]
|
|
|
|
|
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
use std::os::fd::{AsRawFd, RawFd};
|
|
|
|
|
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
use data_plane::arena::{ArenaConfig, ArenaManager, NodeId};
|
|
|
|
|
use data_plane::bootstrap::{
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
self, BOOTSTRAP_MAGIC, BOOTSTRAP_VERSION, BootstrapError, BootstrapHeader, ControlRegion,
|
2026-08-27 22:09:26 +00:00
|
|
|
HEADER_END_OFFSET, HEADER_LEN, parse_bootstrap,
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
};
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
use data_plane::mapped_arena::MappedArena;
|
|
|
|
|
|
|
|
|
|
const ARENA_BYTES: u64 = 1 << 20;
|
|
|
|
|
const GENERATION: u64 = 41;
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
|
|
|
|
|
fn arena() -> ArenaManager {
|
|
|
|
|
ArenaManager::boot(ArenaConfig {
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
node_id: NodeId(7),
|
|
|
|
|
reservation_ceiling: ARENA_BYTES,
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
base_alignment: 64,
|
|
|
|
|
})
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
.expect("arena")
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fn spec() -> bootstrap::BootstrapSpec {
|
|
|
|
|
bootstrap::BootstrapSpec {
|
|
|
|
|
arena_generation: GENERATION,
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
alignment: 64,
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fn header() -> BootstrapHeader {
|
|
|
|
|
BootstrapHeader {
|
|
|
|
|
arena_size: ARENA_BYTES,
|
|
|
|
|
arena_generation: GENERATION,
|
|
|
|
|
control_region: None,
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fn fd_flags(fd: RawFd) -> i32 {
|
|
|
|
|
// SAFETY: F_GETFD only inspects a live descriptor table entry.
|
|
|
|
|
unsafe { libc::fcntl(fd, libc::F_GETFD) }
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fn v2_round_trip_contains_only_arena_identity() {
|
|
|
|
|
let encoded = header().encode();
|
|
|
|
|
let resolved = parse_bootstrap(&encoded, ARENA_BYTES).expect("valid bootstrap");
|
|
|
|
|
|
|
|
|
|
assert_eq!(resolved.arena_size, ARENA_BYTES);
|
|
|
|
|
assert_eq!(resolved.arena_generation, GENERATION);
|
|
|
|
|
assert_eq!(resolved.control_region, None);
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
assert_eq!(
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
u32::from_le_bytes(encoded[0..4].try_into().unwrap()),
|
|
|
|
|
BOOTSTRAP_MAGIC
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
);
|
|
|
|
|
assert_eq!(
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
u16::from_le_bytes(encoded[4..6].try_into().unwrap()),
|
|
|
|
|
BOOTSTRAP_VERSION
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
);
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
assert!(encoded[24..].iter().all(|byte| *byte == 0));
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fn private_control_region_is_bounds_checked_when_present() {
|
|
|
|
|
let control = ControlRegion {
|
|
|
|
|
offset: HEADER_END_OFFSET,
|
|
|
|
|
length: 128,
|
|
|
|
|
};
|
|
|
|
|
let encoded = BootstrapHeader {
|
|
|
|
|
control_region: Some(control),
|
|
|
|
|
..header()
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
.encode();
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
|
|
|
|
|
assert_eq!(
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
parse_bootstrap(&encoded, ARENA_BYTES)
|
|
|
|
|
.expect("valid control region")
|
|
|
|
|
.control_region,
|
|
|
|
|
Some(control)
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fn malformed_header_fails_before_attachment() {
|
|
|
|
|
let encoded = header().encode();
|
|
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&encoded[..HEADER_LEN - 1], ARENA_BYTES),
|
|
|
|
|
Err(BootstrapError::Truncated { .. })
|
|
|
|
|
));
|
|
|
|
|
|
|
|
|
|
let mut bad_magic = encoded;
|
|
|
|
|
bad_magic[0] ^= 0xff;
|
|
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&bad_magic, ARENA_BYTES),
|
|
|
|
|
Err(BootstrapError::BadMagic { .. })
|
|
|
|
|
));
|
|
|
|
|
|
|
|
|
|
let mut bad_version = encoded;
|
|
|
|
|
bad_version[4..6].copy_from_slice(&(BOOTSTRAP_VERSION + 1).to_le_bytes());
|
|
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&bad_version, ARENA_BYTES),
|
|
|
|
|
Err(BootstrapError::UnsupportedVersion { .. })
|
|
|
|
|
));
|
|
|
|
|
|
|
|
|
|
for reserved_at in [6, 47, 63] {
|
|
|
|
|
let mut bad_reserved = encoded;
|
|
|
|
|
bad_reserved[reserved_at] = 1;
|
|
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&bad_reserved, ARENA_BYTES),
|
|
|
|
|
Err(BootstrapError::ReservedBytesNotZero { at }) if at == reserved_at
|
|
|
|
|
));
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
|
|
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&encoded, ARENA_BYTES + 1),
|
|
|
|
|
Err(BootstrapError::ArenaSizeMismatch { .. })
|
|
|
|
|
));
|
|
|
|
|
|
|
|
|
|
let mut zero_generation = encoded;
|
|
|
|
|
zero_generation[16..24].fill(0);
|
|
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&zero_generation, ARENA_BYTES),
|
|
|
|
|
Err(BootstrapError::ZeroArenaGeneration)
|
|
|
|
|
));
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fn malformed_control_geometry_is_rejected() {
|
|
|
|
|
let mut partial = header().encode();
|
|
|
|
|
partial[24..32].copy_from_slice(&HEADER_END_OFFSET.to_le_bytes());
|
|
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&partial, ARENA_BYTES),
|
|
|
|
|
Err(BootstrapError::PartialControlRegion { .. })
|
|
|
|
|
));
|
|
|
|
|
|
|
|
|
|
let overlapping = BootstrapHeader {
|
|
|
|
|
control_region: Some(ControlRegion {
|
|
|
|
|
offset: HEADER_END_OFFSET - 1,
|
|
|
|
|
length: 1,
|
|
|
|
|
}),
|
|
|
|
|
..header()
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
.encode();
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&overlapping, ARENA_BYTES),
|
|
|
|
|
Err(BootstrapError::ControlOverlapsHeader { .. })
|
|
|
|
|
));
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
for control in [
|
|
|
|
|
ControlRegion {
|
|
|
|
|
offset: ARENA_BYTES - 4,
|
|
|
|
|
length: 8,
|
|
|
|
|
},
|
|
|
|
|
ControlRegion {
|
|
|
|
|
offset: u64::MAX - 3,
|
|
|
|
|
length: 8,
|
|
|
|
|
},
|
|
|
|
|
] {
|
|
|
|
|
let encoded = BootstrapHeader {
|
|
|
|
|
control_region: Some(control),
|
|
|
|
|
..header()
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
.encode();
|
|
|
|
|
assert!(matches!(
|
|
|
|
|
parse_bootstrap(&encoded, ARENA_BYTES),
|
|
|
|
|
Err(BootstrapError::ControlOutOfBounds { .. })
|
|
|
|
|
));
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
2026-08-27 22:09:26 +00:00
|
|
|
fn prepared_arena_descriptor_is_close_on_exec() {
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
let mut arena = arena();
|
2026-08-27 22:09:26 +00:00
|
|
|
let prepared = bootstrap::prepare_arena(&mut arena, spec()).expect("bootstrap write");
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
|
2026-08-27 22:09:26 +00:00
|
|
|
assert_eq!(prepared.arena_generation, GENERATION);
|
|
|
|
|
assert_ne!(
|
|
|
|
|
fd_flags(prepared.arena_fd.as_raw_fd()) & libc::FD_CLOEXEC,
|
|
|
|
|
0
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
);
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
|
|
|
|
|
let page = arena.read_arena(0, HEADER_LEN).expect("read header");
|
|
|
|
|
let resolved = parse_bootstrap(&page, arena.arena_len()).expect("written header parses");
|
|
|
|
|
assert_eq!(resolved.arena_generation, GENERATION);
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
2026-08-27 22:09:26 +00:00
|
|
|
fn mapped_arena_owns_mapping_but_closes_transferred_descriptor() {
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
let mut host = arena();
|
2026-08-27 22:09:26 +00:00
|
|
|
let prepared = bootstrap::prepare_arena(&mut host, spec()).expect("bootstrap write");
|
|
|
|
|
let inherited_fd = prepared.arena_fd.as_raw_fd();
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
|
2026-08-27 22:09:26 +00:00
|
|
|
let (mapped, resolved) = MappedArena::map(prepared.arena_fd).expect("map arena");
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
assert_eq!(mapped.len(), ARENA_BYTES as usize);
|
|
|
|
|
assert_eq!(resolved.arena_generation, GENERATION);
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
assert_eq!(
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fd_flags(inherited_fd),
|
|
|
|
|
-1,
|
|
|
|
|
"mapping closes the inherited descriptor"
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
);
|
|
|
|
|
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
host.write_arena(HEADER_END_OFFSET, b"visible")
|
|
|
|
|
.expect("host write");
|
|
|
|
|
let offset = HEADER_END_OFFSET as usize;
|
|
|
|
|
// SAFETY: the asserted range lies inside the live mapping.
|
|
|
|
|
let observed = unsafe { std::slice::from_raw_parts(mapped.base_ptr().add(offset), 7) };
|
|
|
|
|
assert_eq!(observed, b"visible");
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[test]
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
fn writer_rejects_zero_generation_or_alignment() {
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
for bad in [
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
bootstrap::BootstrapSpec {
|
|
|
|
|
arena_generation: 0,
|
|
|
|
|
..spec()
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
},
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
bootstrap::BootstrapSpec {
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
alignment: 0,
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
..spec()
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
},
|
|
|
|
|
] {
|
|
|
|
|
assert!(matches!(
|
2026-08-27 22:09:26 +00:00
|
|
|
bootstrap::prepare_arena(&mut arena(), bad),
|
feat(myelin): add actor-backed job data plane and uploads
Replace the eventfd/ring job bootstrap with one inherited arena descriptor, actor-owned sessions, sealed blob leases, awaitable inbox wakeups, and zero-copy Python mappings. Route VastAI mock provisioning through image-backed local Docker workers and preserve pinned child and controller routes across directory updates.
Add TOML job-file submission to the Fleet UI with generic started, running, and completed feedback, reusable remote job controller routing, cancellation and kill invariants, Tinygrad fixture and image support, and comprehensive Rust, Python, CUDA, and lifecycle-ordering coverage.
2026-08-21 14:45:10 +00:00
|
|
|
Err(bootstrap::BootstrapWriteError::InvalidSpec)
|
feat(data-plane): add exec bootstrap ABI for job processes
Define the spawn-time handoff a job process consumes before any
application code runs: two inherited descriptors named by
SWACTOR_ARENA_FD and SWACTOR_WAKE_FD, and a fixed 48-byte little-endian
header at arena offset 0 naming one control-ring region.
- Canonical header layout, shared parser, and a host writer that leases
the header region and control ring disjointly under the arena's own
placement law, zeroes the ring, stamps its generation, and returns the
handoff (env map, inheritable arena and wake descriptors, host wake
eventfd) only after every write completes.
- Python binding gains swactor.run(main): map the arena read-only, take
ground truth from fstat, validate through the shared parser, arm
FD_CLOEXEC on the wake descriptor, close the arena descriptor after
mapping, and drive main on asyncio with Context.data carrying the
resolved state. Fail-fast BootstrapError before main on any defect.
- Rewrite jobs/tiny_linear_inference.py against the approved path API and
record the slice, invariants, and remaining bridge work in the handoff
doc.
Verified with 12 new data-plane bootstrap guarantee tests (parser defect
table, fuzzed pages, writer round-trip), 29 binding tests across
in-process and exec boundaries, and full data-plane and iroh-driver
suites with no regressions.
2026-08-20 16:43:44 +00:00
|
|
|
));
|
|
|
|
|
}
|
|
|
|
|
}
|