2026-07-23 09:39:17 +00:00
|
|
|
use std::process::ExitCode;
|
feat(mvp-chat): local e2e chat on cuda gpu
Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP.
- prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP
- mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777)
- mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts
- mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink`
- vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream
- apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
|
|
|
|
|
|
|
|
fn main() -> ExitCode {
|
refactor: prune public api
Collapse mvp-system's public surface to three binary entrypoints and make every domain module private, deleting dead provider/worker/membership implementations and inlining provider config.
- lib.rs: expose only run_chat_from_args/run_orchestrator_from_args/run_worker_node_from_env (plus a crate-private in-process helper) and the cached-model consts, and demote chat/node/observability/orchestration/prompt/staging/transport to private mods
- orchestration/mod.rs: make app private, gate engine_builder behind cfg(test), drop docker_cluster from provider_adapters, tighten vastai to pub(super), and replace pub re-exports with pub(super) run_from_args/run_in_process_from_args
- orchestration/config.rs: inline VastAiConfig/ResolvedVastAiConfig/looks_remote_image (removing provider_adapters/vastai/config.rs) and drop the DEFAULT_PIPELINE_CACHED_MODEL_* consts (hoisted to lib.rs)
- orchestration/provider_adapters/vastai: delete the ProviderPlugin impl VastAiProviderPlugin and all client/bootstrap/config accessors; repoint call sites to crate-level #[path] mods for provisioning/node_provisioning/node_actor/gguf_shard/run_fsm/run_plan
- delete orchestration/{membership_readiness,token_endpoint,resource_inventory}, node/{boot_lifecycle,data_plane_bridge(-74)}, and the worker crate-internal modules (control/device_bridge/process_adapter) along with their guarantees tests
- chat/node: narrow node_image and worker_node_runtime to private and expose only pub(super) run_from_args / run_worker_node_from_env
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-29 10:02:50 +00:00
|
|
|
match mvp_system::run_orchestrator_from_args(std::env::args().skip(1)) {
|
feat(mvp-chat): local e2e chat on cuda gpu
Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP.
- prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP
- mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777)
- mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts
- mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink`
- vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream
- apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
|
|
|
Ok(()) => ExitCode::SUCCESS,
|
|
|
|
|
Err(error) => {
|
2026-07-07 10:40:02 +00:00
|
|
|
eprintln!("mvp-orchestrator: {error}");
|
feat(mvp-chat): local e2e chat on cuda gpu
Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP.
- prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP
- mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777)
- mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts
- mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink`
- vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream
- apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
2026-07-01 08:44:25 +00:00
|
|
|
ExitCode::from(1)
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
}
|