Zachery Aaron Shores-Chmielewski
6608824cb0
Stand up an interactive end-to-end chat over a CUDA GPU, provisioning a Dockerized node that loads a GGUF model and serves prompts over TCP.
- prompt_rpc: add the newline-JSON prompt protocol (`SubmitPrompt` + `PromptEvent::{TextDelta,Done,Fault}`) carried over TCP
- mvp_chat: add an interactive REPL client connecting to the prompt RPC port (default 127.0.0.1:19777)
- mvp_orch_one_node / mvp_one_node_chat: add the single-node orchestrator that provisions a `LocalDockerPlugin` node, loads `bartowski/Llama-3.2-1B-Instruct-GGUF` (Q4_K_M), and exposes the prompt RPC listener with boot/route/weight timeouts
- mvp_node: add the GPU worker binary that spawns `tinygrad_worker.py` (default device CUDA) and ships runtime telemetry via a `ClusterFrameSink`
- vastai_provisioning / bootstrap_datastream: add the vast.ai provider adapter (`VastAiProvisioningConfig`, `VastAiLeaseClient`) wrapping `swactor_vastai`, plus a bridge that folds provision stdout onto a per-node datastream
- apps/mvp-node: add CUDA base/runtime Dockerfiles (nvidia/cuda 12.6.3, tinygrad 0.12.0, sshd), `mvp_entrypoint.sh` (sshd + mvp-node, held for postmortem), `local_docker_e2e.sh`, the GGUF tinygrad worker, and one-node-chat/bootstrap/vastai guarantee tests
Signed-off-by: Zachery Aaron Shores-Chmielewski <zacheryasc@gmail.com>
68 lines
2.4 KiB
Text
68 lines
2.4 KiB
Text
# MVP node BASE image — CUDA/Python/tinygrad/sshd/PID-1 foundation.
|
|
# Build from workspace root:
|
|
# docker build -f apps/mvp-node/Dockerfile.base -t swactor-mvp-node-base:cuda12.6 .
|
|
|
|
FROM nvidia/cuda:12.6.3-runtime-ubuntu24.04 AS builder
|
|
|
|
RUN apt-get update && \
|
|
apt-get install -y --no-install-recommends \
|
|
python3 \
|
|
python3-pip \
|
|
ca-certificates \
|
|
cuda-cudart-dev-12-6 && \
|
|
rm -rf /var/lib/apt/lists/*
|
|
|
|
RUN python3 -m pip install --no-cache-dir --break-system-packages \
|
|
--target=/opt/mvp-pydeps \
|
|
tinygrad==0.12.0 numpy && \
|
|
find /opt/mvp-pydeps -depth -type d \
|
|
\( -name '__pycache__' -o -name 'tests' -o -name 'test' \) \
|
|
-exec rm -rf {} + && \
|
|
find /opt/mvp-pydeps -name '*.pyc' -delete && \
|
|
find /opt/mvp-pydeps -type d -name '*.dist-info' -exec rm -rf {} +
|
|
|
|
RUN mkdir -p /opt/mvp-nvrtc-include && \
|
|
cp -rL /usr/local/cuda/include/. /opt/mvp-nvrtc-include/ && \
|
|
test -f /opt/mvp-nvrtc-include/vector_types.h
|
|
|
|
FROM nvidia/cuda:12.6.3-base-ubuntu24.04 AS runtime
|
|
|
|
RUN apt-get update && \
|
|
apt-get install -y --no-install-recommends \
|
|
python3 \
|
|
cuda-cudart-12-6 \
|
|
cuda-nvrtc-12-6 \
|
|
ca-certificates \
|
|
procps \
|
|
openssh-server && \
|
|
apt-get clean && \
|
|
rm -rf /var/lib/apt/lists/* \
|
|
/var/cache/apt/archives/* \
|
|
/var/cache/apt/*.bin \
|
|
/var/log/apt/* \
|
|
/var/log/dpkg.log \
|
|
/tmp/* \
|
|
/var/tmp/* \
|
|
/usr/share/doc/* \
|
|
/usr/share/man/* \
|
|
/usr/share/info/* && \
|
|
find /usr -depth -type d -name '__pycache__' -exec rm -rf {} + 2>/dev/null || true && \
|
|
find /usr -type f -name '*.pyc' -delete 2>/dev/null || true && \
|
|
mkdir -p /usr/local/share/mvp /var/cache/mvp-models /var/log
|
|
|
|
COPY --from=builder /opt/mvp-nvrtc-include/ /usr/local/cuda/include/
|
|
COPY --from=builder /opt/mvp-pydeps /opt/mvp-pydeps
|
|
ENV PYTHONPATH=/opt/mvp-pydeps
|
|
ENV PYTHONDONTWRITEBYTECODE=1
|
|
ENV CUDA=1
|
|
ENV DEV=CUDA
|
|
ENV MVP_MODEL_CACHE_DIR=/var/cache/mvp-models
|
|
ENV MVP_TINYGRAD_WORKER=/usr/local/share/mvp/tinygrad_worker.py
|
|
|
|
COPY apps/mvp-node/mvp_entrypoint.sh /usr/local/bin/mvp_entrypoint.sh
|
|
RUN chmod +x /usr/local/bin/mvp_entrypoint.sh
|
|
|
|
ARG DEPLOY_PUBKEY=""
|
|
RUN if [ -n "$DEPLOY_PUBKEY" ]; then printf '%s\n' "$DEPLOY_PUBKEY" > /etc/mvp_deploy_key.pub; fi
|
|
|
|
ENTRYPOINT ["/usr/local/bin/mvp_entrypoint.sh"]
|