swactor/docs/development_history/IN_BROWSER_RUNTIME.md

# In-Browser Runtime via WebAssembly — Development History

> Covers the work to make swactor run in the browser with WebSocket
> connectivity to a native cluster via a gateway pattern.
>
> *Branch: `in-browser`*

---

## Table of Contents

1. [Overview & Motivation](#1-overview--motivation)
2. [What Was Built](#2-what-was-built)
3. [Development Phases](#3-development-phases)
4. [NodeAddr Abstraction](#4-nodeaddr-abstraction)
5. [Core Wasm Compatibility](#5-core-wasm-compatibility)
6. [WebSocket Transport (Browser)](#6-websocket-transport-browser)
7. [WebSocket Gateway (Native)](#7-websocket-gateway-native)
8. [Enhanced Browser Runtime](#8-enhanced-browser-runtime)
9. [Demo & Examples](#9-demo--examples)
10. [Design Decisions & Tradeoffs](#10-design-decisions--tradeoffs)
11. [Dashboard Improvements](#11-dashboard-improvements)
12. [Known Gaps & Future Work](#12-known-gaps--future-work)

---

## 1. Overview & Motivation

Before this work, swactor had a minimal wasm crate (`crates/wasm/`) that could
run hardcoded Counter/Relay actors locally via `tick()` — no networking, no
JS-defined actors, no connection to a cluster.

The goal: **make a browser node that can run actors locally AND connect to a
native cluster** via WebSocket, with the architecture designed so the browser
could eventually become a full cluster peer (WebRTC P2P).

Two design constraints guided the approach:

- **Start thin, design for full.** Use a gateway pattern for v1 connectivity
  (browser ↔ WebSocket ↔ native node), but introduce the `NodeAddr` abstraction
  now so the distribution layer can eventually support WebSocket and WebRTC
  peers natively.

- **Both Rust and JS actors.** Rust-compiled actors work by defining
  `ActorInterface` impls in the wasm crate (as the existing Counter/Relay do).
  JS actors work via `js_sys::Function` callback wrappers.

---

## 2. What Was Built

| Component | Location | Action | Key Changes |
|-----------|----------|--------|-------------|
| NodeAddr abstraction | `crates/distribution/` | Modified (~30 files) | `SocketAddr` → `NodeAddr` across distribution, simulation, dashboard |
| Core wasm compat | `src/` | Modified (4 files) | `web-time` abstraction, cfg-gated threads, wire encoding extraction |
| WebSocket transport | `crates/wasm/src/` | Created (3 files) | `WsTransport`, `GatewayControl` protocol, `JsActor` wrapper |
| WebSocket gateway | `crates/gateway/` | Created (new crate) | `WsAcceptor`, `WsGateway`, `SessionTransport` |
| Browser runtime | `crates/wasm/src/lib.rs` | Rewritten | `BrowserRuntime` API with JS actor support |
| Demo | `examples/`, `crates/wasm/www/` | Created | Gateway example, HTML demo page |
| Dashboard improvements | `crates/runtime-dashboard/static/` | Created + Modified | Static file extraction, gossip panel, visual fixes, interactivity |
| Gossip accessors | `crates/distribution/src/` | Modified (5 files) | `SwimProbe`/`SwimNode`/`DisseminationQueue`/`DistributedNode` accessors, snapshot types |

---

## 3. Development Phases

### Phase 1 — NodeAddr abstraction in distribution crate

Replaced `SocketAddr` with an extensible `NodeAddr` enum across ~30 files:
distribution crate (15 source files + 12 test files), simulation crate, and
dashboard example. The main challenge was that `NodeAddr` is `Clone` but not
`Copy` (unlike `SocketAddr`), requiring ~25 `.clone()` additions.

### Phase 2 — Core wasm compatibility

Made the core `swactor` crate compile for `wasm32-unknown-unknown`:
- `web-time` behind `cfg(wasm32)` for `Instant`
- cfg-gated `RuntimeHandle`, `run()`, thread imports
- Extracted `encode_wire_envelope`/`decode_wire_envelope` into core `src/transport.rs`
- Distribution crate re-exports the shared wire encoding functions

### Phase 3 — WebSocket transport (browser side)

Created browser-side transport wrapping `web_sys::WebSocket`:
- `WsTransport` — buffers outbound messages until connection opens, decodes
  inbound binary frames, implements `Transport` trait
- `GatewayControl` — control protocol for actor registration/resolution/keepalive
- Same length-prefixed binary wire format as TCP

### Phase 4 — WebSocket gateway (native side)

Created `crates/gateway/` — a native-side WebSocket acceptor + gateway:
- `WsAcceptor` — mirrors `TcpAcceptor` pattern (non-blocking accept, WS upgrade,
  non-blocking read loop, dead connection cleanup)
- `WsGateway` — routes inbound envelopes to runtime, handles control protocol,
  creates `SessionTransport` routes for cluster → browser forwarding
- Sync `tungstenite` in dedicated thread — no tokio dependency

### Phase 5 — Enhanced browser runtime

Replaced the hardcoded wasm MVP with a generic runtime:
- `JsActor` — wraps `js_sys::Function` as an `ActorInterface` impl
- `JsActorCtx` — bridge object passed to JS handlers (`send`, `self_addr`)
- `BrowserRuntime` — JS-facing API: `connect()`, `spawn_js_actor()`,
  `send_json()`, `tick()`, `create_inbox()`, `try_recv()`
- `JsMessage` codec for JSON-based communication
- Legacy `SwactorRuntime` preserved for backwards compatibility

### Phase 6 — Demo and testing

- `examples/ws_gateway.rs` — native gateway node with echo actor
- `crates/wasm/www/index.html` — browser demo with connection UI, actor
  spawning, message sending, and log panel

---

## 4. NodeAddr Abstraction

### The Type

```rust
// crates/distribution/src/types.rs
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum NodeAddr {
    Tcp(SocketAddr),
    // Future: Ws(String), WebRtc(String)
}
```

All distribution APIs (`SwimNode`, `DistributedNode`, `RoutingTable`, etc.)
now accept and return `NodeAddr`. The `TcpTransport` pattern-matches
`NodeAddr::Tcp(addr)` when connecting:

```rust
fn require_tcp(addr: &NodeAddr) -> Result<SocketAddr, Error> {
    match addr {
        NodeAddr::Tcp(sa) => Ok(*sa),
    }
}
```

### Scope of changes

The refactor touched every file in the distribution crate that previously used
`SocketAddr`:

- `types.rs` — `NodeRecord.addr`, `DirectoryEntry` field type
- `messages.rs` — `PingReq.target_addr`, `JoinRequest.addr`, `FindNodeResponse`, `FindValueResponse`
- `swim/node.rs` — `SwimNode.self_addr`, all `NodeAction` variants
- `swim/probe.rs` — `SwimAction` variants, `ProbePhase`, target selection
- `swim/member_list.rs` — `MemberEntry.addr`, `apply()` signature
- `swim/dissemination.rs` — `membership_update()` function
- `kademlia/routing_table.rs` — `NodeEntry.addr`, `insert()` signature
- `kademlia/lookup.rs` — `LookupAction`, `NodeLookup.known`
- `node.rs` — `DistributedNodeConfig.listen_addr`, all handler methods
- `transport.rs` — `TcpTransport::new()`, `send_to()`
- `snapshot.rs` — `addr_str()`
- All 12 test files in `crates/distribution/tests/`
- `crates/simulation/src/distribution/sim.rs`
- `crates/runtime-dashboard/examples/dashboard_demo.rs`

---

## 5. Core Wasm Compatibility

### Time abstraction

```rust
// src/lib.rs
pub(crate) mod time {
    #[cfg(not(target_arch = "wasm32"))]
    pub(crate) use std::time::Instant;
    #[cfg(target_arch = "wasm32")]
    pub(crate) use web_time::Instant;
}
```

`src/runtime.rs` and `src/worker.rs` import `crate::time::Instant` instead of
`std::time::Instant`. The `web-time` crate provides a browser-compatible
`Instant` backed by `performance.now()`.

### cfg-gated thread code

```rust
// src/runtime.rs
#[cfg(not(target_arch = "wasm32"))]
use std::thread::{self, JoinHandle};

#[cfg(not(target_arch = "wasm32"))]
pub struct RuntimeHandle { ... }

#[cfg(not(target_arch = "wasm32"))]
pub fn run(self) -> Result<RuntimeHandle, Error> { ... }
```

Same pattern in `src/worker.rs` for `Worker::run()`. On wasm32, only
`tick()` is available.

### Wire encoding extraction

`encode_wire_envelope` and `decode_wire_envelope` were moved from
`crates/distribution/src/transport.rs` into core `src/transport.rs` so both
the distribution crate and the wasm crate can share them. The distribution
crate re-exports:

```rust
pub use swactor::transport::encode_wire_envelope;
pub use swactor::transport::decode_wire_envelope;
```

### Worker module visibility

On wasm32, the worker module is `pub(crate)` (not `pub`) since external code
shouldn't depend on thread-specific worker internals:

```rust
#[cfg(not(target_arch = "wasm32"))]
pub mod worker;
#[cfg(target_arch = "wasm32")]
pub(crate) mod worker;
```

---

## 6. WebSocket Transport (Browser)

```
crates/wasm/src/
├── ws_transport.rs   — WsTransport (Transport impl over web_sys::WebSocket)
├── protocol.rs       — GatewayControl enum, CONTROL_TYPE_TAG, NULL_ADDRESS
└── js_actor.rs       — JsActor, JsActorCtx, JsMessage
```

### WsTransport

Wraps `web_sys::WebSocket` with outbound buffering and inbound envelope
decoding:

```
  Browser JS                 WsTransport                    Gateway
  ──────────                 ───────────                    ───────
                     ┌─ Connecting ─┐
  rt.connect(url) ──►│ buffer sends │──── WS handshake ────►
                     └──────────────┘
                     ┌─── Open ─────┐
  rt.send_json() ───►│ encode + send│──── binary frame ────►
                     │              │
                     │ decode inbound│◄─── binary frame ────
                     └──────────────┘
  rt.tick()      ───► drain_inbound() → deliver_raw()
```

The transport uses `Rc<RefCell<WsInner>>` internally — safe because wasm32
is single-threaded. `unsafe impl Send + Sync` matches the pattern used by
`Runtime`'s existing `unsafe impl Sync` for `RefCell<Vec<Worker>>`.

### Control Protocol

Control messages use a reserved null address (`[0u8; 32]`) and the type tag
`"swactor::GatewayControl"`:

```rust
pub enum GatewayControl {
    RegisterActor { addr: [u8; 32] },
    UnregisterActor { addr: [u8; 32] },
    ResolveActor { name: String },
    ActorResolved { name: String, addr: [u8; 32] },
    Ping,
    Pong,
}
```

---

## 7. WebSocket Gateway (Native)

```
crates/gateway/src/
├── lib.rs           — WsGateway, SessionTransport, control protocol handler
└── ws_acceptor.rs   — WsAcceptor (mirrors TcpAcceptor), WsSession, SessionId
```

### WsAcceptor

Follows the same non-blocking pattern as `TcpAcceptor` in the distribution
crate:

1. Non-blocking TCP accept
2. WebSocket handshake (briefly blocking per new client)
3. Switch to non-blocking for reads
4. Read binary frames from all sessions
5. Dead connection cleanup in reverse index order

### WsGateway

```
  Browser            WsGateway              Runtime / Cluster
  ───────            ─────────              ──────────────────

  WireEnvelope ─────► route by dest:
                       │
                       ├─ control msg? → handle_control()
                       │                  ├─ RegisterActor → add route
                       │                  ├─ Ping → send Pong
                       │                  └─ ...
                       │
                       └─ regular msg → codec_registry.receive()
                                          └─ runtime.deliver_raw()

  ◄───── WireEnvelope ◄── SessionTransport.send()
                            │
                            └─ TransportRouter lookup hits
                               SessionTransport for browser-owned address
```

When a browser registers an actor, the gateway:
1. Stores `addr → session_id` mapping
2. Creates a `SessionTransport` route in the `TransportRouter`
3. Cluster actors sending to that address hit the route, which
   forwards via the WebSocket session

---

## 8. Enhanced Browser Runtime

### JsActor

```rust
pub struct JsActor {
    handler: js_sys::Function,  // (ctx, type_tag, msg_json) => void
}

impl ActorInterface for JsActor {
    type Incoming = JsMessage;
    type Response = ();
    fn handle(&mut self, ctx: &Ctx, msg: JsMessage) { ... }
}
```

The handler receives a `JsActorCtx` bridge that exposes `send(dest_hex,
type_tag, msg_json)` and `self_addr()` to JavaScript.

### BrowserRuntime API

```
  JavaScript                    BrowserRuntime (Rust/wasm)
  ──────────                    ──────────────────────────

  new BrowserRuntime()     ──► creates Runtime + CodecRegistry + TransportRouter
  rt.connect(url)          ──► WsTransport::connect(url)
  rt.spawn_js_actor(fn)    ──► JsActor wrapper → rt.spawn() → register w/ gateway
  rt.send_json(hex, tag, j)──► JsMessage → rt.send_to()
  rt.tick()                ──► drain WS inbound → deliver_raw → rt.tick()
  rt.create_inbox()        ──► rt.new_inbox::<JsMessage>() → hex address
  rt.try_recv(hex)         ──► inbox.try_recv() → JSON string or undefined
  rt.is_connected()        ──► WsTransport::is_connected()
  rt.actor_count()         ──► rt.stats().actors.len()
```

### JsMessage

A unified message type for all JS actors:

```rust
pub struct JsMessage {
    pub type_tag: String,  // wire protocol type tag (or "local")
    pub payload: String,   // JSON payload
}
```

Registered in the `CodecRegistry` with type tag `"swactor::JsMessage"` and a
simple UTF-8 codec.

---

## 9. Demo & Examples

### Gateway example (`examples/ws_gateway.rs`)

```bash
cargo run --example ws_gateway --features transport
```

Starts a native node on `ws://127.0.0.1:9000` with an echo actor. The gateway
accepts WebSocket connections and bridges messages between browser clients and
the runtime.

### Demo HTML page (`crates/wasm/www/index.html`)

Build and serve:

```bash
cd crates/wasm && wasm-pack build --target web --out-dir www/pkg
cd www && python3 -m http.server 8080
# Open http://localhost:8080
```

Features: connection panel with status indicator, JS actor spawning with
click-to-copy addresses, message sending, and a timestamped event log.

---

## 10. Design Decisions & Tradeoffs

| Decision | Rationale |
|----------|-----------|
| Gateway pattern for v1 | Avoids STUN/TURN complexity; one TCP connection per browser; simple to debug. `NodeAddr` abstraction leaves room for direct WebRTC P2P later. |
| Sync tungstenite (no tokio) | Matches existing `TcpAcceptor` pattern in the distribution crate. Keeps the dependency tree small. The gateway runs a poll loop on a dedicated thread. |
| `NodeAddr` enum (not trait) | Enums are exhaustive, serializable, and cheap to match. Adding a variant (e.g., `Ws(String)`) is a compiler-guided refactor. |
| Same wire format over WS as TCP | No protocol translation — a `WireEnvelope` is the same bytes on both transports. Simplifies debugging and testing. |
| `Rc<RefCell>` in WsTransport | wasm32 is single-threaded. `Arc<Mutex>` would compile but add unnecessary overhead. `unsafe impl Send+Sync` is the standard pattern for wasm-bindgen types. |
| JSON for JS actor messages | The browser's native format. Binary codecs could be added later via `CodecRegistry`. |
| Preserved legacy `SwactorRuntime` | The old Counter/Relay API still works. New code uses `BrowserRuntime`. |

---

## 11. Dashboard Improvements

Alongside the in-browser work, the runtime dashboard received significant
improvements: extraction to static files, visual bug fixes, interactivity,
and a new gossip protocol panel on the distribution page.

### Static file extraction

The dashboard HTML was originally embedded as Rust string constants
(`actors_html.rs`, `dashboard_html.rs`, `distribution_html.rs`). These were
extracted to standalone files under `crates/runtime-dashboard/static/`:

```
static/
├── css/shared.css       — shared dark-theme styles, stat cards, nav
├── js/shared.js         — SSE helper, color palette, colorWithAlpha()
├── index.html           — overview page (worker chart, actor table)
├── actors.html          — actors page (worker bars, depth chart, actor list)
└── distribution.html    — distribution page (graph, gossip, membership)
```

The server now serves these as static files instead of embedding them.

### Visual fixes and performance

- **Bar chart label clipping** — clamped `fillText` y-position to
  `Math.max(12, chartH - h - 4)` so labels for tall bars stay visible
- **Worker details click flakiness** — `innerHTML = ''` every 200ms
  destroyed DOM elements and their click handlers. Replaced with persistent
  `workerNodes` map; headers and handlers are created once, text updated
  in-place
- **Legend wrapping** — added `white-space: nowrap; overflow: hidden;
  text-overflow: ellipsis` to worker legend
- **Performance** — data fingerprinting (`JSON.stringify` comparison) skips
  redundant redraws; differential actor table updates; `requestAnimationFrame`
  throttling

### Actors page interactivity

- **Click-to-filter** — clicking a worker bar in the chart filters the actor
  table to that worker's actors (click again to clear)
- **Depth chart colors** — changed from green-red heatmap to purple/indigo
  palette to distinguish from worker bars
- **Worker-colored rows** — actor rows are tinted with a translucent version
  of their worker's color via `colorWithAlpha(hex, 0.08)`

### Gossip protocol panel (distribution page)

Exposed SWIM gossip internals through new Rust accessors and snapshot types,
then added a dedicated panel to the distribution page.

**Rust changes** — new accessor methods on `SwimProbe` (`tick()`,
`sequence()`, `phase_name()`, `probe_target()`, `suspicion_timers()`,
`config()`), `SwimNode` (`probe()`, `dissemination()`), `DisseminationQueue`
(`pending_entries()`), and `DistributedNode` (`swim_node()`). New snapshot
structs: `GossipInfo`, `SuspicionInfo`, `DisseminationInfo`, `GossipConfig`.
Added `gossip: Option<GossipInfo>` to `DistributionNodeSnapshot` with
`#[serde(default)]` for backward compatibility.

**UI panel** shows:
- **Phase badge** — IDLE (green), PINGING (yellow), INDIRECT (orange)
- **Counters** — protocol round, probes sent, incarnation, pending updates
- **Suspicion timers** — table with progress bars toward timeout
- **Membership events** — persistent rolling log of state transitions
  (new/alive/suspect/dead/gone) built by diffing consecutive snapshots
  client-side, with timestamps and node addresses
- **Recently probed** — list of recent probe targets
- **Config** — human-readable protocol parameters

All node IDs are displayed as socket addresses (via a lookup map built from
the members list) rather than raw hex. When running solo with no peers, the
panel shows "No peers — gossip inactive" instead of zeros.

---

## 12. Known Gaps & Future Work

| Gap | Effort | Impact |
|-----|--------|--------|
| `ResolveActor` control message not implemented | Low | Browser can't discover actors by name |
| No WebSocket reconnection logic | Medium | Browser must refresh on disconnect |
| No authentication on WS connections | Medium | Any client can register actors |
| Gateway doesn't participate in SWIM | Medium | Browser actors aren't in the cluster directory |
| No back-pressure from WS to actors | Low | Fast sender can overwhelm browser |
| WebRTC P2P (direct browser↔browser) | High | Eliminates gateway bottleneck |
| `NodeAddr::Ws` variant in distribution | Medium | Browser as full cluster peer without gateway |
| wasm-pack integration test | Low | Automated browser test in CI |

---

## Verifying

### Native build + tests

```bash
# Full workspace build
cargo build

# All tests (excluding flaky simulation MT test)
cargo test --workspace --exclude simulation

# Distribution tests specifically (134 tests)
cargo test -p distribution

# Gateway crate builds
cargo build -p swactor-gateway
```

### Wasm32 target

```bash
# Core crate compiles for wasm32
cargo build --target wasm32-unknown-unknown -p swactor --no-default-features --features "no_random,transport"

# Wasm crate compiles for wasm32
cargo build --target wasm32-unknown-unknown -p swactor-wasm
```

### Dashboard

```bash
# Build dashboard (includes static files)
cargo build -p runtime-dashboard

# Run the distribution demo (9-node churn simulation with dashboard)
cargo run --example dashboard_demo -p runtime-dashboard --features distribution
# Open http://localhost:3000 → Overview, Actors, Distribution pages

# Verify distribution page gossip panel:
# - Solo node: "No peers — gossip inactive"
# - With peers: phase badge, counters, membership event log
```

### Gateway example (manual)

```bash
# Terminal 1: start gateway
cargo run --example ws_gateway --features transport

# Terminal 2: build wasm + serve demo page
cd crates/wasm && wasm-pack build --target web --out-dir www/pkg
cd www && python3 -m http.server 8080
# Open http://localhost:8080 and click Connect
```