swactor-development-history/cfuzz/CYCLE_17_SUPERVISION.md

# Cycle 17: Supervision Trees with handle_down and Supervisor Actor — Development History

> Commit: `a70bd86` · 4 files · 754 insertions, 7 deletions

---

## Motivation

With monitoring (Cycle 13), lifecycle hooks (Cycle 9), and factory-based restart (Cycle 7) in place, swactor had all the building blocks for supervision trees — the signature feature of Erlang/OTP. Supervision trees provide structured fault tolerance: a parent actor (supervisor) monitors children and restarts them according to configurable policies when they fail.

## Competitor Analysis

| Framework | Supervisor Model | Strategies | Child Spec | Meltdown Protection |
|-----------|-----------------|------------|------------|---------------------|
| Erlang/OTP | Built-in `supervisor` behaviour | one_for_one, one_for_all, rest_for_one, simple_one_for_one | `{Id, MFA, Restart, Shutdown, Type}` | Intensity/period limits |
| Akka | SupervisorStrategy | Resume, Restart, Stop, Escalate + BackoffSupervisor | N/A (inline) | MaxNrOfRetries/withinTimeRange |
| Ractor | `ractor-supervisor` crate | External crate, event-based | SupervisionEvent callback | N/A |
| Bastion | Built-in hierarchy | Redundancy groups | Structural (parent-child) | N/A |
| CAF | No built-in supervisor | Monitor-based (manual) | N/A | N/A |
| **Swactor** | **User-space `Supervisor` actor** | **OneForOne** (Cycle 17), **OneForAll/RestForOne** (Cycle 18) | **`ChildSpec`** | **max_restarts budget** |

### Key Findings
- Swactor has all the building blocks: monitor (Cycle 13), `spawn_restartable` (Cycle 7), lifecycle hooks (Cycle 9), `Down` messages (Cycle 13)
- **Decision**: Supervisor as a user-space actor built on existing primitives (like Ractor's `ractor-supervisor` crate), not a special runtime construct
- **`handle_down` callback** enables any actor to react to monitored deaths without requiring `Incoming = Down` — this is the key API gap that needed filling

## Implementation

### 1. `handle_down` Callback on ActorInterface

The core API addition enabling supervision:

- `fn handle_down(&mut self, ctx: &Ctx, down: Down)` — default no-op, called when a monitored actor dies and the actor's `Incoming` type is NOT `Down`
- Implemented via second downcast attempt in `handle_any`: if the message is `Down` and the actor's `Incoming` type doesn't match, call `handle_down` instead of `handle`
- Fully backward-compatible: actors with `Incoming = Down` still receive via `handle()` as before
- This decouples supervision logic from the actor's primary message type

### 2. `ctx.stop_actor(addr)` — Stop Another Actor

- Sends graceful stop to another actor from handler context
- Uses `StopSignal` through normal message routing (PoisonPill semantics)
- Enables supervisor-controlled shutdown of children

### 3. `Supervisor` Actor

A user-space actor managing child actors:

- **`SupervisorStrategy::OneForOne`** — only the failed child is restarted (Cycle 17)
- **`RestartPolicy`**: `Permanent` (always restart), `Transient` (restart only on panic, not normal stop), `Temporary` (never restart)
- **`ChildSpec`** — `{ id: String, restart: RestartPolicy, factory: Fn(&Ctx) -> Result<ActorAddress> }`
- Children spawned in `on_start`, monitored via `ctx.monitor()`
- Death detected via `handle_down`, restart policy consulted, factory invoked for replacement
- **Meltdown detection**: stops itself when `total_restarts > max_restarts`
- **Cascading shutdown**: `on_stop` sends stop signals to all living children

### ActiveChild Struct
- Tracks `addr: ActorAddress` and `monitor_ref: MonitorRef` per child
- Reused by Router (Cycle 19)

**Key files modified:** `src/actor.rs`, `tests/runtime_api.rs`, `docs/runtime.md`

## Design Decisions

- **User-space actor (not runtime primitive)** — the Supervisor is just an actor that uses existing APIs (monitor, spawn, stop). No special runtime support needed. This validates the composability of the monitoring and lifecycle systems.
- **`handle_down` as opt-in callback** — adding `handle_down` to `ActorInterface` with a default no-op means existing actors don't need to change. Actors that want to react to deaths override it. The alternative (requiring `Incoming = Down`) would force actors to handle `Down` as their primary message type.
- **Factory takes `&Ctx`** — the factory closure receives the context so it can use `ctx.spawn`, `ctx.monitor`, etc. during child creation. This enables the supervisor to monitor new children immediately.
- **Meltdown protection** — if children keep crashing faster than they can be restarted, the supervisor stops itself rather than looping forever. Matches Erlang's intensity/period limits.
- **Cascading shutdown** — when the supervisor stops, all living children receive stop signals. This prevents orphaned actors.

## Tests Added

10 new tests (127 → 138 total, counting 130 behavioral + 7 proptest + 1 doctest):

- `handle_down_receives_death_notification` — handle_down callback fires on monitored death
- `handle_down_skipped_when_incoming_is_down` — backward compat: Incoming=Down uses handle()
- `ctx_stop_actor_stops_target` — one actor stops another via ctx.stop_actor()
- `supervisor_restarts_permanent_child_on_panic` — panic → restart (OneForOne + Permanent)
- `supervisor_does_not_restart_transient_child_on_normal_stop` — Normal stop → no restart
- `supervisor_restarts_transient_child_on_panic` — Panicked → restart (Transient)
- `supervisor_never_restarts_temporary_child` — Temporary → never restart
- `supervisor_meltdown_after_max_restarts` — exceeding max_restarts stops supervisor
- `supervisor_one_for_one_only_restarts_failed_child` — multi-child, only crashed child restarted
- `supervisor_on_stop_kills_children` — supervisor shutdown cascades to children

## Result

- 138 tests pass (130 behavioral + 7 proptest + 1 doctest)
- Zero warnings, full workspace compiles
- Supervisor validates the composability of Cycles 7 (recovery), 9 (lifecycle), and 13 (monitoring)