# Dead-Node Reprobe — Design & Rationale

## Problem

When a network partition heals, SWIM nodes on both sides may have declared each
other Dead. The `alive_members()` filter excludes Dead nodes from probing
targets, so neither side initiates communication — creating a **permanent split**
even after connectivity is restored.

The existing refutation mechanism (incarnation bump on learning of own death
declaration) handles resurrection correctly, but depends on someone *telling*
the dead-declared node about its status. With no probes to dead nodes, nobody
does.

## Solution: Independent Dead-Node Reprobe Cycle

Added a lightweight reprobe mechanism to `SwimProbe` that periodically pings
dead nodes. The existing piggyback + refutation mechanism handles the rest:

1. **Reprober** pings dead node with piggybacked "you are Dead(inc=N)"
2. **Target** receives piggyback, sees it's declared Dead → refutes → bumps incarnation
3. **Target** replies with Ack carrying piggybacked "I'm Alive(new_inc)"
4. **Reprober** applies piggyback → target transitions Dead→Alive

Key insight: we re-enqueue the death declaration in the dissemination queue
before packing the reprobe ping's piggyback. Without this, the original death
declaration's transmit budget would be long exhausted, and the piggyback would
carry no useful membership info.

## Design Decisions

### Why inside SwimProbe (not SwimNode)?

- SwimProbe owns the tick counter, sequence counter, and member list access
- All probe-related logic stays in one place
- The reprobe is a simple independent cycle — doesn't interfere with ProbePhase

### Why no new wire messages?

- `SwimAction::SendPing` works identically for normal probes and reprobes
- The ack from a reprobe targets a different sequence than the current probe
  cycle, so the probe state machine ignores it — but the piggyback is applied
  at the SwimNode layer before the probe state machine sees the ack

### Configuration

- `dead_reprobe_interval: u64` (default: 50 ticks, ~5× probe_interval)
- Set to 0 to disable completely
- Existing test configs use 0 to avoid interference with timing

## Files Changed

| File | Change |
|------|--------|
| `crates/distribution/src/swim/probe.rs` | `dead_reprobe_interval` in config, `maybe_reprobe_dead()` method |
| `crates/distribution/src/swim/member_list.rs` | Added `dead_members()` |
| `crates/distribution/src/swim/node.rs` | Re-enqueue death declaration in `translate_probe_actions` |
| All `SwimConfig` struct literals | Added `dead_reprobe_interval` field |

## Edge Cases

- **Truly dead nodes**: Reprobe ping is lost (no ack), no harm done
- **All members dead**: Reprobe cycles through them round-robin
- **Concurrent reprobe + normal probe**: Independent, different sequences
- **Dissemination budget**: Death re-enqueued fresh each reprobe, not stale

## Alternatives Considered

1. **Dead node grace period + resurrection timer**: More complex, adds new
   state tracking alongside suspicion timers. Rejected for simplicity.
2. **Periodic re-join via seed nodes**: Requires seed node availability,
   doesn't work when seed is itself dead-declared. Rejected.
3. **Direct liveness inference from Ping reception**: Would require changing
   `handle_ping` to special-case pings from dead nodes. More invasive.