zacheryasc/swactor
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
swactor/CLAUDE/notes/progress.md
Developer ef87f7e1b9 feat: per-actor message budget for tick fairness 
Research across ractor, tokio, Erlang/OTP BEAM, Linux CFS, and libuv
revealed that tick_all drained the entire mailbox per actor per tick,
allowing one hot actor to starve all others on the same worker.

- Add `actor_message_budget` to RuntimeConfig (default: 64 msgs/actor/tick)
- Modify tick_all to break after budget messages, yielding to next actor
- budget=0 restores unlimited (backward compatible) behavior
- 3 new fairness tests validating hot-cold actor scenarios
- New fairness benchmark group (cold_latency_under_pressure, throughput_by_budget)
- Fix RuntimeConfig struct literals across workspace crates

Inspired by BEAM's 4000-reduction budget and tokio's 128-op cooperative budget.
All 45 tests pass (42 original + 3 new).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 11:11:30 +00:00

2.7 KiB
Raw Blame History

Progress Log

Current Stage: Phase 1 — Research + First Improvement Cycle

Status: Cycle 1 COMPLETE

Plan Overview

  1. Phase 0: Codebase audit — understand current swactor architecture, existing tests, benchmarks ✅
  2. Phase 1: Broad survey + interleaved improvements
  3. Phase 2: Deeper improvements based on findings
  4. Phase 3: Testing methodology improvements
  5. Phase 4: Final evaluation & documentation

Completed This Session

Cycle 1: Fairness (Message Budget)

  • Research: Studied ractor, tokio, Erlang/OTP BEAM, Linux CFS/EEVDF, libuv
  • Finding: tick_all drained ENTIRE mailbox per actor per tick — critical fairness bug
    • BEAM uses 4000 reduction budget, tokio uses 128-op cooperative budget
    • Swactor had zero budget — one hot actor could starve all others on same worker
  • Implementation: Added actor_message_budget to RuntimeConfig (default: 64)
    • Modified tick_all to break after budget messages per actor
    • budget=0 means unlimited (backward compatible)
  • Tests: 3 new fairness tests (hot_actor_does_not_starve_cold_actor, unlimited_budget_drains_all, budget_messages_drain_across_multiple_ticks)
  • Benchmarks: Added fairness benchmark group (cold_latency_under_pressure, throughput_by_budget)
  • Fixes: Updated RuntimeConfig struct literals across crates (python, runtime-dashboard, mt_benchmarks)
  • Result: 45 tests pass (42 original + 3 new), all workspace crates compile

Research Notes

  • Full analysis in CLAUDE/notes/research_synthesis.md
  • Baseline benchmarks in CLAUDE/notes/baseline_benchmarks.md
  • Constraints in CLAUDE/notes/constraints.md

Next Steps

  • Cycle 2: Stress testing + property-based tests
    • Concurrent spawn+send stress tests
    • Multi-threaded fairness validation
    • Property: message ordering preserved under budget
    • Property: all messages eventually delivered with budget > 0
  • Cycle 3: Adaptive backoff with thread parking
    • Replace spinning with condvar-based parking (from tokio parker design)
    • Benchmark latency improvement under varying load
  • Cycle 4: Enhanced benchmarks
    • Message size sensitivity (8B, 64B, 256B, 1KB)
    • Latency percentiles (p50, p99, p999)
    • Many-to-one fanin contention
    • Cross-worker vs same-worker delivery comparison
  • Cycle 5: Work stealing exploration
    • Evaluate feasibility of actor migration between workers
    • BEAM two-tier approach: reactive steal + periodic migration

Open Questions

  • Should budget be configurable per-actor (not just per-runtime)?
  • Is 64 the right default budget? Benchmarks show budget=32 slightly faster for throughput
  • Thread parking: how to handle the notification mechanism without adding deps?

Blockers

  • (none)