diff --git a/CONTEXT_TRIM.md b/CONTEXT_TRIM.md new file mode 100644 index 0000000..8282ba6 --- /dev/null +++ b/CONTEXT_TRIM.md @@ -0,0 +1,64 @@ +# Context trim — design notes + +## What's shipped + +Worker sessions are now resumed across iterations with `claude --resume `, +and the session JSONL is surgically trimmed between rounds so Anthropic's +prefix cache stays warm without paying for irrelevant history. + +- Agent declares a `KEEP: ` line in `.loop/notes.md`. +- Yoke locates `~/.claude/projects//.jsonl` and rewrites it + to retain only: bootstrap records, the initial user prompt, `Read` + tool_use/tool_result pairs for kept paths, and attachments. Drops: + `thinking`, intermediate `text`, every non-`Read` tool_use, `Read`s of + non-kept files, and the matching `tool_result`s. Re-links the + parent-uuid chain across the gaps; validates tool_use ↔ tool_result + pairing before commit; keeps a `.bak`. +- Judge (brute) and scoper (saga) always run fresh — independence per + behavioral-specification §2.1. +- `YOKE_DISABLE_SESSION_TRIM=1` is the escape hatch. + +## What still needs adding + +1. **Format-drift guard.** The Claude Code session JSONL is undocumented. + A future CLI release could rename a field, change content-block shape, + or move the file. The validator catches most damage post-trim, but + pre-trim we should fingerprint the format (e.g., known top-level keys + on bootstrap records) and bail if it drifts. Today we trust + bail on + validate-fail; a positive check would be safer. + +2. **Recovery from `--resume` failure.** If Claude rejects the resumed + session (deleted, corrupted, version skew), the iteration aborts. We + should detect this from the spawn's exit/early stream error and + transparently retry once with no `--resume` (treat last_session_id as + stale). + +3. **Sandboxed runs.** When `image` is set, the agent runs inside a + Docker container — the session JSONL lives in the container's home, + not the host's. Today `trim_worker_session` no-ops in that case + (silent). Either mount the session dir into the container, or run the + trim inside the container, or document the limitation. + +4. **OpenCode backend.** Trim is Claude-specific. OpenCode users get + `--resume` benefits skipped (different session model). If OpenCode + becomes a first-class target, we need an analogous trim or a + reasoned-down equivalent. + +5. **Fork policy.** v1 has no forking: the trim alone bounds growth. + But sessions still grow monotonically in the *kept* portion, and + long-running tasks will eventually want a hard reset. A + `context-fork-every N` directive (or agent-declared `RESET: TRUE`) + would let users break the conversation cleanly at stage boundaries. + +6. **Periodic-agent isolation.** Periodics spawn fresh sessions but + share the project's session directory. If a periodic ever needed its + own resumable continuity (e.g., a reviewer agent that learns over + runs), today there's no separate session-id tracking for it. + +7. **Saga handoff.** Across saga cycles the scoper is fresh and reads + `saga-log.md` to reconstruct context. A future variant could let the + *brute worker inside saga* keep its session across cycles when the + scoper produces a closely-related sub-plan — but only if the scoper + signals it (otherwise context bleeds between unrelated chunks). + +Next steps involve creating some test vectors to help make behavior standardized diff --git a/Cargo.lock b/Cargo.lock index cbec684..f348711 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -230,6 +230,7 @@ version = "1.0.149" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "83fc039473c5595ace860d8c4fafa220ff474b3fc6bfdb4293327f1a37e94d86" dependencies = [ + "indexmap", "itoa", "memchr", "serde", @@ -432,6 +433,7 @@ dependencies = [ name = "yoke" version = "0.1.0" dependencies = [ + "serde_json", "tempfile", ] diff --git a/Cargo.toml b/Cargo.toml index 8b45735..adf93a0 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -8,5 +8,8 @@ description = "LLM automation loop harness" name = "yoke" path = "src/main.rs" +[dependencies] +serde_json = { version = "1", default-features = false, features = ["std", "preserve_order"] } + [dev-dependencies] tempfile = "3" diff --git a/behavioral-specification.md b/behavioral-specification.md index 28e5104..d8ba19a 100644 --- a/behavioral-specification.md +++ b/behavioral-specification.md @@ -55,13 +55,16 @@ exits successfully. On FAIL, the worker retries. It is a fresh agent invocation with no shared context from the worker. Its only input is `judge.md` and the codebase state. -**2.2 — Verdict survives retries.** -On judge FAIL, `verdict.md` is NOT cleared. The worker sees the judge's -feedback on its next iteration. This is how the worker knows what went wrong. +**2.2 — Verdict survives retries and restarts.** +`verdict.md` is never cleared automatically. On judge FAIL, the worker sees +the judge's feedback on its next iteration. On bailout, the verdict remains +on disk so the user (or agent on restart) can read why the loop failed. +Use `yoke clean` or `yoke stash` to reset. -**2.3 — Guard results survive retries.** -Same as verdict — `guard-results.md` persists across brute retries so the -worker sees what the guards reported. +**2.3 — Guard results survive retries and restarts.** +Same as verdict — `guard-results.md` is never cleared automatically. It +persists across brute retries and restarts so the worker sees what the +guards reported. **2.4 — Notes status reset on retry, nothing else.** On judge FAIL, only the first line of `notes.md` is overwritten to @@ -89,9 +92,11 @@ specific snapshot. `yoke clean` auto-stashes before wiping. ### Invariants -**3.1 — Stash is a lossless round-trip.** -`stash` then `pop` produces identical `.loop/` contents. No file is lost, -truncated, or corrupted. +**3.1 — Stash clears the working directory.** +`stash` snapshots all non-dotfile files in `.loop/`, then removes them. +After stash, `.loop/` contains only dotfile entries (like `.stash/`). +`stash` then `pop` (or `checkout`) restores the original contents — no +file is lost, truncated, or corrupted. **3.2 — Auto-stash before destructive operations.** Both `clean` and `checkout` auto-stash current state before modifying it. You @@ -135,10 +140,11 @@ abort on a single chunk failure. If the scoper produces an empty `sub-plan.md`, the saga aborts. This prevents a brute loop from running with no plan. -**4.4 — Chunk state is isolated but logged.** `notes.md`, `verdict.md`, and -`guard-results.md` are cleared between chunks. Each brute run starts fresh. -Previous chunk state does not leak into the next chunk. Before clearing, -the contents of `notes.md` are appended to `saga-log.md`. +**4.4 — Chunk state persists and is logged.** `notes.md`, `verdict.md`, and +`guard-results.md` are NOT cleared between chunks. The scoper can read why +the previous chunk failed or succeeded. Before each chunk, the contents of +`notes.md` are appended to `saga-log.md`. Use `yoke clean` or `yoke stash` +for a full reset. **4.5 — Saga log is append-only.** `saga-log.md` accumulates the worker's notes from every completed chunk. It is never cleared or truncated during a @@ -169,6 +175,11 @@ is protected and `src/other.rs` is allowed. Longer prefix wins. A `guard-after` referencing a periodic that does not exist is a config error, not a silent no-op. +**5.5 — Hooks are fire-and-forget.** +A `hook` command that exits non-zero or fails to execute produces a warning on +stderr but does not affect the loop's exit code, guard evaluation, or iteration +flow. Hook output is never written to any file the agent reads. + --- ## 6. Mode Switching diff --git a/src/config.rs b/src/config.rs index f8cefe7..6738dd9 100644 --- a/src/config.rs +++ b/src/config.rs @@ -1,5 +1,5 @@ use std::fs; -use std::path::Path; +use std::path::{Path, PathBuf}; #[derive(Debug, Clone, Copy, PartialEq)] pub enum Backend { @@ -39,6 +39,11 @@ pub struct Config { pub judge_every: Option, pub max_judge_failures: u32, pub periodics: Vec, + pub hooks: Vec, + /// Resolved absolute path where NDJSON metrics rows are written. + /// Defaults to `~/.yoke/metrics` so rows survive `yoke clean` / project resets. + pub metrics_dir: PathBuf, + pub metrics_enabled: bool, } struct ConfigBuilder { @@ -52,6 +57,9 @@ struct ConfigBuilder { max_judge_failures: u32, periodics: Vec, pending_guard_afters: Vec<(String, String, usize)>, + hooks: Vec, + metrics_dir: Option, + metrics_enabled: bool, } fn cfg_err(path: &Path, line_num: usize, msg: &str) -> String { @@ -105,6 +113,9 @@ impl ConfigBuilder { max_judge_failures: 3, periodics: Vec::new(), pending_guard_afters: Vec::new(), + hooks: Vec::new(), + metrics_dir: None, + metrics_enabled: true, } } @@ -126,6 +137,19 @@ impl ConfigBuilder { "max-judge-failures" => self.max_judge_failures = parse_positive_u32(value, path, line_num, "max-judge-failures")?, "periodic" => self.periodics.push(parse_periodic(value, path, line_num)?), "guard-after" => self.pending_guard_afters.push(parse_guard_after(value, path, line_num)?), + "hook" => self.hooks.push(value.to_string()), + "metrics-dir" => self.metrics_dir = Some(value.to_string()), + "metrics" => match value.trim() { + "on" => self.metrics_enabled = true, + "off" => self.metrics_enabled = false, + other => { + return Err(cfg_err( + path, + line_num, + &format!("metrics must be 'on' or 'off', got '{}'", other), + )); + } + }, other => return Err(cfg_err(path, line_num, &format!("unknown directive '{}'", other))), } Ok(()) @@ -139,6 +163,10 @@ impl ConfigBuilder { None => return Err(cfg_err(path, ln, &format!("guard-after references unknown periodic '{}'", pname))), } } + let metrics_dir = match self.metrics_dir { + Some(raw) => crate::metrics::expand_tilde(&raw), + None => crate::metrics::default_metrics_dir(), + }; Ok(Config { max_tail: self.max_tail, log_dir: self.log_dir, @@ -149,6 +177,9 @@ impl ConfigBuilder { judge_every: self.judge_every, max_judge_failures: self.max_judge_failures, periodics, + hooks: self.hooks, + metrics_dir, + metrics_enabled: self.metrics_enabled, }) } } diff --git a/src/json.rs b/src/json.rs index 5fdbac1..61adca0 100644 --- a/src/json.rs +++ b/src/json.rs @@ -82,6 +82,32 @@ pub fn extract_str<'a>(line: &'a str, key: &str) -> Option<&'a str> { } } +/// Extract a boolean value for a given key from a flat JSON line. +/// Looks for `"key": true` or `"key": false`. +#[allow(clippy::string_slice)] // all slices at ASCII delimiter positions from .find() +pub fn extract_bool(line: &str, key: &str) -> Option { + let needle = { + let mut pat = String::with_capacity(key.len() + 3); + pat.push('"'); + pat.push_str(key); + pat.push('"'); + pat + }; + + let key_start = line.find(&needle)?; + let after_key = key_start + needle.len(); + let rest = line[after_key..].trim_start(); + let rest = rest.strip_prefix(':')?.trim_start(); + + if rest.starts_with("true") { + Some(true) + } else if rest.starts_with("false") { + Some(false) + } else { + None + } +} + /// Extract a numeric value for a given key from a flat JSON line. /// Looks for `"key": 123.45` and returns the number. /// Returns `None` if the key is not found or the value is not a number. @@ -141,4 +167,12 @@ mod tests { assert!((extract_num(line, "num_turns").unwrap() - 5.0).abs() < 1e-10); assert_eq!(extract_num(line, "missing"), None); } + + #[test] + fn test_extract_bool() { + let line = r#"{"guards_passed":true,"status_done":false}"#; + assert_eq!(extract_bool(line, "guards_passed"), Some(true)); + assert_eq!(extract_bool(line, "status_done"), Some(false)); + assert_eq!(extract_bool(line, "missing"), None); + } } diff --git a/src/main.rs b/src/main.rs index acf1976..f3df5a4 100644 --- a/src/main.rs +++ b/src/main.rs @@ -5,6 +5,8 @@ mod boundary; mod config; mod guard; mod json; +mod metrics; +mod session_trim; mod signal; mod stash; mod stream; @@ -240,6 +242,10 @@ fn print_usage() { " {}run --dry-run{} Single iteration: boundary check + guards only, no Claude", BOLD, RESET ); + eprintln!( + " {}stats{} Inspect persisted run metrics (~/.yoke/metrics/)", + BOLD, RESET + ); eprintln!(); eprintln!("{}OPTIONS:{}", BOLD, RESET); eprintln!(" --help, -h Show this help message"); @@ -347,6 +353,11 @@ fn print_init_help() { struct LoopRunner { backup_dir: PathBuf, child: Option, + /// Claude session id from the most recent agent invocation. When `Some`, + /// the next worker invocation will `--resume` it so the prompt cache and + /// the trimmed conversation prefix carry forward. Cleared on stash + /// checkout / mode switch / explicit reset. + last_session_id: Option, } impl LoopRunner { @@ -354,6 +365,7 @@ impl LoopRunner { Self { backup_dir, child: None, + last_session_id: None, } } } @@ -466,6 +478,20 @@ fn reset_notes_status() { } } +/// Detect the home directory of the default user inside a Docker image. +fn container_home(image: &str) -> String { + let output = Command::new("docker") + .args(["run", "--rm", "--entrypoint", "sh", image, "-c", "echo $HOME"]) + .output(); + match output { + Ok(o) if o.status.success() => { + let h = String::from_utf8_lossy(&o.stdout).trim().to_string(); + if h.is_empty() { "/home/sandbox".to_string() } else { h } + } + _ => "/home/sandbox".to_string(), + } +} + /// Build a docker command that runs the claude CLI inside a container. fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command { let workdir = std::env::current_dir() @@ -473,6 +499,8 @@ fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command { .to_string_lossy() .to_string(); + let container_home = container_home(image); + let mut c = Command::new("docker"); c.args([ "run", @@ -486,6 +514,16 @@ fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command { c.arg("-v").arg(format!("{}:/workspace", workdir)); c.arg("-w").arg("/workspace"); + // Mount Docker socket for Docker-outside-of-Docker + let docker_sock = "/var/run/docker.sock"; + if std::path::Path::new(docker_sock).exists() { + c.arg("-v").arg(format!("{0}:{0}", docker_sock)); + if let Ok(meta) = std::fs::metadata(docker_sock) { + use std::os::unix::fs::MetadataExt; + c.arg("--group-add").arg(meta.gid().to_string()); + } + } + for (key, val) in std::env::vars() { if key.starts_with("CLAUDE_") || key.starts_with("ANTHROPIC_") { c.arg("-e").arg(format!("{}={}", key, val)); @@ -497,15 +535,17 @@ fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command { let claude_dir = home_path.join(".claude"); if claude_dir.exists() { c.arg("-v").arg(format!( - "{}:/home/node/.claude", - claude_dir.display() + "{}:{}/.claude", + claude_dir.display(), + container_home )); } let claude_json = home_path.join(".claude.json"); if claude_json.exists() { c.arg("-v").arg(format!( - "{}:/home/node/.claude.json", - claude_json.display() + "{}:{}/.claude.json", + claude_json.display(), + container_home )); } } @@ -516,23 +556,31 @@ fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command { } /// Build a Command for invoking the agent backend. -fn build_command(config: &Config, prompt: &str) -> Command { +/// +/// `resume` carries a Claude session id when the harness wants to continue +/// the previous round's conversation (preserves prompt cache + the trimmed +/// JSONL prefix). `None` produces a fresh session. OpenCode ignores it. +fn build_command(config: &Config, prompt: &str, resume: Option<&str>) -> Command { match config.backend() { Backend::Claude => { - let claude_args = [ + let mut claude_args: Vec<&str> = vec![ "--verbose", "--output-format", "stream-json", "--include-partial-messages", "--dangerously-skip-permissions", - "-p", - prompt, ]; + if let Some(sid) = resume { + claude_args.push("--resume"); + claude_args.push(sid); + } + claude_args.push("-p"); + claude_args.push(prompt); match config.image { Some(ref image) => build_docker_claude_command(image, &claude_args), None => { let mut c = Command::new("claude"); - c.args(claude_args); + c.args(&claude_args); c } } @@ -553,8 +601,17 @@ fn build_command(config: &Config, prompt: &str) -> Command { } } +/// Wall-clock + stream summary for one agent invocation. +pub(crate) struct AgentRunStats { + pub success: bool, + pub wall_secs: f64, + pub stream: stream::StreamSummary, +} + /// Spawn an agent process, stream its output, kill it when streaming ends, -/// and wait for exit. Returns `(success, iteration_cost)`. +/// and wait for exit. Returns an `AgentRunStats` carrying success, the +/// yoke-measured wall clock around the whole spawn+stream+wait, and the +/// stream-derived summary (cost, thinking, per-tool durations). /// /// The child is stored in `runner` for cleanup-on-drop safety. /// After `filter_stream` returns (normally, broken pipe, or interrupt) @@ -568,18 +625,24 @@ fn invoke_process( log_prefix: &str, iteration: u32, prior_total: f64, -) -> (bool, f64) { + resume: Option<&str>, +) -> AgentRunStats { let backend = config.backend(); let in_container = config.image.is_some() && backend == Backend::Claude; - + let wall_start = std::time::Instant::now(); + log(&format!( - "Launching {} {}(iteration {})...", + "Launching {} {}(iteration {}){}...", label, if in_container { "in container " } else { "" }, - iteration + iteration, + match resume { + Some(sid) => format!(" [resume {}…]", sid.chars().take(8).collect::()), + None => String::new(), + } )); - let mut cmd = build_command(config, prompt); + let mut cmd = build_command(config, prompt, resume); let mut child = match cmd .stdin(Stdio::null()) @@ -595,7 +658,11 @@ fn invoke_process( Backend::OpenCode => "opencode", }; log_error(&format!("failed to spawn '{}': {}", bin, e)); - return (false, 0.0); + return AgentRunStats { + success: false, + wall_secs: wall_start.elapsed().as_secs_f64(), + stream: stream::StreamSummary::default(), + }; } }; @@ -605,10 +672,10 @@ fn invoke_process( PathBuf::from(dir).join(format!("{}-{}.jsonl", log_prefix, iteration)) }); - let mut iter_cost = 0.0; + let mut summary = stream::StreamSummary::default(); if let Some(stdout) = child.stdout.take() { runner.child = Some(child); - iter_cost = match backend { + summary = match backend { Backend::Claude => stream::filter_stream(stdout, log_path.as_deref(), prior_total), Backend::OpenCode => stream_opencode::filter_stream(stdout, log_path.as_deref(), prior_total), }; @@ -618,7 +685,11 @@ fn invoke_process( let _ = ch.kill(); } if signal::interrupted() { - return (false, iter_cost); + return AgentRunStats { + success: false, + wall_secs: wall_start.elapsed().as_secs_f64(), + stream: summary, + }; } } else { runner.child = Some(child); @@ -628,7 +699,11 @@ fn invoke_process( match child.wait() { Ok(s) => { if signal::interrupted() { - return (false, iter_cost); + return AgentRunStats { + success: false, + wall_secs: wall_start.elapsed().as_secs_f64(), + stream: summary, + }; } s.success() } @@ -649,29 +724,86 @@ fn invoke_process( label, if status { "success" } else { "failure" } )); - (status, iter_cost) + AgentRunStats { + success: status, + wall_secs: wall_start.elapsed().as_secs_f64(), + stream: summary, + } } /// Invoke the agent, piping stdout through the stream filter. -/// Returns `(success, iteration_cost)`. -fn invoke_agent(runner: &mut LoopRunner, config: &Config, iteration: u32, prior_total: f64) -> (bool, f64) { +fn invoke_agent(runner: &mut LoopRunner, config: &Config, iteration: u32, prior_total: f64) -> AgentRunStats { let prompt = "Read .loop/protocol.md and follow its instructions."; let label = match config.backend() { Backend::Claude => "Claude", Backend::OpenCode => "OpenCode", }; - invoke_process(runner, config, prompt, label, "iteration", iteration, prior_total) + let resume = runner.last_session_id.clone(); + let stats = invoke_process( + runner, config, prompt, label, "iteration", iteration, prior_total, + resume.as_deref(), + ); + if let Some(sid) = stats.stream.session_id.clone() { + runner.last_session_id = Some(sid); + } + stats } /// Invoke a periodic agent with its protocol file. /// Returns true if the invocation succeeded. +/// +/// Periodic agents always run in a fresh session: they have a different +/// protocol than the main worker and conflating their conversation prefix +/// with the worker's would poison the worker's prompt cache. Periodics +/// don't update `runner.last_session_id`. fn invoke_periodic(runner: &mut LoopRunner, config: &Config, periodic: &Periodic, iteration: u32) -> bool { let prompt = format!("Read {} and follow its instructions.", periodic.path); - let (status, _) = invoke_process( + let stats = invoke_process( runner, config, &prompt, &periodic.name, &format!("periodic-{}", periodic.name), iteration, 0.0, + None, ); - status + stats.success +} + +/// Trim the worker's resumed Claude session in place. No-op on first +/// iteration (no prior session), on docker/sandboxed runs (session file is +/// inside the container — out of reach), or if the agent didn't declare a +/// KEEP list. Any failure is logged and swallowed: better to over-pay +/// than to break the resume. +fn trim_worker_session(runner: &LoopRunner) { + let Some(sid) = runner.last_session_id.as_deref() else { return }; + let Some(home) = std::env::var_os("HOME").map(PathBuf::from) else { return }; + let Ok(cwd) = std::env::current_dir() else { return }; + let session_path = session_trim::session_file_path(&home, &cwd, sid); + if !session_path.exists() { + // Sandboxed (in-container) sessions live in the container's HOME, not the host's. + return; + } + let notes_text = match std::fs::read_to_string(NOTES_PATH) { + Ok(t) => t, + Err(_) => return, + }; + let keep = session_trim::parse_keep_list(¬es_text, &cwd); + match session_trim::trim_session(&session_path, &keep) { + Ok(stats) if stats.skipped => {} + Ok(stats) => { + log(&format!( + "Session trim: kept {}/{} records ({} dropped); KEEP={}", + stats.records_kept, + stats.records_total, + stats.records_dropped, + if stats.keep_paths.is_empty() { + "(none)".to_string() + } else { + stats.keep_paths.join(" ") + } + )); + } + Err(e) => { + log_error(&format!("Session trim failed (resuming anyway): {}", e)); + } + } } /// Render a box-drawn iteration banner to stderr. @@ -846,8 +978,9 @@ VERDICT: FAIL "; - let (status, _) = invoke_process(runner, config, judge_prompt, "judge", "judge", iteration, 0.0); - if !status { + // Judge is always a fresh session — independence per behavioral-specification §2.1. + let stats = invoke_process(runner, config, judge_prompt, "judge", "judge", iteration, 0.0, None); + if !stats.success { return false; } @@ -926,9 +1059,17 @@ fn render_violation_tree(violations: &[String]) { } } +/// Outcome of one boundary-check + guard-run pass. +struct GuardOutcome { + /// True when boundary passed AND every guard passed. + passed: bool, + /// Per-guard results (empty when the boundary check failed and the + /// guards were therefore skipped). + guard_results: Vec, +} + /// Run boundary check and all configured guards. -/// Returns true if everything passed. -fn run_all_guards(config: &Config) -> bool { +fn run_all_guards(config: &Config) -> GuardOutcome { // Boundary check first log("Running diff boundary check..."); let boundary = boundary::check(config); @@ -957,7 +1098,7 @@ fn run_all_guards(config: &Config) -> bool { md.push_str("```\nSkipped due to diff boundary violation.\n```\n\n"); } let _ = fs::write(GUARD_RESULTS_PATH, &md); - return false; + return GuardOutcome { passed: false, guard_results: Vec::new() }; } // Boundary passed — write that to results then run configured guards @@ -977,7 +1118,7 @@ fn run_all_guards(config: &Config) -> bool { render_guard_table(&guard_results); - all_passed + GuardOutcome { passed: all_passed, guard_results } } fn print_clean_help() { @@ -1231,6 +1372,7 @@ enum JudgeEveryAction { } /// Evaluate judge-every logic: fire judge when worker signals DONE or at cadence checkpoints. +/// On firing, records wall time into `judge_secs_out` for the metrics row. fn evaluate_judge_every( runner: &mut LoopRunner, config: &Config, @@ -1238,12 +1380,15 @@ fn evaluate_judge_every( guards_passed: bool, consecutive_judge_failures: &mut u32, judge_every: u32, + judge_secs_out: &mut Option, ) -> JudgeEveryAction { if guards_passed && is_status_done(NOTES_PATH) { // Always fire judge when worker signals DONE eprintln!(); render_section_banner("Judge (DONE)", "Worker DONE — invoking judge", BLUE); + let judge_start = std::time::Instant::now(); let pass = invoke_judge(runner, config, iteration); + *judge_secs_out = Some(judge_start.elapsed().as_secs_f64()); if pass { return JudgeEveryAction::Pass; } @@ -1264,7 +1409,9 @@ fn evaluate_judge_every( &format!("Quality checkpoint (iteration {:>4})", iteration), BLUE, ); + let judge_start = std::time::Instant::now(); let pass = invoke_judge(runner, config, iteration); + *judge_secs_out = Some(judge_start.elapsed().as_secs_f64()); if pass { *consecutive_judge_failures = 0; } else { @@ -1334,10 +1481,52 @@ fn fire_periodic_agents( } } +/// Run fire-and-forget hook commands. Non-zero exits warn but never stop the loop. +fn run_hooks(config: &Config, iteration: u32) { + for cmd in &config.hooks { + let result = Command::new("sh") + .arg("-c") + .arg(cmd) + .env("YOKE_ITERATION", iteration.to_string()) + .stdin(Stdio::null()) + .stdout(Stdio::inherit()) + .stderr(Stdio::inherit()) + .status(); + + match result { + Ok(status) if !status.success() => { + log(&format!( + "{}WARNING: hook exited with {}: {}{}", + ORANGE, + status.code().map_or("signal".to_string(), |c| c.to_string()), + cmd, + RESET + )); + } + Err(e) => { + log(&format!( + "{}WARNING: hook failed to execute: {} — {}{}", + ORANGE, cmd, e, RESET + )); + } + Ok(_) => {} + } + } +} + +/// Per-iteration data captured by `run_iteration_step` for metrics use. +struct IterationStepStats { + guards_passed: bool, + agent_secs: f64, + stream: stream::StreamSummary, + guards_secs: f64, + guard_results: Vec, +} + /// Result of running the worker + guards for a single plan-loop iteration. enum IterationStepResult { /// Worker + guards succeeded, loop should continue. - Continue { guards_passed: bool, cost: f64 }, + Continue(IterationStepStats), /// Terminal outcome — return immediately from the loop. Terminal(PlanLoopOutcome), } @@ -1361,8 +1550,8 @@ fn run_iteration_step( } )); } else { - let (success, iter_cost) = invoke_agent(runner, config, iteration, prior_total); - if !success { + let stats = invoke_agent(runner, config, iteration, prior_total); + if !stats.success { log_error("Claude invocation failed — aborting loop"); return IterationStepResult::Terminal(PlanLoopOutcome::Error); } @@ -1370,8 +1559,10 @@ fn run_iteration_step( log("Interrupted \u{2014} shutting down"); return IterationStepResult::Terminal(PlanLoopOutcome::Interrupt); } - let guards_passed = run_all_guards(config); - if guards_passed { + let guards_start = std::time::Instant::now(); + let guard_outcome = run_all_guards(config); + let guards_secs = guards_start.elapsed().as_secs_f64(); + if guard_outcome.passed { log(&format!("{}{}All guards passed{}", GREEN, BOLD, RESET)); } else { log(&format!( @@ -1379,12 +1570,18 @@ fn run_iteration_step( ORANGE, RESET )); } - return IterationStepResult::Continue { guards_passed, cost: iter_cost }; + return IterationStepResult::Continue(IterationStepStats { + guards_passed: guard_outcome.passed, + agent_secs: stats.wall_secs, + stream: stats.stream, + guards_secs, + guard_results: guard_outcome.guard_results, + }); } // Dry-run path: run guards and exit after one iteration. - let guards_passed = run_all_guards(config); - if guards_passed { + let guard_outcome = run_all_guards(config); + if guard_outcome.passed { log(&format!("{}{}All guards passed{}", GREEN, BOLD, RESET)); log("(dry-run) Guards passed \u{2014} exiting after one iteration"); IterationStepResult::Terminal(PlanLoopOutcome::Done) @@ -1403,7 +1600,14 @@ fn run_iteration_step( /// - `plan_path`: path to the plan file (e.g. PLAN_PATH or SUB_PLAN_PATH) /// - `dry_run`: if true, skip Claude invocation (one iteration only) /// - `nested`: if true, running inside brute (adjusts output banners) -fn run_plan_loop(config: &Config, plan_path: &str, dry_run: bool, nested: bool) -> PlanLoopOutcome { +/// - `mctx`: metrics context carrying run_id + project + output dir +fn run_plan_loop( + config: &Config, + plan_path: &str, + dry_run: bool, + nested: bool, + recorder: &mut metrics::RunRecorder, +) -> PlanLoopOutcome { preflight(plan_path, !nested); validate_loop_config(config); log("Preflight OK"); @@ -1428,6 +1632,9 @@ fn run_plan_loop(config: &Config, plan_path: &str, dry_run: bool, nested: bool) return PlanLoopOutcome::Interrupt; } iteration += 1; + let iter_started_at = metrics::unix_now(); + let iter_wall_start = std::time::Instant::now(); + eprintln!(); let label = if nested { "Plan Iteration" } else { "Iteration" }; render_iteration_banner(label, iteration, nested); @@ -1436,30 +1643,81 @@ fn run_plan_loop(config: &Config, plan_path: &str, dry_run: bool, nested: bool) eprintln!("{}", format_progress_bar(completed, total)); } + let restore_start = std::time::Instant::now(); restore_files(&runner.backup_dir, &protected); + let restore_ms = restore_start.elapsed().as_millis() as u64; - let guards_passed = match run_iteration_step(&mut runner, config, iteration, total_cost, dry_run) { - IterationStepResult::Continue { guards_passed, cost } => { - total_cost += cost; - guards_passed - } + // Trim the worker's resumed session, if any — drops bash output, + // thinking, and Reads not on the agent's KEEP list, so the prefix + // we re-send to Claude stays compact. + trim_worker_session(&runner); + + let step_stats = match run_iteration_step(&mut runner, config, iteration, total_cost, dry_run) { + IterationStepResult::Continue(s) => s, IterationStepResult::Terminal(outcome) => return outcome, }; + total_cost += step_stats.stream.cost_usd; + let periodics_start = std::time::Instant::now(); fire_periodic_agents(&mut runner, config, iteration); + let periodics_secs = periodics_start.elapsed().as_secs_f64(); + let hooks_start = std::time::Instant::now(); + run_hooks(config, iteration); + let hooks_secs = hooks_start.elapsed().as_secs_f64(); + + let mut judge_secs: Option = None; + let mut early_return: Option = None; if let Some(judge_every) = config.judge_every { match evaluate_judge_every( - &mut runner, config, iteration, guards_passed, - &mut consecutive_judge_failures, judge_every, + &mut runner, config, iteration, step_stats.guards_passed, + &mut consecutive_judge_failures, judge_every, &mut judge_secs, ) { - JudgeEveryAction::Pass => return PlanLoopOutcome::JudgePass, - JudgeEveryAction::Bailout => return PlanLoopOutcome::JudgeBailout, - JudgeEveryAction::Continue => continue, + JudgeEveryAction::Pass => early_return = Some(PlanLoopOutcome::JudgePass), + JudgeEveryAction::Bailout => early_return = Some(PlanLoopOutcome::JudgeBailout), + JudgeEveryAction::Continue => {} } } - if guards_passed && is_status_done(NOTES_PATH) { + let status_done = step_stats.guards_passed && is_status_done(NOTES_PATH); + + // Build & flush iteration metrics row before any return path so + // every completed iteration is recorded. + let iter_metrics = metrics::IterationMetrics { + run_id: recorder.ctx.run_id.clone(), + project_slug: recorder.ctx.project_slug.clone(), + mode: recorder.ctx.mode, + iteration, + started_at: iter_started_at, + wall_secs: iter_wall_start.elapsed().as_secs_f64(), + restore_ms, + agent_secs: step_stats.agent_secs, + guards_secs: step_stats.guards_secs, + periodics_secs, + hooks_secs, + judge_secs, + agent_reported_secs: step_stats.stream.agent_reported_secs, + cost_usd: step_stats.stream.cost_usd, + num_turns: step_stats.stream.num_turns, + thinking_secs: step_stats.stream.thinking_secs, + tool_counts: step_stats.stream.tool_counts.clone(), + tool_durations_secs: step_stats.stream.tool_durations_secs.clone(), + guards: step_stats.guard_results.iter().map(|g| metrics::GuardRow { + name: g.name.clone(), + passed: g.passed, + skipped: g.skipped, + elapsed_secs: g.elapsed_secs, + }).collect(), + guards_passed: step_stats.guards_passed, + status_done, + }; + recorder.record_iteration(&iter_metrics); + + if let Some(outcome) = early_return { + return outcome; + } + + if status_done { eprintln!(); eprintln!( "{}{} STATUS: DONE and all guards pass \u{2014} loop complete {}", @@ -1482,10 +1740,11 @@ fn run_loop(dry_run: bool) -> i32 { }; log(&format!( - "Config loaded: max_tail={}, {} scope rules, {} guards, backend={}{}", + "Config loaded: max_tail={}, {} scope rules, {} guards, {} hooks, backend={}{}", config.max_tail, config.scope_rules.len(), config.guards.len(), + config.hooks.len(), match config.backend() { Backend::Claude => "claude", Backend::OpenCode => config.model.as_ref().map_or("opencode", |m| m.as_str()), @@ -1496,13 +1755,71 @@ fn run_loop(dry_run: bool) -> i32 { ) )); - match run_plan_loop(&config, PLAN_PATH, dry_run, false) { + let mctx = metrics::MetricsContext::new( + config.metrics_dir.clone(), + "loop", + config.metrics_enabled, + ); + if mctx.enabled { + log(&format!( + "Metrics: run_id={}, file={}", + mctx.run_id, + mctx.iteration_path().display() + )); + } + let mut recorder = metrics::RunRecorder::new(mctx); + recorder.on_finalize(|r| render_run_summary(r)); + + let outcome = run_plan_loop(&config, PLAN_PATH, dry_run, false, &mut recorder); + recorder.set_outcome(plan_outcome_label(&outcome)); + drop(recorder); + match outcome { PlanLoopOutcome::Done | PlanLoopOutcome::JudgePass => 0, PlanLoopOutcome::JudgeBailout | PlanLoopOutcome::Error => 1, PlanLoopOutcome::Interrupt => 130, } } +fn plan_outcome_label(o: &PlanLoopOutcome) -> &'static str { + match o { + PlanLoopOutcome::Done => "done", + PlanLoopOutcome::JudgePass => "judge_pass", + PlanLoopOutcome::JudgeBailout => "bailout", + PlanLoopOutcome::Error => "error", + PlanLoopOutcome::Interrupt => "interrupt", + } +} + +fn brute_outcome_label(o: &BruteResult) -> &'static str { + match o { + BruteResult::Pass => "judge_pass", + BruteResult::Bailout => "bailout", + BruteResult::Interrupt => "interrupt", + BruteResult::Error => "error", + } +} + +/// Render the end-of-run metrics table to stderr. +fn render_run_summary(r: &metrics::RunMetrics) { + eprintln!(); + eprintln!("{}{}╔══════════════════════════════════════════════════════════════════╗{}", BOLD, ORANGE, RESET); + eprintln!("{}{}║ Run summary ║{}", BOLD, ORANGE, RESET); + eprintln!("{}{}╚══════════════════════════════════════════════════════════════════╝{}", BOLD, ORANGE, RESET); + let dim = DIM; + let reset = RESET; + eprintln!(" {}mode:{} {}", dim, reset, r.mode); + eprintln!(" {}run_id:{} {}", dim, reset, r.run_id); + eprintln!(" {}project:{} {}", dim, reset, r.project_slug); + eprintln!(" {}outcome:{} {}", dim, reset, r.outcome); + eprintln!(" {}iterations:{} {}", dim, reset, r.iterations); + eprintln!(" {}wall:{} {:.1}s", dim, reset, r.total_wall_secs); + eprintln!(" {}agent:{} {:.1}s", dim, reset, r.total_agent_secs); + eprintln!(" {}thinking:{} {:.1}s", dim, reset, r.total_thinking_secs); + eprintln!(" {}guards:{} {:.1}s", dim, reset, r.total_guards_secs); + eprintln!(" {}cost:{} ${:.2}", dim, reset, r.total_cost_usd); + eprintln!(); +} + /// Result of a brute loop run. enum BruteResult { /// Judge said PASS — feature is complete. @@ -1516,7 +1833,7 @@ enum BruteResult { } /// Protected files for the brute runner. -const BRUTE_PROTECTED_FILES: &[&str] = &[PROTOCOL_PATH, JUDGE_PATH, CONF_PATH]; +const BRUTE_PROTECTED_FILES: &[&str] = &[PROTOCOL_PATH, PLAN_PATH, JUDGE_PATH, CONF_PATH]; fn run_brute_inner(dry_run: bool) -> BruteResult { // Load config @@ -1568,14 +1885,25 @@ fn run_brute_inner(dry_run: bool) -> BruteResult { } } - // Clear verdict and guard results for the first brute attempt. - // Subsequent retries preserve these so the worker can read feedback. - let _ = fs::write(VERDICT_PATH, ""); - let _ = fs::write(GUARD_RESULTS_PATH, ""); - log("Preflight OK"); - run_brute_core(&config, PLAN_PATH, dry_run) + let mctx = metrics::MetricsContext::new( + config.metrics_dir.clone(), + "brute", + config.metrics_enabled, + ); + if mctx.enabled { + log(&format!( + "Metrics: run_id={}, file={}", + mctx.run_id, + mctx.iteration_path().display() + )); + } + let mut recorder = metrics::RunRecorder::new(mctx); + recorder.on_finalize(|r| render_run_summary(r)); + let result = run_brute_core(&config, PLAN_PATH, dry_run, &mut recorder); + recorder.set_outcome(brute_outcome_label(&result)); + result } /// Core brute loop logic, parameterized by plan path. Used by both standalone @@ -1584,7 +1912,13 @@ fn run_brute_inner(dry_run: bool) -> BruteResult { /// - `config`: already-loaded Config /// - `plan_path`: path to the plan file /// - `dry_run_inner`: if true, skip Claude invocation (one iteration only) -fn run_brute_core(config: &Config, plan_path: &str, dry_run: bool) -> BruteResult { +/// - `mctx`: metrics context (shared with nested plan loop) +fn run_brute_core( + config: &Config, + plan_path: &str, + dry_run: bool, + recorder: &mut metrics::RunRecorder, +) -> BruteResult { // Backup protected files let backup_dir = backup_files(BRUTE_PROTECTED_FILES); log(&format!("Backups in {}", backup_dir.display())); @@ -1615,7 +1949,7 @@ fn run_brute_core(config: &Config, plan_path: &str, dry_run: bool) -> BruteResul log("(dry-run) Skipping worker invocation"); } else { log("Using plan runner as worker..."); - match run_plan_loop(config, plan_path, false, true) { + match run_plan_loop(config, plan_path, false, true, recorder) { PlanLoopOutcome::JudgePass => { // Embedded judge already passed — skip standalone judge log("Plan runner completed with embedded judge PASS"); @@ -1646,8 +1980,8 @@ fn run_brute_core(config: &Config, plan_path: &str, dry_run: bool) -> BruteResul // Run guards (skip when not dry-run — the plan loop already ran them) if dry_run { - let passed = run_all_guards(config); - if !passed { + let outcome = run_all_guards(config); + if !outcome.passed { log(&format!( "{}Guards failed{}", ORANGE, RESET )); @@ -1724,10 +2058,11 @@ fn run_brute(dry_run: bool) -> i32 { /// Returns true if the invocation succeeded. fn invoke_scoper(runner: &mut LoopRunner, config: &Config, cycle: u32) -> bool { let scoper_prompt = "Read .loop/saga-protocol.md and follow its instructions."; - let (status, _) = invoke_process( - runner, config, scoper_prompt, "scoper", "scoper", cycle, 0.0, + // Scoper is always a fresh session — saga cycles are independent by design. + let stats = invoke_process( + runner, config, scoper_prompt, "scoper", "scoper", cycle, 0.0, None, ); - status + stats.success } /// Run the saga loop: scoper (Agent 1) decomposes the spec into sub-plans, @@ -1790,6 +2125,7 @@ fn run_saga_cycle( cycle: u32, dry_run: bool, saga_protected: &[&str], + recorder: &mut metrics::RunRecorder, ) -> Option { restore_files(&runner.backup_dir, saga_protected); @@ -1864,11 +2200,11 @@ fn run_saga_cycle( } } - let _ = fs::write(NOTES_PATH, ""); - let _ = fs::write(VERDICT_PATH, ""); - let _ = fs::write(GUARD_RESULTS_PATH, ""); + // Preserve notes, verdict, and guard-results between cycles so the + // scoper (and the user) can read why the previous chunk failed or + // succeeded. Use `yoke clean` or `yoke stash` for a full reset. - match run_brute_core(config, SUB_PLAN_PATH, false) { + match run_brute_core(config, SUB_PLAN_PATH, false, recorder) { BruteResult::Pass => { log(&format!( "{}Sub-plan passed \u{2014} looping back to scoper{}", @@ -1925,19 +2261,40 @@ fn run_saga(dry_run: bool) -> i32 { let backup_dir = backup_files(&saga_protected); log(&format!("Backups in {}", backup_dir.display())); + let mctx = metrics::MetricsContext::new( + config.metrics_dir.clone(), + "saga", + config.metrics_enabled, + ); + if mctx.enabled { + log(&format!( + "Metrics: run_id={}, file={}", + mctx.run_id, + mctx.iteration_path().display() + )); + } + let mut recorder = metrics::RunRecorder::new(mctx); + recorder.on_finalize(|r| render_run_summary(r)); + let mut runner = LoopRunner::new(backup_dir); let mut cycle: u32 = 0; loop { if signal::interrupted() { log("Interrupted \u{2014} shutting down"); + // outcome defaults to "interrupt" on Drop return 130; } cycle += 1; eprintln!(); render_iteration_banner("Saga Cycle", cycle, false); - if let Some(exit_code) = run_saga_cycle(&mut runner, &config, cycle, dry_run, &saga_protected) { + if let Some(exit_code) = run_saga_cycle(&mut runner, &config, cycle, dry_run, &saga_protected, &mut recorder) { + recorder.set_outcome(match exit_code { + 0 => "done", + 130 => "interrupt", + _ => "error", + }); return exit_code; } } @@ -2032,6 +2389,185 @@ fn cmd_layer(args: &[String]) -> ! { process::exit(apply_layers(&names_arg)); } +fn print_stats_help() { + eprintln!( + "{}{}[yoke stats]{} inspect persisted run metrics", + ORANGE, BOLD, RESET + ); + eprintln!(); + eprintln!("{}USAGE:{}", BOLD, RESET); + eprintln!(" yoke stats List recent runs for the current project"); + eprintln!(" yoke stats --last N Limit to last N runs"); + eprintln!(" yoke stats --project Target a different project"); + eprintln!(" yoke stats --run Show per-iteration rows for one run"); + eprintln!(); + eprintln!( + "Reads NDJSON from {}~/.yoke/metrics//{} (or `metrics-dir` from yoke.conf).", + DIM, RESET + ); +} + +/// Resolve the metrics root directory, preferring an existing yoke.conf +/// (so `metrics-dir` overrides apply) and falling back to the default +/// `~/.yoke/metrics/` when no config is present. +fn resolve_metrics_dir() -> PathBuf { + if Path::new(CONF_PATH).exists() { + if let Ok(c) = Config::load(Path::new(CONF_PATH)) { + return c.metrics_dir; + } + } + metrics::default_metrics_dir() +} + +fn cmd_stats(args: &[String]) -> ! { + if args.get(2).is_some_and(|a| a == "--help" || a == "-h") { + print_stats_help(); + process::exit(0); + } + + let mut last: Option = None; + let mut project: Option = None; + let mut run_id: Option = None; + let mut i = 2; + while i < args.len() { + match args[i].as_str() { + "--last" => { + let v = match args.get(i + 1) { + Some(v) => v, + None => { + log_error("--last requires a value"); + process::exit(2); + } + }; + last = Some(match v.parse() { + Ok(n) => n, + Err(_) => { + log_error(&format!("--last expects an integer, got '{}'", v)); + process::exit(2); + } + }); + i += 2; + } + "--project" => { + project = Some(match args.get(i + 1) { + Some(v) => v.clone(), + None => { + log_error("--project requires a slug"); + process::exit(2); + } + }); + i += 2; + } + "--run" => { + run_id = Some(match args.get(i + 1) { + Some(v) => v.clone(), + None => { + log_error("--run requires a run-id"); + process::exit(2); + } + }); + i += 2; + } + other => { + log_error(&format!("unknown flag '{}'", other)); + print_stats_help(); + process::exit(2); + } + } + } + + let metrics_dir = resolve_metrics_dir(); + let slug = project.unwrap_or_else(metrics::project_slug); + let slug_dir = metrics_dir.join(&slug); + + if let Some(rid) = run_id { + process::exit(stats_show_run(&slug_dir, &rid, &slug)); + } else { + process::exit(stats_list_runs(&slug_dir, &slug, last)); + } +} + +fn stats_list_runs(slug_dir: &Path, slug: &str, last: Option) -> i32 { + let mut runs = metrics::read_runs(slug_dir); + if runs.is_empty() { + log(&format!( + "No runs recorded for project '{}' at {}", + slug, + slug_dir.display() + )); + log("Run `yoke run` to populate metrics."); + return 0; + } + // run_id format is lex-sortable → desc gives most-recent first + runs.sort_by(|a, b| b.run_id.cmp(&a.run_id)); + if let Some(n) = last { + runs.truncate(n); + } + + log(&format!( + "Runs for project '{}' ({} shown, source: {})", + slug, + runs.len(), + slug_dir.display() + )); + println!(); + println!( + " {}{:<22} {:<6} {:<11} {:>4} {:>8} {:>8}{}", + BOLD, "RUN_ID", "MODE", "OUTCOME", "ITER", "WALL", "COST", RESET + ); + for r in &runs { + println!( + " {:<22} {:<6} {:<11} {:>4} {:>7.1}s {:>7.2}", + r.run_id, r.mode, r.outcome, r.iterations, r.total_wall_secs, r.total_cost_usd + ); + } + 0 +} + +fn stats_show_run(slug_dir: &Path, run_id: &str, slug: &str) -> i32 { + let path = slug_dir.join(format!("{}.ndjson", run_id)); + if !path.exists() { + log_error(&format!("no metrics file at {}", path.display())); + return 1; + } + let iters = metrics::read_iterations(&path); + if iters.is_empty() { + log(&format!( + "Run {} has no iteration rows ({})", + run_id, + path.display() + )); + return 0; + } + log(&format!( + "Run {} (project '{}', {} iteration{})", + run_id, + slug, + iters.len(), + if iters.len() == 1 { "" } else { "s" } + )); + println!(); + println!( + " {}{:>4} {:>7} {:>7} {:>7} {:>7} {:>5} {:>7} {:>6} {:>4}{}", + BOLD, "ITER", "WALL", "AGENT", "THINK", "GUARDS", "TURNS", "COST", "GUARDS", "DONE", RESET + ); + for r in &iters { + println!( + " {:>4} {:>6.1}s {:>6.1}s {:>6.1}s {:>6.1}s {:>5} $ {:>5.2} {:>6} {:>4}", + r.iteration, + r.wall_secs, + r.agent_secs, + r.thinking_secs, + r.guards_secs, + r.num_turns, + r.cost_usd, + if r.guards_passed { "pass" } else { "fail" }, + if r.status_done { "yes" } else { "no" } + ); + } + 0 +} + fn cmd_run(args: &[String]) -> ! { if args.get(2).is_some_and(|a| a == "--help" || a == "-h") { print_run_help(); @@ -2091,6 +2627,7 @@ fn main() { "stash" => cmd_stash(&args), "layer" => cmd_layer(&args), "run" => cmd_run(&args), + "stats" => cmd_stats(&args), "--help" | "-h" | "help" => { print_usage(); } diff --git a/src/metrics.rs b/src/metrics.rs new file mode 100644 index 0000000..7f53fb4 --- /dev/null +++ b/src/metrics.rs @@ -0,0 +1,750 @@ +//! Persistent metrics for yoke runs. Rows are written under `~/.yoke/metrics/` +//! (or the configured `metrics-dir`) as NDJSON — one row per iteration, one +//! row per completed run. Storage lives outside the project tree so it +//! survives `yoke clean`, `yoke stash`, branch resets, and project deletes. + +use std::collections::BTreeMap; +use std::fs::{self, OpenOptions}; +use std::io::Write; +use std::path::{Path, PathBuf}; +use std::time::{SystemTime, UNIX_EPOCH}; + +#[derive(Debug, Clone)] +pub struct GuardRow { + pub name: String, + pub passed: bool, + pub skipped: bool, + pub elapsed_secs: f64, +} + +#[derive(Debug, Clone)] +pub struct IterationMetrics { + pub run_id: String, + pub project_slug: String, + pub mode: &'static str, + pub iteration: u32, + pub started_at: u64, + + pub wall_secs: f64, + pub restore_ms: u64, + pub agent_secs: f64, + pub guards_secs: f64, + pub periodics_secs: f64, + pub hooks_secs: f64, + pub judge_secs: Option, + + pub agent_reported_secs: Option, + pub cost_usd: f64, + pub num_turns: u32, + + pub thinking_secs: f64, + + pub tool_counts: BTreeMap, + pub tool_durations_secs: BTreeMap, + + pub guards: Vec, + + pub guards_passed: bool, + pub status_done: bool, +} + +#[derive(Debug, Clone)] +pub struct RunMetrics { + pub run_id: String, + pub project_slug: String, + pub mode: &'static str, + pub started_at: u64, + pub ended_at: u64, + pub iterations: u32, + pub outcome: &'static str, + + pub total_wall_secs: f64, + pub total_agent_secs: f64, + pub total_thinking_secs: f64, + pub total_guards_secs: f64, + pub total_cost_usd: f64, +} + +/// Run-scoped state that travels with the loop: identifies the run, knows +/// where to write, and tells the loop whether persistence is enabled. +#[derive(Debug, Clone)] +pub struct MetricsContext { + pub run_id: String, + pub project_slug: String, + pub mode: &'static str, + pub root_dir: PathBuf, + pub enabled: bool, +} + +impl MetricsContext { + pub fn new(root_dir: PathBuf, mode: &'static str, enabled: bool) -> Self { + Self { + run_id: new_run_id(), + project_slug: project_slug(), + mode, + root_dir, + enabled, + } + } + + pub fn iteration_path(&self) -> PathBuf { + self.root_dir + .join(&self.project_slug) + .join(format!("{}.ndjson", self.run_id)) + } + + pub fn runs_path(&self) -> PathBuf { + self.root_dir.join(&self.project_slug).join("runs.ndjson") + } +} + +/// Accumulates per-iteration totals and writes a final `RunMetrics` row +/// on Drop. Drop runs on normal exit, on early-return, and on the first +/// SIGINT (which sets the interrupted flag and lets the loop unwind). A +/// second SIGINT calls `_exit` directly and bypasses Drop — accepted +/// tradeoff for a panic-button. +pub struct RunRecorder { + pub ctx: MetricsContext, + started_at: u64, + wall_start: std::time::Instant, + iterations: u32, + total_agent_secs: f64, + total_thinking_secs: f64, + total_guards_secs: f64, + total_cost_usd: f64, + outcome: &'static str, + /// Closure invoked with the finalized RunMetrics from Drop. Allows the + /// caller to render an end-of-run summary table without forcing Drop + /// to know about ANSI rendering or print to stderr unconditionally. + on_finalize: Option>, +} + +impl RunRecorder { + pub fn new(ctx: MetricsContext) -> Self { + Self { + ctx, + started_at: unix_now(), + wall_start: std::time::Instant::now(), + iterations: 0, + total_agent_secs: 0.0, + total_thinking_secs: 0.0, + total_guards_secs: 0.0, + total_cost_usd: 0.0, + // Default to "interrupt" so an unwinding stack still records + // something meaningful — clean exit paths override this. + outcome: "interrupt", + on_finalize: None, + } + } + + pub fn on_finalize(&mut self, cb: F) + where + F: FnMut(&RunMetrics) + Send + 'static, + { + self.on_finalize = Some(Box::new(cb)); + } + + /// Write the iteration row and accumulate its totals in one step. + pub fn record_iteration(&mut self, m: &IterationMetrics) { + write_iteration_row(&self.ctx, m); + self.iterations += 1; + self.total_agent_secs += m.agent_secs; + self.total_thinking_secs += m.thinking_secs; + self.total_guards_secs += m.guards_secs; + self.total_cost_usd += m.cost_usd; + } + + pub fn set_outcome(&mut self, outcome: &'static str) { + self.outcome = outcome; + } + + fn finalize(&mut self) -> RunMetrics { + RunMetrics { + run_id: self.ctx.run_id.clone(), + project_slug: self.ctx.project_slug.clone(), + mode: self.ctx.mode, + started_at: self.started_at, + ended_at: unix_now(), + iterations: self.iterations, + outcome: self.outcome, + total_wall_secs: self.wall_start.elapsed().as_secs_f64(), + total_agent_secs: self.total_agent_secs, + total_thinking_secs: self.total_thinking_secs, + total_guards_secs: self.total_guards_secs, + total_cost_usd: self.total_cost_usd, + } + } +} + +impl Drop for RunRecorder { + fn drop(&mut self) { + let run = self.finalize(); + write_run_row(&self.ctx, &run); + if let Some(mut cb) = self.on_finalize.take() { + cb(&run); + } + } +} + +pub fn unix_now() -> u64 { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map(|d| d.as_secs()) + .unwrap_or(0) +} + +/// Sortable run id: `YYYYMMDDTHHMMSS-NNNNNN`, where the suffix is the +/// six-digit sub-second microsecond count. Lexicographic sort matches +/// chronological order. No external time crate dep. +pub fn new_run_id() -> String { + let now = SystemTime::now().duration_since(UNIX_EPOCH).unwrap_or_default(); + let secs = now.as_secs(); + let micros = now.subsec_micros(); + let (y, m, d, hh, mm, ss) = unix_to_utc(secs); + format!( + "{:04}{:02}{:02}T{:02}{:02}{:02}-{:06}", + y, m, d, hh, mm, ss, micros + ) +} + +/// Convert unix-seconds-since-epoch (UTC) to (year, month, day, hour, minute, second). +/// Howard Hinnant's date algorithm — integer-only, no leap-second handling. +fn unix_to_utc(secs: u64) -> (i32, u32, u32, u32, u32, u32) { + let days = (secs / 86400) as i64; + let tod = secs % 86400; + let hh = (tod / 3600) as u32; + let mm = ((tod % 3600) / 60) as u32; + let ss = (tod % 60) as u32; + + let z = days + 719468; + let era = if z >= 0 { z / 146097 } else { (z - 146096) / 146097 }; + let doe = (z - era * 146097) as u64; + let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365; + let y = yoe as i64 + era * 400; + let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); + let mp = (5 * doy + 2) / 153; + let d = doy - (153 * mp + 2) / 5 + 1; + let m = if mp < 10 { mp + 3 } else { mp - 9 }; + let y = if m <= 2 { y + 1 } else { y }; + + (y as i32, m as u32, d as u32, hh, mm, ss) +} + +/// Stable identifier for the current working directory: basename + FNV-1a +/// hash of the absolute path. Two checkouts with the same basename get +/// different slugs. +pub fn project_slug() -> String { + let cwd = std::env::current_dir().unwrap_or_else(|_| PathBuf::from(".")); + let abs = cwd.canonicalize().unwrap_or(cwd.clone()); + let base = abs + .file_name() + .and_then(|s| s.to_str()) + .unwrap_or("project"); + let hash = fnv1a_32(abs.to_string_lossy().as_bytes()); + format!("{}-{:08x}", sanitize(base), hash) +} + +fn sanitize(s: &str) -> String { + s.chars() + .map(|c| if c.is_ascii_alphanumeric() || c == '-' || c == '_' { c } else { '_' }) + .collect() +} + +fn fnv1a_32(bytes: &[u8]) -> u32 { + let mut h: u32 = 0x811c9dc5; + for &b in bytes { + h ^= b as u32; + h = h.wrapping_mul(0x01000193); + } + h +} + +/// Expand a leading `~` or `~/` against `$HOME`. Returns the input +/// unchanged if no tilde prefix or no HOME. +pub fn expand_tilde(s: &str) -> PathBuf { + if let Some(home) = std::env::var_os("HOME") { + if s == "~" { + return PathBuf::from(home); + } + if let Some(rest) = s.strip_prefix("~/") { + return PathBuf::from(home).join(rest); + } + } + PathBuf::from(s) +} + +pub fn default_metrics_dir() -> PathBuf { + expand_tilde("~/.yoke/metrics") +} + +pub fn write_iteration_row(ctx: &MetricsContext, m: &IterationMetrics) { + if !ctx.enabled { + return; + } + append_line(&ctx.iteration_path(), &serialize_iteration(m)); +} + +pub fn write_run_row(ctx: &MetricsContext, r: &RunMetrics) { + if !ctx.enabled { + return; + } + append_line(&ctx.runs_path(), &serialize_run(r)); +} + +fn append_line(path: &Path, line: &str) { + if let Some(parent) = path.parent() { + if let Err(e) = fs::create_dir_all(parent) { + eprintln!( + "[yoke] WARNING: cannot create metrics dir {}: {}", + parent.display(), + e + ); + return; + } + } + match OpenOptions::new().create(true).append(true).open(path) { + Ok(mut f) => { + // Single write_all keeps the row atomic on POSIX for small lines. + let mut buf = line.to_string(); + buf.push('\n'); + if let Err(e) = f.write_all(buf.as_bytes()) { + eprintln!("[yoke] WARNING: failed to write metrics row: {}", e); + } + } + Err(e) => { + eprintln!( + "[yoke] WARNING: cannot open metrics file {}: {}", + path.display(), + e + ); + } + } +} + +// ─── JSON emission ──────────────────────────────────────────────────────── + +fn serialize_iteration(m: &IterationMetrics) -> String { + let mut s = String::with_capacity(512); + s.push('{'); + push_str_field(&mut s, "run_id", &m.run_id, true); + push_str_field(&mut s, "project_slug", &m.project_slug, false); + push_str_field(&mut s, "mode", m.mode, false); + push_u32_field(&mut s, "iteration", m.iteration, false); + push_u64_field(&mut s, "started_at", m.started_at, false); + push_f64_field(&mut s, "wall_secs", m.wall_secs, false); + push_u64_field(&mut s, "restore_ms", m.restore_ms, false); + push_f64_field(&mut s, "agent_secs", m.agent_secs, false); + push_f64_field(&mut s, "guards_secs", m.guards_secs, false); + push_f64_field(&mut s, "periodics_secs", m.periodics_secs, false); + push_f64_field(&mut s, "hooks_secs", m.hooks_secs, false); + push_opt_f64_field(&mut s, "judge_secs", m.judge_secs); + push_opt_f64_field(&mut s, "agent_reported_secs", m.agent_reported_secs); + push_f64_field(&mut s, "cost_usd", m.cost_usd, false); + push_u32_field(&mut s, "num_turns", m.num_turns, false); + push_f64_field(&mut s, "thinking_secs", m.thinking_secs, false); + s.push_str(",\"tool_counts\":"); + push_map_u32(&mut s, &m.tool_counts); + s.push_str(",\"tool_durations_secs\":"); + push_map_f64(&mut s, &m.tool_durations_secs); + s.push_str(",\"guards\":["); + for (i, g) in m.guards.iter().enumerate() { + if i > 0 { + s.push(','); + } + s.push('{'); + push_str_field(&mut s, "name", &g.name, true); + push_bool_field(&mut s, "passed", g.passed, false); + push_bool_field(&mut s, "skipped", g.skipped, false); + push_f64_field(&mut s, "elapsed_secs", g.elapsed_secs, false); + s.push('}'); + } + s.push(']'); + push_bool_field(&mut s, "guards_passed", m.guards_passed, false); + push_bool_field(&mut s, "status_done", m.status_done, false); + s.push('}'); + s +} + +fn serialize_run(r: &RunMetrics) -> String { + let mut s = String::with_capacity(256); + s.push('{'); + push_str_field(&mut s, "run_id", &r.run_id, true); + push_str_field(&mut s, "project_slug", &r.project_slug, false); + push_str_field(&mut s, "mode", r.mode, false); + push_u64_field(&mut s, "started_at", r.started_at, false); + push_u64_field(&mut s, "ended_at", r.ended_at, false); + push_u32_field(&mut s, "iterations", r.iterations, false); + push_str_field(&mut s, "outcome", r.outcome, false); + push_f64_field(&mut s, "total_wall_secs", r.total_wall_secs, false); + push_f64_field(&mut s, "total_agent_secs", r.total_agent_secs, false); + push_f64_field(&mut s, "total_thinking_secs", r.total_thinking_secs, false); + push_f64_field(&mut s, "total_guards_secs", r.total_guards_secs, false); + push_f64_field(&mut s, "total_cost_usd", r.total_cost_usd, false); + s.push('}'); + s +} + +fn push_str_field(s: &mut String, k: &str, v: &str, first: bool) { + if !first { + s.push(','); + } + s.push('"'); + s.push_str(k); + s.push_str("\":\""); + escape_str_into(s, v); + s.push('"'); +} + +fn push_u32_field(s: &mut String, k: &str, v: u32, first: bool) { + if !first { + s.push(','); + } + s.push('"'); + s.push_str(k); + s.push_str("\":"); + s.push_str(&v.to_string()); +} + +fn push_u64_field(s: &mut String, k: &str, v: u64, first: bool) { + if !first { + s.push(','); + } + s.push('"'); + s.push_str(k); + s.push_str("\":"); + s.push_str(&v.to_string()); +} + +fn push_f64_field(s: &mut String, k: &str, v: f64, first: bool) { + if !first { + s.push(','); + } + s.push('"'); + s.push_str(k); + s.push_str("\":"); + push_f64_value(s, v); +} + +fn push_opt_f64_field(s: &mut String, k: &str, v: Option) { + s.push(','); + s.push('"'); + s.push_str(k); + s.push_str("\":"); + match v { + Some(x) => push_f64_value(s, x), + None => s.push_str("null"), + } +} + +fn push_f64_value(s: &mut String, v: f64) { + if v.is_finite() { + // 4 dp is more than enough for second-scale measurements + s.push_str(&format!("{:.4}", v)); + } else { + s.push_str("null"); + } +} + +fn push_bool_field(s: &mut String, k: &str, v: bool, first: bool) { + if !first { + s.push(','); + } + s.push('"'); + s.push_str(k); + s.push_str("\":"); + s.push_str(if v { "true" } else { "false" }); +} + +fn escape_str_into(s: &mut String, v: &str) { + for c in v.chars() { + match c { + '"' => s.push_str("\\\""), + '\\' => s.push_str("\\\\"), + '\n' => s.push_str("\\n"), + '\r' => s.push_str("\\r"), + '\t' => s.push_str("\\t"), + c if (c as u32) < 0x20 => s.push_str(&format!("\\u{:04x}", c as u32)), + c => s.push(c), + } + } +} + +fn push_map_u32(s: &mut String, m: &BTreeMap) { + s.push('{'); + for (i, (k, v)) in m.iter().enumerate() { + if i > 0 { + s.push(','); + } + s.push('"'); + escape_str_into(s, k); + s.push_str("\":"); + s.push_str(&v.to_string()); + } + s.push('}'); +} + +fn push_map_f64(s: &mut String, m: &BTreeMap) { + s.push('{'); + for (i, (k, v)) in m.iter().enumerate() { + if i > 0 { + s.push(','); + } + s.push('"'); + escape_str_into(s, k); + s.push_str("\":"); + push_f64_value(s, *v); + } + s.push('}'); +} + +// ─── Read side: parse rows for `yoke stats` ────────────────────────────── +// +// The write side uses `&'static str` for mode/outcome — small win on the +// hot path. The read side comes from runtime data so it uses owned +// `String` fields and a separate row type. Keeps both sides honest +// without forcing one to bend to the other. + +/// One parsed row from `runs.ndjson`. Some fields aren't shown in the +/// current `yoke stats` output but are populated for callers that want +/// to filter or aggregate. +#[allow(dead_code)] +#[derive(Debug, Clone)] +pub struct RunRow { + pub run_id: String, + pub project_slug: String, + pub mode: String, + pub started_at: u64, + pub ended_at: u64, + pub iterations: u32, + pub outcome: String, + pub total_wall_secs: f64, + pub total_agent_secs: f64, + pub total_thinking_secs: f64, + pub total_guards_secs: f64, + pub total_cost_usd: f64, +} + +/// One parsed row from `.ndjson`. +#[allow(dead_code)] +#[derive(Debug, Clone)] +pub struct IterRow { + pub iteration: u32, + pub mode: String, + pub started_at: u64, + pub wall_secs: f64, + pub agent_secs: f64, + pub thinking_secs: f64, + pub guards_secs: f64, + pub num_turns: u32, + pub cost_usd: f64, + pub guards_passed: bool, + pub status_done: bool, +} + +pub fn read_runs(slug_dir: &Path) -> Vec { + let path = slug_dir.join("runs.ndjson"); + let content = match fs::read_to_string(&path) { + Ok(c) => c, + Err(_) => return Vec::new(), + }; + let mut out = Vec::new(); + for line in content.lines() { + if line.trim().is_empty() { + continue; + } + if let Some(r) = parse_run_row(line) { + out.push(r); + } + } + out +} + +pub fn read_iterations(file: &Path) -> Vec { + let content = match fs::read_to_string(file) { + Ok(c) => c, + Err(_) => return Vec::new(), + }; + let mut out = Vec::new(); + for line in content.lines() { + if line.trim().is_empty() { + continue; + } + if let Some(r) = parse_iter_row(line) { + out.push(r); + } + } + out +} + +fn parse_run_row(line: &str) -> Option { + use crate::json::{extract_num, extract_str}; + Some(RunRow { + run_id: extract_str(line, "run_id")?.to_string(), + project_slug: extract_str(line, "project_slug")?.to_string(), + mode: extract_str(line, "mode")?.to_string(), + started_at: extract_num(line, "started_at")? as u64, + ended_at: extract_num(line, "ended_at")? as u64, + iterations: extract_num(line, "iterations")? as u32, + outcome: extract_str(line, "outcome")?.to_string(), + total_wall_secs: extract_num(line, "total_wall_secs").unwrap_or(0.0), + total_agent_secs: extract_num(line, "total_agent_secs").unwrap_or(0.0), + total_thinking_secs: extract_num(line, "total_thinking_secs").unwrap_or(0.0), + total_guards_secs: extract_num(line, "total_guards_secs").unwrap_or(0.0), + total_cost_usd: extract_num(line, "total_cost_usd").unwrap_or(0.0), + }) +} + +fn parse_iter_row(line: &str) -> Option { + use crate::json::{extract_bool, extract_num, extract_str}; + Some(IterRow { + iteration: extract_num(line, "iteration")? as u32, + mode: extract_str(line, "mode").unwrap_or("").to_string(), + started_at: extract_num(line, "started_at").unwrap_or(0.0) as u64, + wall_secs: extract_num(line, "wall_secs").unwrap_or(0.0), + agent_secs: extract_num(line, "agent_secs").unwrap_or(0.0), + thinking_secs: extract_num(line, "thinking_secs").unwrap_or(0.0), + guards_secs: extract_num(line, "guards_secs").unwrap_or(0.0), + num_turns: extract_num(line, "num_turns").unwrap_or(0.0) as u32, + cost_usd: extract_num(line, "cost_usd").unwrap_or(0.0), + guards_passed: extract_bool(line, "guards_passed").unwrap_or(false), + status_done: extract_bool(line, "status_done").unwrap_or(false), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn slug_is_stable_across_calls() { + let a = project_slug(); + let b = project_slug(); + assert_eq!(a, b); + } + + #[test] + fn run_id_is_sortable_and_unique() { + let a = new_run_id(); + std::thread::sleep(std::time::Duration::from_millis(2)); + let b = new_run_id(); + assert_ne!(a, b); + assert!(b >= a, "run ids should be lexicographically sortable"); + } + + #[test] + fn tilde_expands() { + if let Some(home) = std::env::var_os("HOME") { + let p = expand_tilde("~/foo"); + assert_eq!(p, PathBuf::from(home).join("foo")); + } + } + + #[test] + fn unix_to_utc_known_dates() { + // 2021-01-01 00:00:00 UTC = 1609459200 + assert_eq!(unix_to_utc(1609459200), (2021, 1, 1, 0, 0, 0)); + // 1970-01-01 00:00:00 UTC + assert_eq!(unix_to_utc(0), (1970, 1, 1, 0, 0, 0)); + // 2024-02-29 12:34:56 UTC (leap day) = 1709210096 + assert_eq!(unix_to_utc(1709210096), (2024, 2, 29, 12, 34, 56)); + } + + #[test] + fn run_row_round_trips_through_ndjson() { + let original = RunMetrics { + run_id: "20260518T000000-000001".to_string(), + project_slug: "demo-12345678".to_string(), + mode: "brute", + started_at: 1700000000, + ended_at: 1700000050, + iterations: 3, + outcome: "judge_pass", + total_wall_secs: 50.0, + total_agent_secs: 42.5, + total_thinking_secs: 4.1, + total_guards_secs: 3.2, + total_cost_usd: 0.84, + }; + let serialized = serialize_run(&original); + let parsed = parse_run_row(&serialized).expect("parse should succeed"); + assert_eq!(parsed.run_id, original.run_id); + assert_eq!(parsed.mode, original.mode); + assert_eq!(parsed.outcome, original.outcome); + assert_eq!(parsed.iterations, original.iterations); + assert!((parsed.total_wall_secs - original.total_wall_secs).abs() < 1e-3); + assert!((parsed.total_cost_usd - original.total_cost_usd).abs() < 1e-3); + } + + #[test] + fn iter_row_parses_back() { + let m = IterationMetrics { + run_id: "20260518T000000-000001".to_string(), + project_slug: "demo-12345678".to_string(), + mode: "loop", + iteration: 7, + started_at: 1700000000, + wall_secs: 12.5, + restore_ms: 3, + agent_secs: 10.0, + guards_secs: 2.0, + periodics_secs: 0.0, + hooks_secs: 0.5, + judge_secs: None, + agent_reported_secs: Some(9.5), + cost_usd: 0.13, + num_turns: 5, + thinking_secs: 1.2, + tool_counts: BTreeMap::new(), + tool_durations_secs: BTreeMap::new(), + guards: Vec::new(), + guards_passed: true, + status_done: false, + }; + let line = serialize_iteration(&m); + let row = parse_iter_row(&line).expect("parse iter row"); + assert_eq!(row.iteration, 7); + assert_eq!(row.mode, "loop"); + assert_eq!(row.num_turns, 5); + assert!(row.guards_passed); + assert!(!row.status_done); + assert!((row.thinking_secs - 1.2).abs() < 1e-3); + } + + #[test] + fn serialize_iteration_is_valid_json_shape() { + let m = IterationMetrics { + run_id: "20260101T000000-abcd".to_string(), + project_slug: "yoke-deadbeef".to_string(), + mode: "loop", + iteration: 3, + started_at: 1700000000, + wall_secs: 42.5, + restore_ms: 17, + agent_secs: 38.0, + guards_secs: 3.2, + periodics_secs: 0.0, + hooks_secs: 0.1, + judge_secs: None, + agent_reported_secs: Some(36.7), + cost_usd: 0.42, + num_turns: 7, + thinking_secs: 4.1, + tool_counts: [("Edit".to_string(), 3u32), ("Bash".to_string(), 1)].into_iter().collect(), + tool_durations_secs: [("Edit".to_string(), 1.8), ("Bash".to_string(), 12.3)].into_iter().collect(), + guards: vec![GuardRow { + name: "cargo test".to_string(), + passed: true, + skipped: false, + elapsed_secs: 2.5, + }], + guards_passed: true, + status_done: false, + }; + let s = serialize_iteration(&m); + assert!(s.starts_with('{') && s.ends_with('}')); + assert!(s.contains("\"run_id\":\"20260101T000000-abcd\"")); + assert!(s.contains("\"thinking_secs\":4.1000")); + assert!(s.contains("\"judge_secs\":null")); + assert!(s.contains("\"tool_durations_secs\":{")); + } +} diff --git a/src/session_trim.rs b/src/session_trim.rs new file mode 100644 index 0000000..b381fe1 --- /dev/null +++ b/src/session_trim.rs @@ -0,0 +1,620 @@ +//! Session-JSONL trimming for Claude Code prompt-cache reuse across rounds. +//! +//! Yoke uses `claude --resume ` to carry a worker's conversation across +//! iterations so Anthropic's prefix cache stays warm. Naive resume grows the +//! session monotonically — bash outputs, thinking blocks, intermediate Reads +//! that are no longer relevant — all of it stays in the prefix and is paid +//! for at cache-read rates every round. +//! +//! This module trims the session file between rounds. The agent declares a +//! `KEEP: ...` line in `.loop/notes.md` listing the file Reads +//! whose results should stay in conversation history. Everything else is +//! dropped, the parent-uuid chain is re-linked across the gaps, and the file +//! is atomically rewritten in place. +//! +//! Safety: if the trim's own validation fails (broken parent chain, orphaned +//! tool_use without tool_result, parse error), the original session is kept +//! untouched and we log a warning. `YOKE_DISABLE_SESSION_TRIM=1` skips the +//! whole pass. + +use std::collections::{HashMap, HashSet}; +use std::fs; +use std::io::Write; +use std::path::{Path, PathBuf}; + +use serde_json::Value; + +#[derive(Debug, Default)] +pub struct TrimStats { + pub records_total: usize, + pub records_kept: usize, + pub records_dropped: usize, + pub keep_paths: Vec, + pub skipped: bool, +} + +/// Parse a `KEEP:` line from notes.md (or any text). Returns absolute paths +/// resolved against `cwd`. Multiple `KEEP:` lines are unioned. Missing or +/// `*` token is treated as "keep nothing" — the caller decides what that +/// means, but this fn just returns the explicit paths. +pub fn parse_keep_list(notes_text: &str, cwd: &Path) -> HashSet { + let mut out = HashSet::new(); + for line in notes_text.lines() { + let trimmed = line.trim_start(); + let rest = match trimmed.strip_prefix("KEEP:") { + Some(r) => r, + None => continue, + }; + for tok in rest.split_whitespace() { + if tok == "*" { + continue; + } + let p = PathBuf::from(tok); + let abs = if p.is_absolute() { p } else { cwd.join(p) }; + // Best-effort canonicalize so symlinks / .. don't cause mismatches. + let final_path = fs::canonicalize(&abs).unwrap_or(abs); + out.insert(final_path); + } + } + out +} + +/// Locate Claude Code's session file on disk. +/// Format: `/.claude/projects//.jsonl`. +pub fn session_file_path(home: &Path, cwd: &Path, session_id: &str) -> PathBuf { + let cwd_str = cwd.to_string_lossy(); + // Claude Code's slug: replace `/` with `-`. A leading slash becomes a + // leading dash. `.` characters in path components are preserved. + let slug = cwd_str.replace('/', "-"); + home.join(".claude") + .join("projects") + .join(slug) + .join(format!("{}.jsonl", session_id)) +} + +/// Trim a session file in place. Returns stats. If the trim aborts safely +/// (escape hatch / no kept content / validation failure), the original file +/// is untouched. +pub fn trim_session(session_path: &Path, keep_paths: &HashSet) -> Result { + let mut stats = TrimStats { + keep_paths: keep_paths + .iter() + .map(|p| p.to_string_lossy().to_string()) + .collect(), + ..TrimStats::default() + }; + + if std::env::var_os("YOKE_DISABLE_SESSION_TRIM").is_some() { + stats.skipped = true; + return Ok(stats); + } + if !session_path.exists() { + return Err(format!("session file not found: {}", session_path.display())); + } + + let content = fs::read_to_string(session_path) + .map_err(|e| format!("read {}: {}", session_path.display(), e))?; + let raw_lines: Vec<&str> = content.lines().filter(|l| !l.trim().is_empty()).collect(); + stats.records_total = raw_lines.len(); + + let mut records: Vec = Vec::with_capacity(raw_lines.len()); + for (i, line) in raw_lines.iter().enumerate() { + let v: Value = serde_json::from_str(line) + .map_err(|e| format!("parse line {}: {}", i + 1, e))?; + records.push(v); + } + + let decisions = classify(&records, keep_paths); + + let trimmed = relink_and_emit(&records, &raw_lines, &decisions)?; + stats.records_kept = trimmed.lines().filter(|l| !l.trim().is_empty()).count(); + stats.records_dropped = stats.records_total.saturating_sub(stats.records_kept); + + validate(&trimmed)?; + + // Atomic write: tmp → rename. Keep one .bak for recovery / debugging. + let bak_path = session_path.with_extension("jsonl.bak"); + let _ = fs::copy(session_path, &bak_path); + let tmp_path = session_path.with_extension("jsonl.tmp"); + { + let mut f = fs::File::create(&tmp_path) + .map_err(|e| format!("create tmp: {}", e))?; + f.write_all(trimmed.as_bytes()) + .map_err(|e| format!("write tmp: {}", e))?; + f.sync_all().ok(); + } + fs::rename(&tmp_path, session_path) + .map_err(|e| format!("rename: {}", e))?; + + Ok(stats) +} + +/// Per-record decision: keep as-is, drop entirely, or keep with a rewritten +/// parentUuid. +#[derive(Debug, Clone)] +enum Decision { + Keep, + Drop, +} + +fn classify(records: &[Value], keep_paths: &HashSet) -> Vec { + // Two-pass: first identify which tool_use ids we keep, then decide each record. + let mut kept_tool_use_ids: HashSet = HashSet::new(); + for r in records { + if !is_assistant(r) { + continue; + } + let Some(block) = first_content_block(r) else { continue }; + if block.get("type").and_then(|v| v.as_str()) != Some("tool_use") { + continue; + } + let name = block.get("name").and_then(|v| v.as_str()).unwrap_or(""); + if name != "Read" { + continue; + } + let path_str = block + .get("input") + .and_then(|v| v.get("file_path")) + .and_then(|v| v.as_str()); + let Some(path_str) = path_str else { continue }; + let p = PathBuf::from(path_str); + let canonical = fs::canonicalize(&p).unwrap_or(p); + if keep_paths.contains(&canonical) + && let Some(id) = block.get("id").and_then(|v| v.as_str()) + { + kept_tool_use_ids.insert(id.to_string()); + } + } + + records + .iter() + .map(|r| decide(r, &kept_tool_use_ids)) + .collect() +} + +fn decide(record: &Value, kept_tool_use_ids: &HashSet) -> Decision { + // Metadata records (no top-level uuid OR parentUuid is absent and type is bookkeeping): + // always keep. + let typ = record.get("type").and_then(|v| v.as_str()).unwrap_or(""); + match typ { + // Non-conversation bookkeeping — keep unchanged. + "permission-mode" + | "file-history-snapshot" + | "queue-operation" + | "ai-title" + | "last-prompt" + | "attachment" => return Decision::Keep, + _ => {} + } + + // user / assistant: examine content. + let Some(msg) = record.get("message") else { + return Decision::Keep; + }; + let role = msg.get("role").and_then(|v| v.as_str()).unwrap_or(""); + + // Initial user prompt: content is a plain string, not an array. Always keep. + if role == "user" { + match msg.get("content") { + Some(Value::String(_)) => return Decision::Keep, + Some(Value::Array(arr)) => { + // tool_result wrapper. Keep only if its tool_use_id was kept. + if arr.is_empty() { + return Decision::Keep; + } + let block = &arr[0]; + let btyp = block.get("type").and_then(|v| v.as_str()).unwrap_or(""); + if btyp == "tool_result" { + let id = block.get("tool_use_id").and_then(|v| v.as_str()).unwrap_or(""); + if kept_tool_use_ids.contains(id) { + return Decision::Keep; + } else { + return Decision::Drop; + } + } + // Other user content (unusual): keep defensively. + return Decision::Keep; + } + _ => return Decision::Keep, + } + } + + if role == "assistant" { + let Some(block) = first_content_block(record) else { + return Decision::Drop; + }; + let btyp = block.get("type").and_then(|v| v.as_str()).unwrap_or(""); + match btyp { + "thinking" => Decision::Drop, + "text" => Decision::Drop, + "tool_use" => { + let id = block.get("id").and_then(|v| v.as_str()).unwrap_or(""); + if kept_tool_use_ids.contains(id) { + Decision::Keep + } else { + Decision::Drop + } + } + _ => Decision::Drop, + } + } else { + // Unknown role — keep, don't make things worse. + Decision::Keep + } +} + +fn is_assistant(record: &Value) -> bool { + record.get("type").and_then(|v| v.as_str()) == Some("assistant") +} + +fn first_content_block(record: &Value) -> Option<&Value> { + record + .get("message")? + .get("content")? + .as_array()? + .first() +} + +/// Build the trimmed JSONL output. For surviving records whose parentUuid +/// points to a dropped record, walks up the parent chain to find the nearest +/// surviving ancestor and rewrites the field. +fn relink_and_emit(records: &[Value], raw: &[&str], decisions: &[Decision]) -> Result { + // uuid → parentUuid index for ALL records that have a uuid. Used to walk + // up the chain when re-linking. + let mut parent_of: HashMap> = HashMap::new(); + let mut kept_uuids: HashSet = HashSet::new(); + for (i, r) in records.iter().enumerate() { + let Some(uuid) = r.get("uuid").and_then(|v| v.as_str()) else { continue }; + let parent = r + .get("parentUuid") + .and_then(|v| v.as_str()) + .map(|s| s.to_string()); + parent_of.insert(uuid.to_string(), parent); + if matches!(decisions[i], Decision::Keep) { + kept_uuids.insert(uuid.to_string()); + } + } + + // For each kept uuid, compute the rewritten parentUuid (nearest kept + // ancestor or null). + let mut rewritten_parent: HashMap> = HashMap::new(); + for uuid in &kept_uuids { + let mut cur = parent_of.get(uuid).cloned().flatten(); + while let Some(p) = cur { + if kept_uuids.contains(&p) { + rewritten_parent.insert(uuid.clone(), Some(p)); + break; + } + cur = parent_of.get(&p).cloned().flatten(); + } + if !rewritten_parent.contains_key(uuid) { + rewritten_parent.insert(uuid.clone(), None); + } + } + + let mut out = String::with_capacity(raw.iter().map(|l| l.len() + 1).sum()); + for (i, r) in records.iter().enumerate() { + if matches!(decisions[i], Decision::Drop) { + continue; + } + let uuid_opt = r.get("uuid").and_then(|v| v.as_str()).map(|s| s.to_string()); + // If this record has a uuid AND its rewritten parent differs from + // the on-disk parent, re-serialize. Otherwise emit raw. + let needs_rewrite = match &uuid_opt { + Some(uuid) => { + let orig = parent_of.get(uuid).cloned().flatten(); + let new = rewritten_parent.get(uuid).cloned().flatten(); + orig != new + } + None => false, + }; + if needs_rewrite { + let mut v = r.clone(); + let uuid = uuid_opt.unwrap(); + let new_parent = rewritten_parent.get(&uuid).cloned().flatten(); + if let Some(obj) = v.as_object_mut() { + match new_parent { + Some(p) => { + obj.insert("parentUuid".to_string(), Value::String(p)); + } + None => { + obj.insert("parentUuid".to_string(), Value::Null); + } + } + } + let s = serde_json::to_string(&v) + .map_err(|e| format!("serialize: {}", e))?; + out.push_str(&s); + out.push('\n'); + } else { + out.push_str(raw[i]); + out.push('\n'); + } + } + + Ok(out) +} + +/// Validate that the trimmed JSONL is internally consistent: +/// every tool_use has a matching tool_result downstream. +fn validate(trimmed: &str) -> Result<(), String> { + let mut tool_use_ids: HashSet = HashSet::new(); + let mut tool_result_ids: HashSet = HashSet::new(); + for (i, line) in trimmed.lines().enumerate() { + if line.trim().is_empty() { + continue; + } + let v: Value = serde_json::from_str(line) + .map_err(|e| format!("validate parse line {}: {}", i + 1, e))?; + let Some(arr) = v.get("message").and_then(|m| m.get("content")).and_then(|c| c.as_array()) else { + continue; + }; + for block in arr { + match block.get("type").and_then(|v| v.as_str()) { + Some("tool_use") => { + if let Some(id) = block.get("id").and_then(|v| v.as_str()) { + tool_use_ids.insert(id.to_string()); + } + } + Some("tool_result") => { + if let Some(id) = block.get("tool_use_id").and_then(|v| v.as_str()) { + tool_result_ids.insert(id.to_string()); + } + } + _ => {} + } + } + } + for id in &tool_use_ids { + if !tool_result_ids.contains(id) { + return Err(format!("orphan tool_use {}: no matching tool_result", id)); + } + } + for id in &tool_result_ids { + if !tool_use_ids.contains(id) { + return Err(format!("orphan tool_result {}: no matching tool_use", id)); + } + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use std::io::Write; + use std::sync::Mutex; + + /// Serializes tests that read/mutate the `YOKE_DISABLE_SESSION_TRIM` env + /// var. cargo runs tests in parallel within a binary; without this they + /// race on a process-global. + static ENV_LOCK: Mutex<()> = Mutex::new(()); + + /// Clear the env var before running the closure, then drop the guard. + fn with_clean_env(f: F) { + let _g = ENV_LOCK.lock().unwrap(); + // SAFETY: tests are serialized via ENV_LOCK; no other thread will + // observe a partial write to environ. + unsafe { std::env::remove_var("YOKE_DISABLE_SESSION_TRIM") }; + f(); + } + + #[test] + fn parse_keep_list_basic() { + let cwd = PathBuf::from("/tmp"); + let s = "STATUS: IN_PROGRESS\nKEEP: src/a.rs /abs/b.rs\nfoo\n"; + let got = parse_keep_list(s, &cwd); + assert!(got.iter().any(|p| p.ends_with("a.rs"))); + assert!(got.iter().any(|p| p == &PathBuf::from("/abs/b.rs"))); + } + + #[test] + fn parse_keep_list_missing() { + let cwd = PathBuf::from("/tmp"); + let s = "STATUS: DONE\n"; + assert!(parse_keep_list(s, &cwd).is_empty()); + } + + #[test] + fn parse_keep_list_star_means_nothing() { + let cwd = PathBuf::from("/tmp"); + let s = "KEEP: *\n"; + assert!(parse_keep_list(s, &cwd).is_empty()); + } + + #[test] + fn session_file_path_slug() { + let p = session_file_path( + Path::new("/home/u"), + Path::new("/workspace"), + "abc-123", + ); + assert_eq!(p, PathBuf::from("/home/u/.claude/projects/-workspace/abc-123.jsonl")); + } + + /// Tiny realistic session: bootstrap + initial prompt + 2 Reads + a Bash + /// + a thinking block. Used by scenario tests below. + fn write_fixture(dir: &Path, paths: &[&str]) -> PathBuf { + let mut lines: Vec = Vec::new(); + // queue-operation (metadata) — has no uuid/parentUuid. + lines.push(r#"{"type":"queue-operation","sessionId":"s1"}"#.to_string()); + // initial user prompt (root of conversation) + lines.push(r#"{"type":"user","uuid":"u-root","parentUuid":null,"message":{"role":"user","content":"Read .loop/protocol.md and follow its instructions."}}"#.to_string()); + // assistant thinking — should always be dropped + lines.push(r#"{"type":"assistant","uuid":"u-think","parentUuid":"u-root","message":{"role":"assistant","content":[{"type":"thinking","thinking":"..."}]}}"#.to_string()); + // Read tool_use for paths[0] + let p0 = paths.first().copied().unwrap_or("/tmp/a.rs"); + lines.push(format!( + r#"{{"type":"assistant","uuid":"u-read-a","parentUuid":"u-think","message":{{"role":"assistant","content":[{{"type":"tool_use","id":"tu-a","name":"Read","input":{{"file_path":"{}"}}}}]}}}}"#, + p0 + )); + // tool_result for the Read + lines.push(r#"{"type":"user","uuid":"u-res-a","parentUuid":"u-read-a","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"tu-a","content":"file A contents"}]}}"#.to_string()); + // Bash tool_use — should always be dropped + lines.push(r#"{"type":"assistant","uuid":"u-bash","parentUuid":"u-res-a","message":{"role":"assistant","content":[{"type":"tool_use","id":"tu-bash","name":"Bash","input":{"command":"ls"}}]}}"#.to_string()); + lines.push(r#"{"type":"user","uuid":"u-res-bash","parentUuid":"u-bash","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"tu-bash","content":"a\nb\nc"}]}}"#.to_string()); + // Read tool_use for paths[1] (a second file) + let p1 = paths.get(1).copied().unwrap_or("/tmp/b.rs"); + lines.push(format!( + r#"{{"type":"assistant","uuid":"u-read-b","parentUuid":"u-res-bash","message":{{"role":"assistant","content":[{{"type":"tool_use","id":"tu-b","name":"Read","input":{{"file_path":"{}"}}}}]}}}}"#, + p1 + )); + lines.push(r#"{"type":"user","uuid":"u-res-b","parentUuid":"u-read-b","message":{"role":"user","content":[{"type":"tool_result","tool_use_id":"tu-b","content":"file B contents"}]}}"#.to_string()); + + let path = dir.join("s1.jsonl"); + let mut f = fs::File::create(&path).unwrap(); + for l in &lines { + writeln!(f, "{}", l).unwrap(); + } + path + } + + fn read_records(path: &Path) -> Vec { + fs::read_to_string(path) + .unwrap() + .lines() + .filter(|l| !l.trim().is_empty()) + .map(|l| serde_json::from_str(l).unwrap()) + .collect() + } + + #[test] + fn trim_with_one_keep_drops_other_reads_and_bash_and_thinking() { + let tmp = tempfile::tempdir().unwrap(); + let a_path = tmp.path().join("a.rs"); + let b_path = tmp.path().join("b.rs"); + fs::write(&a_path, "fn a(){}").unwrap(); + fs::write(&b_path, "fn b(){}").unwrap(); + let session = write_fixture( + tmp.path(), + &[a_path.to_str().unwrap(), b_path.to_str().unwrap()], + ); + + let mut keep = HashSet::new(); + keep.insert(fs::canonicalize(&a_path).unwrap()); + + let mut stats_opt = None; + with_clean_env(|| { + stats_opt = Some(trim_session(&session, &keep).expect("trim ok")); + }); + let stats = stats_opt.unwrap(); + assert!(!stats.skipped); + assert!(stats.records_dropped >= 4, "should drop thinking + bash pair + b read pair, got {:?}", stats); + + let recs = read_records(&session); + // Surviving tool_use ids: only tu-a; tu-b and tu-bash are gone. + let tool_use_ids: Vec = recs + .iter() + .filter_map(|r| { + r.get("message") + .and_then(|m| m.get("content")) + .and_then(|c| c.as_array()) + .and_then(|a| a.first()) + .filter(|b| b.get("type").and_then(|v| v.as_str()) == Some("tool_use")) + .and_then(|b| b.get("id").and_then(|v| v.as_str()).map(|s| s.to_string())) + }) + .collect(); + assert_eq!(tool_use_ids, vec!["tu-a".to_string()]); + + // Every tool_use has a paired tool_result (validate() enforces this on write; + // re-check here for the behavior we promise). + let tool_result_ids: Vec = recs + .iter() + .filter_map(|r| { + r.get("message") + .and_then(|m| m.get("content")) + .and_then(|c| c.as_array()) + .and_then(|a| a.first()) + .filter(|b| b.get("type").and_then(|v| v.as_str()) == Some("tool_result")) + .and_then(|b| b.get("tool_use_id").and_then(|v| v.as_str()).map(|s| s.to_string())) + }) + .collect(); + assert_eq!(tool_result_ids, vec!["tu-a".to_string()]); + + // No thinking blocks survive. + for r in &recs { + let typ_opt = r + .get("message") + .and_then(|m| m.get("content")) + .and_then(|c| c.as_array()) + .and_then(|a| a.first()) + .and_then(|b| b.get("type")) + .and_then(|v| v.as_str()); + assert_ne!(typ_opt, Some("thinking")); + } + } + + #[test] + fn trim_relinks_parent_uuids_to_nearest_surviving_ancestor() { + let tmp = tempfile::tempdir().unwrap(); + let a_path = tmp.path().join("a.rs"); + let b_path = tmp.path().join("b.rs"); + fs::write(&a_path, "x").unwrap(); + fs::write(&b_path, "y").unwrap(); + let session = write_fixture( + tmp.path(), + &[a_path.to_str().unwrap(), b_path.to_str().unwrap()], + ); + + // Keep only b. Records dropped between root and b-read should result + // in u-read-b's new parent being u-root (the only surviving ancestor). + let mut keep = HashSet::new(); + keep.insert(fs::canonicalize(&b_path).unwrap()); + + with_clean_env(|| { + trim_session(&session, &keep).expect("trim ok"); + }); + + let recs = read_records(&session); + let read_b = recs + .iter() + .find(|r| r.get("uuid").and_then(|v| v.as_str()) == Some("u-read-b")) + .expect("u-read-b survives"); + let new_parent = read_b.get("parentUuid").and_then(|v| v.as_str()); + // u-think, u-read-a, u-res-a, u-bash, u-res-bash all dropped → parent + // walks up to u-root. + assert_eq!(new_parent, Some("u-root")); + } + + #[test] + fn trim_skipped_when_env_var_set() { + let tmp = tempfile::tempdir().unwrap(); + let session = write_fixture(tmp.path(), &["/tmp/a.rs"]); + let original = fs::read_to_string(&session).unwrap(); + + let _g = ENV_LOCK.lock().unwrap(); + // SAFETY: serialized by ENV_LOCK above. + unsafe { std::env::set_var("YOKE_DISABLE_SESSION_TRIM", "1") }; + let stats = trim_session(&session, &HashSet::new()).expect("trim ok"); + unsafe { std::env::remove_var("YOKE_DISABLE_SESSION_TRIM") }; + assert!(stats.skipped); + assert_eq!(original, fs::read_to_string(&session).unwrap()); + } + + #[test] + fn trim_empty_keep_drops_all_tool_use_pairs() { + let tmp = tempfile::tempdir().unwrap(); + let session = write_fixture(tmp.path(), &["/tmp/a.rs", "/tmp/b.rs"]); + + with_clean_env(|| { + trim_session(&session, &HashSet::new()).expect("trim ok"); + }); + + let recs = read_records(&session); + for r in &recs { + let block = r + .get("message") + .and_then(|m| m.get("content")) + .and_then(|c| c.as_array()) + .and_then(|a| a.first()); + if let Some(b) = block { + let btyp = b.get("type").and_then(|v| v.as_str()).unwrap_or(""); + assert_ne!(btyp, "tool_use", "no tool_use should survive empty keep"); + assert_ne!(btyp, "tool_result", "no tool_result should survive empty keep"); + assert_ne!(btyp, "thinking", "no thinking should survive"); + } + } + // queue-operation, initial user prompt should survive. + assert!(recs.iter().any(|r| r.get("type").and_then(|v| v.as_str()) == Some("queue-operation"))); + assert!(recs.iter().any(|r| r.get("uuid").and_then(|v| v.as_str()) == Some("u-root"))); + } +} diff --git a/src/stash.rs b/src/stash.rs index 5c18268..eaf3ac5 100644 --- a/src/stash.rs +++ b/src/stash.rs @@ -180,6 +180,9 @@ pub(crate) fn stash_create(mode: &str) -> i32 { } match stash_snapshot(mode) { Ok(hash) => { + for (name, _) in &collect_stashable_files() { + let _ = fs::remove_file(Path::new(".loop").join(name)); + } log(&format!("stashed → {}{}{}", BLUE, hash, RESET)); 0 } @@ -321,7 +324,7 @@ pub(crate) fn print_stash_help() { eprintln!(); eprintln!("{}SUBCOMMANDS:{}", BOLD, RESET); eprintln!( - " {}(none){} Snapshot all .loop/ files to a new stash entry", + " {}(none){} Snapshot .loop/ files to stash, then clear the directory", BOLD, RESET ); eprintln!( diff --git a/src/stream.rs b/src/stream.rs index 9a59ae8..ecbcb17 100644 --- a/src/stream.rs +++ b/src/stream.rs @@ -1,4 +1,4 @@ -use std::collections::HashMap; +use std::collections::{BTreeMap, HashMap}; use std::io::{self, BufRead, BufReader, Write}; use std::path::Path; use std::process::ChildStdout; @@ -10,18 +10,44 @@ use crate::json::{extract_num, extract_str, unescape_json}; const BG_RED: &str = "\x1b[48;2;80;30;30m"; const BG_GREEN: &str = "\x1b[48;2;30;60;30m"; +/// Aggregated, persistable view of one agent invocation's stream. +/// +/// Produced by `filter_stream` after the child's stdout closes. Carries +/// everything the metrics layer wants: cost, agent-reported wall time, +/// thinking total, and per-tool durations + counts. +#[derive(Debug, Clone, Default)] +pub struct StreamSummary { + pub cost_usd: f64, + /// Wall clock as reported by the agent's `result` event (Claude only). + pub agent_reported_secs: Option, + pub num_turns: u32, + pub thinking_secs: f64, + pub tool_counts: BTreeMap, + pub tool_durations_secs: BTreeMap, + /// Claude session ID extracted from the init event. Used by the harness + /// to `--resume` the same session on the next iteration, preserving the + /// prompt cache. `None` for OpenCode or if the init event was missed. + pub session_id: Option, +} + struct StreamState { turn_num: u32, current_msg_id: Option, seen_init: bool, in_thinking: bool, thinking_start: Option, + thinking_total_secs: f64, iteration_cost: f64, iteration_duration_secs: f64, - /// Maps tool_use id → tool name, so tool_result can look up its origin. - tool_use_names: HashMap, + /// Maps tool_use id → (tool name, start instant), so tool_result can + /// look up its origin and elapsed time. + tool_use_starts: HashMap, /// Counts of tool_use events by tool name (for iteration summary strip). tool_counts: HashMap, + /// Wall-clock duration accumulated per tool name across the iteration. + tool_durations: HashMap, + /// Captured from the `system`/`init` event so the harness can `--resume`. + session_id: Option, } impl StreamState { @@ -32,10 +58,30 @@ impl StreamState { seen_init: false, in_thinking: false, thinking_start: None, + thinking_total_secs: 0.0, iteration_cost: 0.0, iteration_duration_secs: 0.0, - tool_use_names: HashMap::new(), + tool_use_starts: HashMap::new(), tool_counts: HashMap::new(), + tool_durations: HashMap::new(), + session_id: None, + } + } + + fn into_summary(self) -> StreamSummary { + let agent_reported_secs = if self.iteration_duration_secs > 0.0 { + Some(self.iteration_duration_secs) + } else { + None + }; + StreamSummary { + cost_usd: self.iteration_cost, + agent_reported_secs, + num_turns: self.turn_num, + thinking_secs: self.thinking_total_secs, + tool_counts: self.tool_counts.into_iter().collect(), + tool_durations_secs: self.tool_durations.into_iter().collect(), + session_id: self.session_id, } } } @@ -255,7 +301,7 @@ fn handle_assistant(out: &mut (impl Write + ?Sized), line: &str, state: &mut Str return Ok(()); } if let (Some(id), Some(name)) = (extract_str(line, "id"), extract_str(line, "name")) { - state.tool_use_names.insert(id.to_string(), name.to_string()); + state.tool_use_starts.insert(id.to_string(), (name.to_string(), Instant::now())); *state.tool_counts.entry(name.to_string()).or_insert(0) += 1; } let desc = format_tool_call(line); @@ -291,6 +337,7 @@ fn handle_stream_event(out: &mut (impl Write + ?Sized), line: &str, state: &mut if state.in_thinking { if let Some(start) = state.thinking_start { let elapsed = start.elapsed().as_secs_f64(); + state.thinking_total_secs += elapsed; write!(out, "\r{}{}thinking {:.1}s{}", DIM, BLUE, elapsed, RESET)?; } state.in_thinking = false; @@ -301,11 +348,24 @@ fn handle_stream_event(out: &mut (impl Write + ?Sized), line: &str, state: &mut Ok(()) } -/// Handle "user" events: render tool_result success/error badges. -fn handle_tool_result(out: &mut (impl Write + ?Sized), line: &str, state: &StreamState) -> io::Result<()> { +/// Handle "user" events: render tool_result success/error badges and +/// accumulate the wall-clock duration of each tool call by name. +fn handle_tool_result(out: &mut (impl Write + ?Sized), line: &str, state: &mut StreamState) -> io::Result<()> { if !line.contains("\"tool_result\"") { return Ok(()); } + + // Pair this result with its tool_use; drop the entry and accumulate elapsed. + // Done for both success and error so failed tools still show up in metrics. + let tool_use_id = extract_str(line, "tool_use_id").map(|s| s.to_string()); + let tool_name_owned: Option = tool_use_id.and_then(|id| { + state.tool_use_starts.remove(&id).map(|(name, start)| { + let elapsed = start.elapsed().as_secs_f64(); + *state.tool_durations.entry(name.clone()).or_insert(0.0) += elapsed; + name + }) + }); + let is_error = line.contains("\"is_error\":true") || line.contains("\"is_error\": true"); if is_error { writeln!(out, " {}← {}{}✗{}", GRAY, RESET, RED, RESET)?; @@ -315,10 +375,7 @@ fn handle_tool_result(out: &mut (impl Write + ?Sized), line: &str, state: &Strea } return Ok(()); } - let tool_name = extract_str(line, "tool_use_id") - .and_then(|id| state.tool_use_names.get(id)) - .map(|s| s.as_str()); - let badge = match tool_name { + let badge = match tool_name_owned.as_deref() { Some(name @ ("Grep" | "Glob")) => format_grep_glob_badge(name, line), _ => String::new(), }; @@ -348,6 +405,9 @@ fn process_line(out: &mut (impl Write + ?Sized), line: &str, state: &mut StreamS if extract_str(line, "subtype") == Some("init") && !state.seen_init { state.seen_init = true; let sid = extract_str(line, "session_id").unwrap_or("?"); + if sid != "?" { + state.session_id = Some(sid.to_string()); + } let sid_short: String = sid.chars().take(12).collect(); let model = extract_str(line, "model").unwrap_or("?"); writeln!(out, "{}{}[stream]{} session {}… model={}", ORANGE, BOLD, RESET, sid_short, model)?; @@ -379,26 +439,39 @@ fn process_line(out: &mut (impl Write + ?Sized), line: &str, state: &mut StreamS } /// Build a compact one-line iteration summary strip from accumulated state. -/// Format: `⟪ 6 turns │ 3 edits │ 1 bash │ 42s │ $0.38 ⟫` +/// Format: `⟪ 6 turns │ 3 edit 1.8s │ 1 bash 12.3s │ thinking 4.1s │ 42s │ $0.38 ⟫` +/// +/// Tool entries show count + elapsed for the top-3 tools by wall time, then +/// count-only for the rest (stops the strip overflowing 80 cols). fn format_summary_strip(state: &StreamState) -> String { let mut parts: Vec = Vec::new(); - // Turns parts.push(format!("{} turn{}", state.turn_num, if state.turn_num == 1 { "" } else { "s" })); - // Tool counts — show the most interesting tools in a stable order - let tool_order = ["Edit", "Write", "Read", "Bash", "Grep", "Glob"]; - for tool in &tool_order { - if let Some(&count) = state.tool_counts.get(*tool) { - let label = tool.to_lowercase(); + // Rank tools by elapsed time; top-3 get the "Ns" suffix, the rest are + // count-only. Tools with no recorded duration (e.g. tool_result never + // arrived) sort to the end of the timed list. + let mut ranked: Vec<(&String, u32, f64)> = state + .tool_counts + .iter() + .map(|(name, &count)| { + let secs = state.tool_durations.get(name).copied().unwrap_or(0.0); + (name, count, secs) + }) + .collect(); + ranked.sort_by(|a, b| b.2.partial_cmp(&a.2).unwrap_or(std::cmp::Ordering::Equal)); + + for (i, (name, count, secs)) in ranked.iter().enumerate() { + let label = name.to_lowercase(); + if i < 3 && *secs > 0.0 { + parts.push(format!("{} {} {:.1}s", count, label, secs)); + } else { parts.push(format!("{} {}", count, label)); } } - // Any tools not in the predefined order - for (name, &count) in &state.tool_counts { - if !tool_order.contains(&name.as_str()) { - parts.push(format!("{} {}", count, name.to_lowercase())); - } + + if state.thinking_total_secs > 0.0 { + parts.push(format!("thinking {:.1}s", state.thinking_total_secs)); } // Duration @@ -411,18 +484,19 @@ fn format_summary_strip(state: &StreamState) -> String { } /// Shared stream loop: reads lines from stdout, tees to log, calls processor per line, -/// prints a summary strip, and returns the iteration cost. +/// prints a summary strip, and finalizes state into a caller-defined return type. /// /// Used by both Claude and OpenCode stream filters to avoid duplicating the -/// BufReader/signal-check/log-tee boilerplate. -pub fn run_stream_loop( +/// BufReader/signal-check/log-tee boilerplate. `finalize` consumes the state +/// so the caller can move out of it (e.g. into a `StreamSummary`). +pub fn run_stream_loop( stdout: ChildStdout, log_path: Option<&Path>, - state: &mut S, + mut state: S, mut process: impl FnMut(&mut dyn Write, &str, &mut S) -> io::Result<()>, summarize: impl FnOnce(&S) -> Option, - get_cost: impl FnOnce(&S) -> f64, -) -> f64 { + finalize: impl FnOnce(S) -> R, +) -> R { let reader = BufReader::new(stdout); let mut log_file = log_path.and_then(|p| { std::fs::create_dir_all(p.parent().unwrap_or(Path::new("."))).ok(); @@ -449,30 +523,30 @@ pub fn run_stream_loop( let _ = writeln!(f, "{}", line); } - if process(&mut out, &line, state).is_err() { + if process(&mut out, &line, &mut state).is_err() { break; } } - if let Some(strip) = summarize(state) { + if let Some(strip) = summarize(&state) { let _ = writeln!(out, "{}", strip); } let _ = out.flush(); - get_cost(state) + finalize(state) } /// Filter NDJSON stream from Claude and format as rich ANSI output on stdout. /// `prior_total` is the accumulated cost from previous iterations. -/// Returns the cost of this iteration (from the `result` event). -pub fn filter_stream(stdout: ChildStdout, log_path: Option<&Path>, prior_total: f64) -> f64 { - let mut state = StreamState::new(); +/// Returns a `StreamSummary` carrying cost, thinking time, tool durations, etc. +pub fn filter_stream(stdout: ChildStdout, log_path: Option<&Path>, prior_total: f64) -> StreamSummary { + let state = StreamState::new(); run_stream_loop( stdout, log_path, - &mut state, + state, |out, line, st| process_line(out, line, st, prior_total), |st| if st.turn_num > 0 { Some(format_summary_strip(st)) } else { None }, - |st| st.iteration_cost, + |st| st.into_summary(), ) } diff --git a/src/stream_opencode.rs b/src/stream_opencode.rs index 6866b24..a8d564c 100644 --- a/src/stream_opencode.rs +++ b/src/stream_opencode.rs @@ -5,6 +5,7 @@ use std::process::ChildStdout; use crate::ansi::{BLUE, BOLD, CYAN, DIM, GRAY, GREEN, MAGENTA, ORANGE, RED, RESET, YELLOW}; use crate::json::{extract_num, extract_str, unescape_json}; +use crate::stream::StreamSummary; struct StreamState { turn_num: u32, @@ -166,14 +167,22 @@ fn format_summary_strip(state: &StreamState) -> String { format!("{} ⟪ {} ⟫{}", DIM, parts.join(" │ "), RESET) } -pub fn filter_stream(stdout: ChildStdout, log_path: Option<&Path>, _prior_total: f64) -> f64 { - let mut state = StreamState::new(); +pub fn filter_stream(stdout: ChildStdout, log_path: Option<&Path>, _prior_total: f64) -> StreamSummary { + let state = StreamState::new(); crate::stream::run_stream_loop( stdout, log_path, - &mut state, + state, |out, line, st| process_line(out, line, st), |st| if st.turn_num > 0 { Some(format_summary_strip(st)) } else { None }, - |st| st.iteration_cost, + |st| StreamSummary { + cost_usd: st.iteration_cost, + agent_reported_secs: None, + num_turns: st.turn_num, + thinking_secs: 0.0, + tool_counts: st.tool_counts.into_iter().collect(), + tool_durations_secs: std::collections::BTreeMap::new(), + session_id: None, + }, ) } diff --git a/src/templates/brute/protocol.md b/src/templates/brute/protocol.md index fec039a..e4195b7 100644 --- a/src/templates/brute/protocol.md +++ b/src/templates/brute/protocol.md @@ -43,6 +43,34 @@ The first line of `.loop/notes.md` must be one of: - `STATUS: IN_PROGRESS` — You have more work to do (stages remain, or you expect guard failures). - `STATUS: DONE` — All stages are implemented and you believe guards will pass. +## KEEP: Carrying File Context Across Iterations + +Your conversation history persists across worker iterations via `--resume`. +To keep Claude's prompt cache warm without ballooning, the outer loop trims +your session between rounds: it drops `Bash` output, thinking, and any file +`Read` results that aren't on your KEEP list. Everything else (text turns, +intermediate `Edit`/`Grep`/`Glob` results) is also dropped. + +After STATUS, on its own line in `.loop/notes.md`, list the file paths you +want to keep cached for the next iteration: + +``` +STATUS: IN_PROGRESS +KEEP: src/foo.rs src/bar.rs tests/baz.rs +``` + +Rules: + +- Space-separated repo-relative paths (or absolute). +- List files you read **this iteration** and will still need next iteration. +- Don't list `.loop/*` files — those live on disk and are re-read fresh. +- Keep the list tight. Every kept file is paid for at cache-read rates + every round it stays. Drop a file once it's no longer relevant. +- Omit `KEEP:` (or `KEEP: *`) to keep nothing. + +Note: the judge always runs in a fresh session — your KEEP list does not +affect the judge. + ## What Happens After You Exit 1. Guards run (diff boundary check + configured guard commands). diff --git a/src/templates/brute/yoke.conf b/src/templates/brute/yoke.conf index 1f319b1..b9e16b2 100644 --- a/src/templates/brute/yoke.conf +++ b/src/templates/brute/yoke.conf @@ -1,70 +1,172 @@ -# Yoke configuration (brute mode) -# Lines starting with # are comments. Blank lines are ignored. +# ╔══════════════════════════════════════════════════════════════════════╗ +# ║ Yoke configuration — brute mode ║ +# ╚══════════════════════════════════════════════════════════════════════╝ +# +# Brute mode adds a judge on top of the plan loop. The worker iterates +# until STATUS: DONE + guards pass, then a fresh judge agent (with zero +# worker context) independently verifies the result. +# +# Outer loop (brute): +# 1. Run plan loop (worker iterates until DONE + guards pass) +# 2. Invoke judge — reads judge.md, tests the feature, writes verdict.md +# 3. VERDICT: PASS → exit 0 +# 4. VERDICT: FAIL → reset STATUS to IN_PROGRESS, retry from step 1 +# (verdict.md and guard-results.md are preserved so the worker +# sees what went wrong on its next attempt) +# 5. After max-judge-failures consecutive FAILs → bail out (exit 1) +# +# Inner loop (plan, per iteration): +# 1. Restore protected files (protocol.md, plan.md, yoke.conf) +# 2. Invoke the agent +# 3. Diff boundary check +# 4. Run guards → results to guard-results.md +# 5. Fire periodic agents (if cadence matches) +# 6. Run hooks +# 7. Check: STATUS: DONE + all guards pass → exit inner loop # ── Backend ──────────────────────────────────────────────────────────── -# Model to use for the agent. If unset, defaults to Claude CLI. -# Use provider/model format for OpenRouter or other opencode providers. +# Which LLM backend to use. Leave commented for Claude CLI (default). +# Setting `model` switches to the OpenCode backend, which supports +# OpenRouter, OpenAI, Anthropic API, and other providers. +# # model openrouter/anthropic/claude-sonnet-4 # model openai/gpt-4o # model anthropic/claude-sonnet-4 -# ── Sandbox ────────────────────────────────────────────────────────── -# Docker image to run the agent inside. Required unless you pass --no-sandbox. -# Note: sandbox is not currently supported with the 'model' directive. +# ── Sandbox ──────────────────────────────────────────────────────────── +# Docker image to run the agent inside. Your working directory is +# bind-mounted into the container at /workspace. Required unless you +# pass --no-sandbox on the command line. +# +# Note: sandbox is not currently supported with the `model` directive. + image claude-code-sandbox:latest -# ── Output ─────────────────────────────────────────────────────────── -# Max lines of tail output kept per guard in guard-results.md. +# ── Output ───────────────────────────────────────────────────────────── +# max-tail: max lines of output kept *per guard* in guard-results.md. +# Only affects what the agent reads back — full output still streams to +# your terminal. Default 200 is enough for most test suites; raise it +# if your guards produce essential output beyond 200 lines. + max-tail 200 -# Uncomment to save raw stream-json output per iteration. +# log-dir: save raw stream-json output for each iteration. Useful for +# debugging agent behavior or auditing token usage. Files are named +# /iteration-.jsonl. +# # log-dir .loop/logs +# ── Metrics ──────────────────────────────────────────────────────────── +# Per-iteration timing + cost records are written as NDJSON under +# ~/.yoke/metrics// by default. Survives `yoke clean`. +# +# metrics-dir ~/.yoke/metrics # default +# metrics off # opt out of disk writes + # ── Scope rules (diff boundary enforcement) ────────────────────────── -# Controls what files the agent is allowed to change. Most-specific -# (longest prefix) match wins. +# After each iteration yoke diffs the working tree and checks every +# changed file against these rules. If any file is out of scope, ALL +# guards are skipped and the agent gets only boundary feedback. +# Files under .loop/ are always exempt (yoke's own infrastructure). +# +# Three directives, most-specific (longest prefix) match wins: # # allow — any change permitted (add, modify, delete) -# add-only — new files only; existing files cannot be modified -# no-modify — no changes at all (adds or modifications rejected) +# add-only — new files OK; edits to existing files rejected +# no-modify — no changes at all (adds or edits rejected) +# +# The special prefix "." matches every path (root catch-all). +# +# Examples: +# allow src/ # full access to source +# allow tests/ # full access to tests +# add-only docs/ # can add new docs, not edit existing +# no-modify .github/ # CI config is off-limits +# no-modify package-lock.json # protect a specific file +# allow . # fallback: everything else allowed allow . -# ── Guards (run after each plan stage, fail-fast) ──────────────────── -# Shell commands executed after each agent iteration. If any guard -# exits non-zero the iteration fails and results are fed back. +# ── Guards (post-iteration validation) ──────────────────────────────── +# Shell commands that validate the agent's work. All guards run in +# parallel; results are collected in declared order and written to +# .loop/guard-results.md. The agent reads this file on its next turn, +# so failed guards become automatic feedback. # -# NOTE: avoid "cargo check" as the sole guard — its type-error output -# can confuse the agent into chasing compiler noise instead of finishing -# the task. Prefer a test suite or linter that validates behaviour. +# If the boundary check fails, guards are skipped entirely — the agent +# must fix scope violations before guards will run again. +# +# The inner loop only exits when STATUS: DONE *and* all guards pass. +# +# Examples: +# guard cargo test +# guard npm test +# guard python -m pytest tests/ -x +# guard go test ./... +# guard make check +# guard ./scripts/validate.sh +# +# TIP: avoid type-checkers (cargo check, tsc --noEmit) as the sole +# guard — their verbose output can distract the agent from the real +# task. Pair them with a test suite that validates behavior. -# guard cargo check +# guard cargo test # ── Judge cadence ───────────────────────────────────────────────────── -# By default the judge runs only after the worker signals DONE. -# Set judge-every to fire the judge as a quality checkpoint every N -# worker iterations (it still always fires on DONE too). +# By default the judge runs only after the worker signals DONE + guards +# pass. These settings give you finer control over judge timing. +# +# judge-every : also fire the judge as a mid-loop quality checkpoint +# every N worker iterations (the judge still always fires on DONE too, +# regardless of cadence). Mid-loop verdicts provide early feedback +# without stopping the worker. # # judge-every 5 -# Max consecutive judge failures before bailing out (default: 3). -# The counter resets to 0 after any passing verdict. +# max-judge-failures : max consecutive FAILs before bailing out. +# Default: 3. The counter is exact — bailout happens on the Nth FAIL, +# and resets to 0 after any PASS. On retry, verdict.md and +# guard-results.md are preserved so the worker sees judge feedback. # # max-judge-failures 3 -# ── Periodic agents (cadence-based supplementary agents) ────────────── -# Invoke an additional agent protocol at a fixed cadence (every N -# worker iterations). Useful for code cleanup, review passes, etc. -# The agent name is derived from the filename (cleaner.md → "cleaner"). +# ── Periodic agents ─────────────────────────────────────────────────── +# Supplementary agents invoked at a fixed cadence (every N iterations). +# Useful for cleanup passes, code review, metrics collection, etc. +# Each periodic gets its own fresh agent session. # # periodic # -# periodic .loop/cleaner.md 10 - -# Guards that run after a specific periodic agent completes. Results -# are written to .loop/periodic--results.md (separate from the -# worker's guard-results.md so the worker isn't confused). +# The agent name is derived from the filename stem: +# .loop/cleaner.md → name is "cleaner" +# .loop/reviewer.md → name is "reviewer" +# +# Examples: +# periodic .loop/cleaner.md 10 # cleanup every 10 iterations +# periodic .loop/reviewer.md 5 # review pass every 5 iterations +# +# guard-after: shell commands that run after a specific periodic agent +# completes. Results are written to .loop/periodic--results.md +# (kept separate from the worker's guard-results.md). Failures produce +# warnings but do not affect the main loop. # # guard-after # -# guard-after cleaner cargo test +# Example combo: +# periodic .loop/cleaner.md 10 +# guard-after cleaner cargo test +# guard-after cleaner cargo clippy -- -D warnings + +# ── Hooks (fire-and-forget post-iteration commands) ─────────────────── +# Shell commands that run after each iteration (after guards and +# periodics). Unlike guards, hook failures never block the loop or +# affect its exit code — non-zero exits produce a warning, nothing more. +# Output goes to your terminal only, never to files the agent reads. +# +# The YOKE_ITERATION env var contains the current iteration number. +# +# Examples: +# hook echo "iteration $YOKE_ITERATION done" +# hook git add -A && git commit -m "auto: iteration $YOKE_ITERATION" || true +# hook ./scripts/notify.sh +# hook curl -s -X POST "$WEBHOOK_URL" -d "{\"iteration\": $YOKE_ITERATION}" diff --git a/src/templates/loop/protocol.md b/src/templates/loop/protocol.md index 166af9e..ca687eb 100644 --- a/src/templates/loop/protocol.md +++ b/src/templates/loop/protocol.md @@ -37,6 +37,37 @@ The first line of `.loop/notes.md` must be one of: The outer loop reads this line. It exits only when `STATUS: DONE` **and** all guards pass. +## KEEP: Carrying File Context Across Iterations + +Your conversation history persists across iterations via `--resume`. To keep +Claude's prompt cache warm without ballooning, the outer loop trims your +session between rounds: it drops `Bash` output, thinking, and any file +`Read` results that aren't on your KEEP list. Everything else (text turns, +intermediate `Edit`/`Grep`/`Glob` results) is also dropped. + +After STATUS, on its own line in `.loop/notes.md`, list the file paths you +want to keep cached for the next iteration: + +``` +STATUS: IN_PROGRESS +KEEP: src/foo.rs src/bar.rs tests/baz.rs +``` + +Rules: + +- Space-separated repo-relative paths (or absolute — both work). +- List files you read **this iteration** and will still need next iteration. +- Don't list `.loop/notes.md`, `.loop/plan.md`, `.loop/protocol.md`, + `.loop/guard-results.md`, or `.loop/verdict.md` — those live on disk and + you re-read them fresh every iteration. Listing them is harmless but wastes + a slot. +- Keep the list tight. Every kept file is paid for (at cache-read rates, + ~10% of fresh) every round it stays kept. Drop a file once you're confident + you won't need it again. +- Omit the `KEEP:` line entirely (or `KEEP: *`) to keep nothing — your + conversation prefix shrinks to just the bootstrap. Use this after major + refactors or when you've moved to an unrelated area of the code. + ## What the Guards Check After you exit, the outer loop runs guards defined in `.loop/yoke.conf`. diff --git a/src/templates/loop/yoke.conf b/src/templates/loop/yoke.conf index 060c6e2..2c206d8 100644 --- a/src/templates/loop/yoke.conf +++ b/src/templates/loop/yoke.conf @@ -1,78 +1,163 @@ -# Yoke configuration -# Lines starting with # are comments. Blank lines are ignored. +# ╔══════════════════════════════════════════════════════════════════════╗ +# ║ Yoke configuration — loop mode ║ +# ╚══════════════════════════════════════════════════════════════════════╝ +# +# Loop mode iterates an agent until the job is done. Each iteration: +# +# 1. Restore protected files (protocol.md, plan.md, yoke.conf) +# 2. Invoke the agent (reads protocol.md, does work, updates notes.md) +# 3. Diff boundary check (are changed files within allowed scope?) +# 4. Run guards (test suites, linters — results go to guard-results.md) +# 5. Fire periodic agents (if cadence matches this iteration) +# 6. Run hooks (fire-and-forget side effects) +# 7. Check exit: STATUS: DONE in notes.md AND all guards pass → exit 0 +# +# Protected files are backed up at start and restored every iteration, +# so the agent can never permanently corrupt its own instructions. # ── Backend ──────────────────────────────────────────────────────────── -# Model to use for the agent. If unset, defaults to Claude CLI. -# Use provider/model format for OpenRouter or other opencode providers. +# Which LLM backend to use. Leave commented for Claude CLI (default). +# Setting `model` switches to the OpenCode backend, which supports +# OpenRouter, OpenAI, Anthropic API, and other providers. +# # model openrouter/anthropic/claude-sonnet-4 # model openai/gpt-4o # model anthropic/claude-sonnet-4 -# ── Sandbox ────────────────────────────────────────────────────────── -# Docker image to run the agent inside. Required unless you pass --no-sandbox. -# Note: sandbox is not currently supported with the 'model' directive. +# ── Sandbox ──────────────────────────────────────────────────────────── +# Docker image to run the agent inside. Your working directory is +# bind-mounted into the container at /workspace. Required unless you +# pass --no-sandbox on the command line. +# +# Note: sandbox is not currently supported with the `model` directive. + image claude-code-sandbox:latest -# ── Output ─────────────────────────────────────────────────────────── -# Max lines of tail output kept per guard in guard-results.md. -# Keeps the results file from exploding on verbose commands. +# ── Output ───────────────────────────────────────────────────────────── +# max-tail: max lines of output kept *per guard* in guard-results.md. +# Only affects what the agent reads back — full output still streams to +# your terminal. Default 200 is enough for most test suites; raise it +# if your guards produce essential output beyond 200 lines. + max-tail 200 -# Uncomment to save raw stream-json output per iteration. -# Each iteration writes to /iteration-.jsonl. +# log-dir: save raw stream-json output for each iteration. Useful for +# debugging agent behavior or auditing token usage. Files are named +# /iteration-.jsonl. +# # log-dir .loop/logs +# ── Metrics ──────────────────────────────────────────────────────────── +# Per-iteration timing + cost records are written as NDJSON, one row per +# iteration, plus a row-per-run with totals. By default these live under +# ~/.yoke/metrics// so they survive `yoke clean`, `yoke +# stash`, and project deletes. +# +# Inspect with: jq . ~/.yoke/metrics//.ndjson +# +# metrics-dir ~/.yoke/metrics # default +# metrics off # opt out of disk writes + # ── Scope rules (diff boundary enforcement) ────────────────────────── -# Controls what files the agent is allowed to change. After each iteration -# yoke diffs the working tree and checks every changed file against -# these rules. Most-specific (longest prefix) match wins. +# After each iteration yoke diffs the working tree and checks every +# changed file against these rules. If any file is out of scope, ALL +# guards are skipped and the agent gets only boundary feedback. +# Files under .loop/ are always exempt (yoke's own infrastructure). +# +# Three directives, most-specific (longest prefix) match wins: # -# Directives: # allow — any change permitted (add, modify, delete) -# add-only — new files only; existing files cannot be modified -# no-modify — no changes at all (adds or modifications rejected) +# add-only — new files OK; edits to existing files rejected +# no-modify — no changes at all (adds or edits rejected) # -# The prefix "." matches every path (root catch-all). +# The special prefix "." matches every path (root catch-all). # # Examples: -# allow src/ # full access under src/ -# add-only tests/ # can create new test files, not edit existing -# no-modify .github/ # CI config is off-limits -# allow . # fallback: everything else is allowed +# allow src/ # full access to source +# allow tests/ # full access to tests +# add-only docs/ # can add new docs, not edit existing +# no-modify .github/ # CI config is off-limits +# no-modify package-lock.json # protect a specific file +# allow . # fallback: everything else allowed allow . -# ── Guards (run in order, fail-fast) ───────────────────────────────── -# Shell commands executed after each agent iteration. If any guard -# exits non-zero the iteration is marked failed, remaining guards are -# skipped, and the results are fed back on the next pass. +# ── Guards (post-iteration validation) ──────────────────────────────── +# Shell commands that validate the agent's work. All guards run in +# parallel; results are collected in declared order and written to +# .loop/guard-results.md. The agent reads this file on its next turn, +# so failed guards become automatic feedback. # -# Common examples: -# guard cargo check +# If the boundary check fails, guards are skipped entirely — the agent +# must fix scope violations before guards will run again. +# +# The loop only exits when STATUS: DONE *and* all guards pass. If the +# agent declares DONE but a guard fails, it keeps iterating. +# +# Examples: # guard cargo test -# guard npm run lint +# guard npm test # guard python -m pytest tests/ -x -# guard make test +# guard go test ./... +# guard make check +# guard ./scripts/validate.sh # -# NOTE: avoid "cargo check" as the sole guard — its type-error output -# can confuse the agent into chasing compiler noise instead of finishing -# the task. Prefer a test suite or linter that validates behaviour. +# TIP: avoid type-checkers (cargo check, tsc --noEmit) as the sole +# guard — their verbose output can distract the agent from the real +# task. Pair them with a test suite that validates behavior. -# guard cargo check +# guard cargo test -# ── Periodic agents (cadence-based supplementary agents) ────────────── -# Invoke an additional agent protocol at a fixed cadence (every N -# worker iterations). Useful for code cleanup, review passes, etc. -# The agent name is derived from the filename (cleaner.md → "cleaner"). +# ── Session continuity (KEEP) ───────────────────────────────────────── +# The agent's Claude session is resumed across iterations to preserve the +# prompt cache. Between iterations, yoke trims the session JSONL down to +# the files the agent declares on a `KEEP:` line in .loop/notes.md, e.g.: +# +# STATUS: IN_PROGRESS +# KEEP: src/foo.rs tests/bar.rs +# +# Bash output, thinking, and other tool results are dropped. Only kept +# file Reads survive. .loop/notes.md, .loop/plan.md, .loop/protocol.md, +# .loop/guard-results.md are re-read fresh each iteration and don't need +# to be listed. Set YOKE_DISABLE_SESSION_TRIM=1 to skip trimming. + +# ── Periodic agents ─────────────────────────────────────────────────── +# Supplementary agents invoked at a fixed cadence (every N iterations). +# Useful for cleanup passes, code review, metrics collection, etc. +# Each periodic gets its own fresh agent session. # # periodic # -# periodic .loop/cleaner.md 10 - -# Guards that run after a specific periodic agent completes. Results -# are written to .loop/periodic--results.md (separate from the -# worker's guard-results.md so the worker isn't confused). +# The agent name is derived from the filename stem: +# .loop/cleaner.md → name is "cleaner" +# .loop/reviewer.md → name is "reviewer" +# +# Examples: +# periodic .loop/cleaner.md 10 # cleanup every 10 iterations +# periodic .loop/reviewer.md 5 # review pass every 5 iterations +# +# guard-after: shell commands that run after a specific periodic agent +# completes. Results are written to .loop/periodic--results.md +# (kept separate from the worker's guard-results.md). Failures produce +# warnings but do not affect the main loop. # # guard-after # -# guard-after cleaner cargo test +# Example combo: +# periodic .loop/cleaner.md 10 +# guard-after cleaner cargo test +# guard-after cleaner cargo clippy -- -D warnings + +# ── Hooks (fire-and-forget post-iteration commands) ─────────────────── +# Shell commands that run after each iteration (after guards and +# periodics). Unlike guards, hook failures never block the loop or +# affect its exit code — non-zero exits produce a warning, nothing more. +# Output goes to your terminal only, never to files the agent reads. +# +# The YOKE_ITERATION env var contains the current iteration number. +# +# Examples: +# hook echo "iteration $YOKE_ITERATION done" +# hook git add -A && git commit -m "auto: iteration $YOKE_ITERATION" || true +# hook ./scripts/notify.sh +# hook curl -s -X POST "$WEBHOOK_URL" -d "{\"iteration\": $YOKE_ITERATION}" diff --git a/src/templates/saga/yoke.conf b/src/templates/saga/yoke.conf index de48efb..eeb2913 100644 --- a/src/templates/saga/yoke.conf +++ b/src/templates/saga/yoke.conf @@ -1,70 +1,179 @@ -# Yoke configuration (saga mode) -# Lines starting with # are comments. Blank lines are ignored. +# ╔══════════════════════════════════════════════════════════════════════╗ +# ║ Yoke configuration — saga mode ║ +# ╚══════════════════════════════════════════════════════════════════════╝ +# +# Saga mode orchestrates large tasks by decomposing them into chunks. +# A scoper agent reads specification.md, writes a sub-plan, and a brute +# loop implements + verifies each chunk. On chunk failure the scoper +# re-scopes rather than aborting. +# +# Saga cycle: +# 1. Invoke scoper — reads spec, writes sub-plan.md, updates saga-notes.md +# 2. If saga-notes.md says STATUS: DONE → exit 0 (all chunks complete) +# 3. Run brute loop on sub-plan.md: +# a. Worker iterates until DONE + guards pass +# b. Judge verifies → PASS: next chunk / FAIL: retry +# c. After max-judge-failures consecutive FAILs → bailout +# 4. On brute PASS → loop back to scoper for next chunk +# 5. On brute bailout → loop back to scoper to re-scope the chunk +# +# Inner plan loop (per worker iteration): +# 1. Restore protected files (protocol.md, plan.md, yoke.conf) +# 2. Invoke the agent +# 3. Diff boundary check +# 4. Run guards → results to guard-results.md +# 5. Fire periodic agents (if cadence matches) +# 6. Run hooks +# 7. Check: STATUS: DONE + all guards pass → exit inner loop +# +# Worker notes are appended to saga-log.md between chunks so the scoper +# has full context of what has been accomplished so far. # ── Backend ──────────────────────────────────────────────────────────── -# Model to use for the agent. If unset, defaults to Claude CLI. -# Use provider/model format for OpenRouter or other opencode providers. +# Which LLM backend to use. Leave commented for Claude CLI (default). +# Setting `model` switches to the OpenCode backend, which supports +# OpenRouter, OpenAI, Anthropic API, and other providers. +# # model openrouter/anthropic/claude-sonnet-4 # model openai/gpt-4o # model anthropic/claude-sonnet-4 -# ── Sandbox ────────────────────────────────────────────────────────── -# Docker image to run the agent inside. Required unless you pass --no-sandbox. -# Note: sandbox is not currently supported with the 'model' directive. +# ── Sandbox ──────────────────────────────────────────────────────────── +# Docker image to run the agent inside. Your working directory is +# bind-mounted into the container at /workspace. Required unless you +# pass --no-sandbox on the command line. +# +# Note: sandbox is not currently supported with the `model` directive. + image claude-code-sandbox:latest -# ── Output ─────────────────────────────────────────────────────────── -# Max lines of tail output kept per guard in guard-results.md. +# ── Output ───────────────────────────────────────────────────────────── +# max-tail: max lines of output kept *per guard* in guard-results.md. +# Only affects what the agent reads back — full output still streams to +# your terminal. Default 200 is enough for most test suites; raise it +# if your guards produce essential output beyond 200 lines. + max-tail 200 -# Uncomment to save raw stream-json output per iteration. +# log-dir: save raw stream-json output for each iteration. Useful for +# debugging agent behavior or auditing token usage. Files are named +# /iteration-.jsonl. +# # log-dir .loop/logs +# ── Metrics ──────────────────────────────────────────────────────────── +# Per-iteration timing + cost records are written as NDJSON under +# ~/.yoke/metrics// by default. Survives `yoke clean`. +# +# metrics-dir ~/.yoke/metrics # default +# metrics off # opt out of disk writes + # ── Scope rules (diff boundary enforcement) ────────────────────────── -# Controls what files the agent is allowed to change. Most-specific -# (longest prefix) match wins. +# After each iteration yoke diffs the working tree and checks every +# changed file against these rules. If any file is out of scope, ALL +# guards are skipped and the agent gets only boundary feedback. +# Files under .loop/ are always exempt (yoke's own infrastructure). +# +# Three directives, most-specific (longest prefix) match wins: # # allow — any change permitted (add, modify, delete) -# add-only — new files only; existing files cannot be modified -# no-modify — no changes at all (adds or modifications rejected) +# add-only — new files OK; edits to existing files rejected +# no-modify — no changes at all (adds or edits rejected) +# +# The special prefix "." matches every path (root catch-all). +# +# Examples: +# allow src/ # full access to source +# allow tests/ # full access to tests +# add-only docs/ # can add new docs, not edit existing +# no-modify .github/ # CI config is off-limits +# no-modify package-lock.json # protect a specific file +# allow . # fallback: everything else allowed allow . -# ── Guards (run after each plan stage, fail-fast) ──────────────────── -# Shell commands executed after each agent iteration. If any guard -# exits non-zero the iteration fails and results are fed back. +# ── Guards (post-iteration validation) ──────────────────────────────── +# Shell commands that validate the agent's work. All guards run in +# parallel; results are collected in declared order and written to +# .loop/guard-results.md. The agent reads this file on its next turn, +# so failed guards become automatic feedback. # -# NOTE: avoid "cargo check" as the sole guard — its type-error output -# can confuse the agent into chasing compiler noise instead of finishing -# the task. Prefer a test suite or linter that validates behaviour. +# If the boundary check fails, guards are skipped entirely — the agent +# must fix scope violations before guards will run again. +# +# The inner loop only exits when STATUS: DONE *and* all guards pass. +# +# Examples: +# guard cargo test +# guard npm test +# guard python -m pytest tests/ -x +# guard go test ./... +# guard make check +# guard ./scripts/validate.sh +# +# TIP: avoid type-checkers (cargo check, tsc --noEmit) as the sole +# guard — their verbose output can distract the agent from the real +# task. Pair them with a test suite that validates behavior. -# guard cargo check +# guard cargo test # ── Judge cadence ───────────────────────────────────────────────────── -# By default the judge runs only after the worker signals DONE. -# Set judge-every to fire the judge as a quality checkpoint every N -# worker iterations (it still always fires on DONE too). +# By default the judge runs only after the worker signals DONE + guards +# pass. These settings give you finer control over judge timing. +# +# judge-every : also fire the judge as a mid-loop quality checkpoint +# every N worker iterations (the judge still always fires on DONE too, +# regardless of cadence). Mid-loop verdicts provide early feedback +# without stopping the worker. # # judge-every 5 -# Max consecutive judge failures before bailing out (default: 3). -# The counter resets to 0 after any passing verdict. +# max-judge-failures : max consecutive FAILs before bailing out. +# Default: 3. The counter is exact — bailout happens on the Nth FAIL, +# and resets to 0 after any PASS. On retry, verdict.md and +# guard-results.md are preserved so the worker sees judge feedback. +# In saga mode, bailout returns control to the scoper for re-scoping +# rather than aborting the entire saga. # # max-judge-failures 3 -# ── Periodic agents (cadence-based supplementary agents) ────────────── -# Invoke an additional agent protocol at a fixed cadence (every N -# worker iterations). Useful for code cleanup, review passes, etc. -# The agent name is derived from the filename (cleaner.md → "cleaner"). +# ── Periodic agents ─────────────────────────────────────────────────── +# Supplementary agents invoked at a fixed cadence (every N iterations). +# Useful for cleanup passes, code review, metrics collection, etc. +# Each periodic gets its own fresh agent session. # # periodic # -# periodic .loop/cleaner.md 10 - -# Guards that run after a specific periodic agent completes. Results -# are written to .loop/periodic--results.md (separate from the -# worker's guard-results.md so the worker isn't confused). +# The agent name is derived from the filename stem: +# .loop/cleaner.md → name is "cleaner" +# .loop/reviewer.md → name is "reviewer" +# +# Examples: +# periodic .loop/cleaner.md 10 # cleanup every 10 iterations +# periodic .loop/reviewer.md 5 # review pass every 5 iterations +# +# guard-after: shell commands that run after a specific periodic agent +# completes. Results are written to .loop/periodic--results.md +# (kept separate from the worker's guard-results.md). Failures produce +# warnings but do not affect the main loop. # # guard-after # -# guard-after cleaner cargo test +# Example combo: +# periodic .loop/cleaner.md 10 +# guard-after cleaner cargo test +# guard-after cleaner cargo clippy -- -D warnings + +# ── Hooks (fire-and-forget post-iteration commands) ─────────────────── +# Shell commands that run after each iteration (after guards and +# periodics). Unlike guards, hook failures never block the loop or +# affect its exit code — non-zero exits produce a warning, nothing more. +# Output goes to your terminal only, never to files the agent reads. +# +# The YOKE_ITERATION env var contains the current iteration number. +# +# Examples: +# hook echo "iteration $YOKE_ITERATION done" +# hook git add -A && git commit -m "auto: iteration $YOKE_ITERATION" || true +# hook ./scripts/notify.sh +# hook curl -s -X POST "$WEBHOOK_URL" -d "{\"iteration\": $YOKE_ITERATION}" diff --git a/tests/judge_adversarial.rs b/tests/judge_adversarial.rs index 842a7da..81e51b4 100644 --- a/tests/judge_adversarial.rs +++ b/tests/judge_adversarial.rs @@ -97,26 +97,22 @@ fn stash_roundtrip_after_extraction() { let stderr = String::from_utf8_lossy(&out.stderr); assert!(stderr.contains("mode=brute"), "stash log should show mode=brute, got:\n{}", stderr); - // Clean — this should auto-stash then wipe - let out = Command::new(&yoke) - .args(["clean"]) - .current_dir(project) - .output() - .expect("yoke clean"); - assert!(out.status.success(), "yoke clean failed: {}", String::from_utf8_lossy(&out.stderr)); + // Verify stash cleared .loop/ working files + assert!(!loop_dir.join("plan.md").exists(), "plan.md should be gone after stash"); + assert!(!loop_dir.join("notes.md").exists(), "notes.md should be gone after stash"); + assert!(loop_dir.join(".stash").exists(), ".stash/ should survive stash clear"); - // Verify plan.md was emptied by clean - let plan = fs::read_to_string(loop_dir.join("plan.md")).unwrap(); - assert!(plan.is_empty(), "plan.md should be empty after clean, got: {:?}", plan); - - // Pop — should restore the auto-stashed state (which is the post-clean state, - // but let's verify we can pop without error, meaning the index is intact) + // Pop — should restore the stashed state with our distinctive content let out = Command::new(&yoke) .args(["stash", "pop"]) .current_dir(project) .output() .expect("yoke stash pop"); assert!(out.status.success(), "yoke stash pop failed: {}", String::from_utf8_lossy(&out.stderr)); + + // Verify round-trip: files restored with original content + let plan = fs::read_to_string(loop_dir.join("plan.md")).unwrap(); + assert!(plan.contains("Build the widget"), "plan.md should be restored after pop, got: {:?}", plan); } // ── Test 2: plan loop exits on STATUS: DONE with generalized is_status_done ── diff --git a/tests/metrics_persistence.rs b/tests/metrics_persistence.rs new file mode 100644 index 0000000..8ff0121 --- /dev/null +++ b/tests/metrics_persistence.rs @@ -0,0 +1,256 @@ +//! Behavioral test: a completed yoke run persists per-iteration and per-run +//! metrics rows under the user's home directory (`~/.yoke/metrics/`) and the +//! rows survive `yoke clean`. +//! +//! Uses a mock `claude` script so no real agent invocation happens. The +//! subprocess gets HOME pointed at the test tempdir so writes don't escape +//! the test sandbox. + +use std::fs; +use std::os::unix::fs::PermissionsExt; +use std::process::Command; + +fn yoke_bin() -> std::path::PathBuf { + let mut path = std::env::current_exe() + .expect("current_exe") + .parent() + .expect("parent of test binary") + .parent() + .expect("parent of deps dir") + .to_path_buf(); + path.push("yoke"); + path +} + +const PROTOCOL: &str = "\ +# Protocol + +Single-iteration test. Write STATUS: DONE and exit. +"; + +const PLAN: &str = "\ +## Stage 1 + +Be done. +"; + +const CONF: &str = "\ +allow . +"; + +const MOCK_CLAUDE: &str = r#"#!/usr/bin/env bash +set -euo pipefail +# Always: write STATUS: DONE so the loop exits after one iteration. +printf 'STATUS: DONE\n\n## Stage 1\nDone.\n' > .loop/notes.md +exit 0 +"#; + +#[test] +fn metrics_rows_persist_under_home_and_survive_clean() { + // Build yoke + let status = Command::new("cargo") + .args(["build", "--quiet"]) + .status() + .expect("cargo build"); + assert!(status.success(), "cargo build failed"); + + let yoke = yoke_bin(); + let tmp = tempfile::tempdir().expect("tempdir"); + let project = tmp.path().join("proj"); + fs::create_dir(&project).expect("create proj"); + + // Isolated HOME so metrics land under tmp/.yoke/metrics/... + let fake_home = tmp.path().join("home"); + fs::create_dir(&fake_home).expect("create home"); + + // Project files + let loop_dir = project.join(".loop"); + fs::create_dir(&loop_dir).expect("create .loop"); + fs::write(loop_dir.join("protocol.md"), PROTOCOL).unwrap(); + fs::write(loop_dir.join("plan.md"), PLAN).unwrap(); + fs::write(loop_dir.join("yoke.conf"), CONF).unwrap(); + fs::write(loop_dir.join("notes.md"), "").unwrap(); + fs::write(loop_dir.join("guard-results.md"), "").unwrap(); + + // Mock claude on PATH + let mock_bin_dir = project.join("mock-bin"); + fs::create_dir(&mock_bin_dir).expect("create mock-bin"); + let mock_claude_path = mock_bin_dir.join("claude"); + fs::write(&mock_claude_path, MOCK_CLAUDE).unwrap(); + fs::set_permissions(&mock_claude_path, fs::Permissions::from_mode(0o755)).unwrap(); + + // Boundary check needs a git repo + let git = |args: &[&str]| { + let out = Command::new("git") + .args(args) + .current_dir(&project) + .env("GIT_CONFIG_NOSYSTEM", "1") + .env("GIT_AUTHOR_NAME", "test") + .env("GIT_AUTHOR_EMAIL", "test@test") + .env("GIT_COMMITTER_NAME", "test") + .env("GIT_COMMITTER_EMAIL", "test@test") + .output() + .unwrap_or_else(|e| panic!("git {:?} failed: {}", args, e)); + assert!( + out.status.success(), + "git {:?} failed: {}", + args, + String::from_utf8_lossy(&out.stderr) + ); + }; + git(&["init"]); + fs::write(project.join("seed.txt"), "x\n").unwrap(); + git(&["add", "seed.txt"]); + git(&["-c", "user.name=t", "-c", "user.email=t@t", "commit", "-m", "init"]); + + let original_path = std::env::var("PATH").unwrap_or_default(); + let test_path = format!("{}:{}", mock_bin_dir.display(), original_path); + + let output = Command::new(&yoke) + .args(["run", "--no-sandbox"]) + .current_dir(&project) + .env("PATH", &test_path) + .env("HOME", &fake_home) + .output() + .expect("failed to run yoke"); + let stderr = String::from_utf8_lossy(&output.stderr); + assert!( + output.status.success(), + "yoke should exit 0 on STATUS: DONE.\nstderr:\n{}", + stderr + ); + + // Find the metrics directory: ~/.yoke/metrics// + let metrics_dir = fake_home.join(".yoke").join("metrics"); + assert!( + metrics_dir.exists(), + "metrics dir should be created at {}", + metrics_dir.display() + ); + let project_dirs: Vec<_> = fs::read_dir(&metrics_dir) + .expect("read metrics dir") + .filter_map(|e| e.ok()) + .map(|e| e.path()) + .collect(); + assert_eq!( + project_dirs.len(), + 1, + "expected exactly one project slug dir, got {:?}", + project_dirs + ); + let slug_dir = &project_dirs[0]; + + // The slug dir must contain runs.ndjson + at least one .ndjson + let mut runs_path = None; + let mut iter_path = None; + for entry in fs::read_dir(slug_dir).expect("read slug dir").flatten() { + let name = entry.file_name().to_string_lossy().into_owned(); + if name == "runs.ndjson" { + runs_path = Some(entry.path()); + } else if name.ends_with(".ndjson") { + iter_path = Some(entry.path()); + } + } + let runs_path = runs_path.expect("runs.ndjson should exist"); + let iter_path = iter_path.expect("iteration ndjson file should exist"); + + // Run row: contains outcome=done and iterations >= 1 + let runs_content = fs::read_to_string(&runs_path).expect("read runs.ndjson"); + assert!( + runs_content.contains("\"outcome\":\"done\""), + "runs.ndjson should record outcome=done.\ncontent: {}", + runs_content + ); + assert!( + runs_content.contains("\"iterations\":1"), + "runs.ndjson should report 1 iteration.\ncontent: {}", + runs_content + ); + + // Iteration row: has mode=loop, status_done=true, restore_ms field + let iter_content = fs::read_to_string(&iter_path).expect("read iter ndjson"); + assert!( + iter_content.contains("\"mode\":\"loop\""), + "iter row should carry mode=loop.\ncontent: {}", + iter_content + ); + assert!( + iter_content.contains("\"status_done\":true"), + "iter row should record status_done=true.\ncontent: {}", + iter_content + ); + assert!( + iter_content.contains("\"restore_ms\":"), + "iter row should record restore_ms.\ncontent: {}", + iter_content + ); + + // Survive `yoke clean` + let clean_output = Command::new(&yoke) + .args(["clean"]) + .current_dir(&project) + .env("HOME", &fake_home) + .output() + .expect("yoke clean"); + assert!(clean_output.status.success(), "yoke clean failed"); + assert!( + runs_path.exists(), + "runs.ndjson must survive yoke clean (lives outside the project)" + ); + assert!( + iter_path.exists(), + "iteration ndjson must survive yoke clean" + ); + + // `yoke stats` lists the run we just recorded + let stats_output = Command::new(&yoke) + .args(["stats"]) + .current_dir(&project) + .env("HOME", &fake_home) + .output() + .expect("yoke stats"); + assert!(stats_output.status.success(), "yoke stats failed"); + let stats_stdout = String::from_utf8_lossy(&stats_output.stdout); + let stats_stderr = String::from_utf8_lossy(&stats_output.stderr); + let combined = format!("{}{}", stats_stdout, stats_stderr); + // Derive run-id from the file name we found earlier + let run_id = iter_path + .file_stem() + .and_then(|s| s.to_str()) + .expect("run id from file stem"); + assert!( + combined.contains(run_id), + "yoke stats should list the run-id {} in its output.\ncombined:\n{}", + run_id, + combined + ); + assert!( + combined.contains("done"), + "yoke stats should show the run outcome 'done'.\ncombined:\n{}", + combined + ); + + // `yoke stats --run ` shows the iteration table + let run_output = Command::new(&yoke) + .args(["stats", "--run", run_id]) + .current_dir(&project) + .env("HOME", &fake_home) + .output() + .expect("yoke stats --run"); + assert!(run_output.status.success(), "yoke stats --run failed"); + let run_stdout = String::from_utf8_lossy(&run_output.stdout); + let run_stderr = String::from_utf8_lossy(&run_output.stderr); + let run_combined = format!("{}{}", run_stdout, run_stderr); + assert!( + run_combined.contains("ITER"), + "yoke stats --run should print an iteration table header.\noutput:\n{}", + run_combined + ); + assert!( + // We produced one iteration; the row should show iteration "1" + // and one of the "pass" / "yes" status badges. + run_combined.contains("pass"), + "yoke stats --run should report guard status.\noutput:\n{}", + run_combined + ); +}