From fa65ddb605dac8d9a5dcbd844629647abfbc3d42 Mon Sep 17 00:00:00 2001 From: Zachery Aaron Shores-Chmielewski Date: Thu, 23 Jul 2026 14:39:31 +0400 Subject: [PATCH] stash: changes to thinking behavior --- CONTEXT_TRIM.md | 64 ----------- behavioral-specification.md | 200 ---------------------------------- src/config.rs | 57 ++++++++++ src/main.rs | 19 +++- src/templates/brute/yoke.conf | 23 ++++ src/templates/loop/yoke.conf | 23 ++++ src/templates/saga/yoke.conf | 23 ++++ 7 files changed, 143 insertions(+), 266 deletions(-) delete mode 100644 CONTEXT_TRIM.md delete mode 100644 behavioral-specification.md diff --git a/CONTEXT_TRIM.md b/CONTEXT_TRIM.md deleted file mode 100644 index 8282ba6..0000000 --- a/CONTEXT_TRIM.md +++ /dev/null @@ -1,64 +0,0 @@ -# Context trim — design notes - -## What's shipped - -Worker sessions are now resumed across iterations with `claude --resume `, -and the session JSONL is surgically trimmed between rounds so Anthropic's -prefix cache stays warm without paying for irrelevant history. - -- Agent declares a `KEEP: ` line in `.loop/notes.md`. -- Yoke locates `~/.claude/projects//.jsonl` and rewrites it - to retain only: bootstrap records, the initial user prompt, `Read` - tool_use/tool_result pairs for kept paths, and attachments. Drops: - `thinking`, intermediate `text`, every non-`Read` tool_use, `Read`s of - non-kept files, and the matching `tool_result`s. Re-links the - parent-uuid chain across the gaps; validates tool_use ↔ tool_result - pairing before commit; keeps a `.bak`. -- Judge (brute) and scoper (saga) always run fresh — independence per - behavioral-specification §2.1. -- `YOKE_DISABLE_SESSION_TRIM=1` is the escape hatch. - -## What still needs adding - -1. **Format-drift guard.** The Claude Code session JSONL is undocumented. - A future CLI release could rename a field, change content-block shape, - or move the file. The validator catches most damage post-trim, but - pre-trim we should fingerprint the format (e.g., known top-level keys - on bootstrap records) and bail if it drifts. Today we trust + bail on - validate-fail; a positive check would be safer. - -2. **Recovery from `--resume` failure.** If Claude rejects the resumed - session (deleted, corrupted, version skew), the iteration aborts. We - should detect this from the spawn's exit/early stream error and - transparently retry once with no `--resume` (treat last_session_id as - stale). - -3. **Sandboxed runs.** When `image` is set, the agent runs inside a - Docker container — the session JSONL lives in the container's home, - not the host's. Today `trim_worker_session` no-ops in that case - (silent). Either mount the session dir into the container, or run the - trim inside the container, or document the limitation. - -4. **OpenCode backend.** Trim is Claude-specific. OpenCode users get - `--resume` benefits skipped (different session model). If OpenCode - becomes a first-class target, we need an analogous trim or a - reasoned-down equivalent. - -5. **Fork policy.** v1 has no forking: the trim alone bounds growth. - But sessions still grow monotonically in the *kept* portion, and - long-running tasks will eventually want a hard reset. A - `context-fork-every N` directive (or agent-declared `RESET: TRUE`) - would let users break the conversation cleanly at stage boundaries. - -6. **Periodic-agent isolation.** Periodics spawn fresh sessions but - share the project's session directory. If a periodic ever needed its - own resumable continuity (e.g., a reviewer agent that learns over - runs), today there's no separate session-id tracking for it. - -7. **Saga handoff.** Across saga cycles the scoper is fresh and reads - `saga-log.md` to reconstruct context. A future variant could let the - *brute worker inside saga* keep its session across cycles when the - scoper produces a closely-related sub-plan — but only if the scoper - signals it (otherwise context bleeds between unrelated chunks). - -Next steps involve creating some test vectors to help make behavior standardized diff --git a/behavioral-specification.md b/behavioral-specification.md deleted file mode 100644 index d8ba19a..0000000 --- a/behavioral-specification.md +++ /dev/null @@ -1,200 +0,0 @@ -# Yoke Behavioral Specification - -This document defines what yoke promises to its users. Every statement here is -a testable invariant over observables — files, exit codes, process behavior. -No statement references internal functions, line numbers, or implementation -details. These invariants survive refactors and rewrites. - ---- - -## 1. The Loop - -**Story:** You write a plan, a protocol, and some guards. You run `yoke run`. -An agent executes your plan iteratively. Each iteration, it reads the protocol, -does work, and updates notes. Guards check the work. When the agent writes -`STATUS: DONE` and all guards pass, yoke exits. - -### Invariants - -**1.1 — The spec is immutable from the agent's perspective.** -`protocol.md`, `plan.md`, and `yoke.conf` are backed up before the loop and -restored before every iteration. The agent can overwrite them during its turn, -but those changes do not persist to the next iteration. - -**1.2 — Termination requires both signals.** -The loop only exits when `STATUS: DONE` appears in `notes.md` AND all guards -pass. Neither condition alone is sufficient. If guards fail but status is DONE, -the loop continues with feedback. If guards pass but status is not DONE, the -loop continues. - -**1.3 — Guard feedback is visible.** -`guard-results.md` is written after every iteration. The agent sees it on its -next turn. No guard result is silently swallowed. - -**1.4 — Boundary violations block guards.** -If the diff boundary check fails, all configured guards are skipped (not run). -The agent gets boundary feedback only. Guards do not run on invalid state. - -**1.5 — Interrupts are clean.** -SIGINT kills the child process immediately. The loop does not exit -mid-iteration leaving partial state — it completes the signal check and exits -at the next safe point with code 130. - ---- - -## 2. The Judge - -**Story:** In brute mode, after the worker says DONE and guards pass, a -separate fresh agent (the judge) runs. It reads `judge.md`, tests the feature, -and writes `VERDICT: PASS` or `VERDICT: FAIL` to `verdict.md`. On PASS, yoke -exits successfully. On FAIL, the worker retries. - -### Invariants - -**2.1 — The judge is independent.** -It is a fresh agent invocation with no shared context from the worker. Its only -input is `judge.md` and the codebase state. - -**2.2 — Verdict survives retries and restarts.** -`verdict.md` is never cleared automatically. On judge FAIL, the worker sees -the judge's feedback on its next iteration. On bailout, the verdict remains -on disk so the user (or agent on restart) can read why the loop failed. -Use `yoke clean` or `yoke stash` to reset. - -**2.3 — Guard results survive retries and restarts.** -Same as verdict — `guard-results.md` is never cleared automatically. It -persists across brute retries and restarts so the worker sees what the -guards reported. - -**2.4 — Notes status reset on retry, nothing else.** -On judge FAIL, only the first line of `notes.md` is overwritten to -`STATUS: IN_PROGRESS`. The rest of the file — the agent's prior iteration -notes — is preserved. All other files remain as-is. The worker starts with a -clean status but full context from both its own notes and the judge's verdict. - -**2.5 — Bailout is exact.** -If `max-judge-failures` consecutive judge FAILs occur, yoke exits non-zero. -The count is exact — `max-judge-failures 2` means bailout on the 2nd -consecutive FAIL, not the 3rd. - -**2.6 — Judge-every overrides cadence on DONE.** -If `judge-every` is configured and the worker signals DONE, the judge fires -immediately regardless of whether the iteration is on the cadence boundary. -DONE always triggers judgment. - ---- - -## 3. The Stash - -**Story:** `yoke stash` saves the current `.loop/` state. `yoke stash pop` -restores the most recent snapshot. `yoke stash checkout ` restores a -specific snapshot. `yoke clean` auto-stashes before wiping. - -### Invariants - -**3.1 — Stash clears the working directory.** -`stash` snapshots all non-dotfile files in `.loop/`, then removes them. -After stash, `.loop/` contains only dotfile entries (like `.stash/`). -`stash` then `pop` (or `checkout`) restores the original contents — no -file is lost, truncated, or corrupted. - -**3.2 — Auto-stash before destructive operations.** -Both `clean` and `checkout` auto-stash current state before modifying it. You -can always recover what was there before. - -**3.3 — Mode tag is recorded.** -Each stash entry records the mode (loop/brute/saga) that was active when it -was created. This tag is preserved in the index and survives restore -operations. - -**3.4 — Index is append-only.** -Stash never modifies or deletes existing index lines. New entries are appended. -The index is a history, not a mutable pointer. - -**3.5 — Prefix matching is unambiguous.** -`checkout abc` matches any entry starting with `abc`. If multiple entries -match, yoke errors instead of guessing. No silent wrong restore. - ---- - -## 4. The Saga - -**Story:** Saga mode has a scoper agent that reads `specification.md`, -decomposes it into chunks, writes each chunk to `sub-plan.md`, and a brute -loop implements and verifies each chunk. When all chunks are done, the scoper -writes `STATUS: DONE` to `saga-notes.md`. - -### Invariants - -**4.1 — Saga completion checks saga-notes, not notes.** -The saga loop checks `saga-notes.md` for DONE. `notes.md` is local to each -brute chunk and is cleared between chunks. Checking `notes.md` would be -checking the wrong file. - -**4.2 — Brute bailout triggers re-scoping, not abort.** -If brute fails `max-judge-failures` times on a chunk, control returns to the -scoper. The scoper can re-scope the same chunk differently. The saga does not -abort on a single chunk failure. - -**4.3 — Sub-plan must be non-empty.** -If the scoper produces an empty `sub-plan.md`, the saga aborts. This prevents -a brute loop from running with no plan. - -**4.4 — Chunk state persists and is logged.** `notes.md`, `verdict.md`, and -`guard-results.md` are NOT cleared between chunks. The scoper can read why -the previous chunk failed or succeeded. Before each chunk, the contents of -`notes.md` are appended to `saga-log.md`. Use `yoke clean` or `yoke stash` -for a full reset. - -**4.5 — Saga log is append-only.** `saga-log.md` accumulates the worker's -notes from every completed chunk. It is never cleared or truncated during a -saga run. Each entry is labeled with its chunk number. - ---- - -## 5. Config - -**Story:** `yoke.conf` defines the rules of the loop — what files are -protected, what guards run, how the judge behaves. It is parsed once at -startup and applied consistently throughout the run. - -### Invariants - -**5.1 — Valid configs parse.** -Every legal combination of directives parses without error. - -**5.2 — Invalid configs fail loudly.** -Unknown directives, malformed values, and missing required fields produce clear -errors — not silent defaults. - -**5.3 — Scope rules resolve most-specific-wins.** -If `allow src/` and `no-modify src/main.rs` are both configured, `src/main.rs` -is protected and `src/other.rs` is allowed. Longer prefix wins. - -**5.4 — Guard-after requires its periodic.** -A `guard-after` referencing a periodic that does not exist is a config error, -not a silent no-op. - -**5.5 — Hooks are fire-and-forget.** -A `hook` command that exits non-zero or fails to execute produces a warning on -stderr but does not affect the loop's exit code, guard evaluation, or iteration -flow. Hook output is never written to any file the agent reads. - ---- - -## 6. Mode Switching - -**Story:** You can switch between loop, brute, and saga without losing -progress. Each mode's state is snapshotted when you leave it and restored -when you return. - -### Invariants - -**6.1 — Mode switch stashes current state.** -Switching from mode A to B stashes all of A's files via `yoke stash`. The -stash entry is tagged with mode A. Current state is always recoverable. - -**6.2 — Mode switch always fresh-inits.** -After stashing, the target mode is initialized with fresh template files. -Previous sessions are not auto-restored. Use `yoke stash checkout` to -restore a prior session. diff --git a/src/config.rs b/src/config.rs index 6738dd9..7138619 100644 --- a/src/config.rs +++ b/src/config.rs @@ -7,6 +7,26 @@ pub enum Backend { OpenCode, } +#[derive(Debug, Clone, Copy, PartialEq)] +pub enum Thinking { + Off, + Low, + Medium, + High, +} + +impl Thinking { + /// MAX_THINKING_TOKENS value to forward to the Claude CLI. + pub fn max_tokens(self) -> u32 { + match self { + Thinking::Off => 0, + Thinking::Low => 2000, + Thinking::Medium => 10000, + Thinking::High => 32000, + } + } +} + #[derive(Debug, Clone, PartialEq)] pub enum ScopeTag { Allow, @@ -34,6 +54,8 @@ pub struct Config { pub log_dir: Option, pub image: Option, pub model: Option, + pub claude_model: Option, + pub thinking: Option, pub scope_rules: Vec, pub guards: Vec, pub judge_every: Option, @@ -51,6 +73,8 @@ struct ConfigBuilder { log_dir: Option, image: Option, model: Option, + claude_model: Option, + thinking: Option, scope_rules: Vec, guards: Vec, judge_every: Option, @@ -107,6 +131,8 @@ impl ConfigBuilder { log_dir: None, image: None, model: None, + claude_model: None, + thinking: None, scope_rules: Vec::new(), guards: Vec::new(), judge_every: None, @@ -129,6 +155,22 @@ impl ConfigBuilder { "log-dir" => self.log_dir = Some(value.to_string()), "image" => self.image = Some(value.to_string()), "model" => self.model = Some(value.to_string()), + "claude-model" => self.claude_model = Some(value.to_string()), + "thinking" => { + self.thinking = Some(match value.trim() { + "off" => Thinking::Off, + "low" => Thinking::Low, + "medium" => Thinking::Medium, + "high" => Thinking::High, + other => { + return Err(cfg_err( + path, + line_num, + &format!("thinking must be 'off', 'low', 'medium', or 'high', got '{}'", other), + )); + } + }); + } "allow" => self.scope_rules.push(ScopeRule { tag: ScopeTag::Allow, prefix: value.to_string() }), "add-only" => self.scope_rules.push(ScopeRule { tag: ScopeTag::AddOnly, prefix: value.to_string() }), "no-modify" => self.scope_rules.push(ScopeRule { tag: ScopeTag::NoModify, prefix: value.to_string() }), @@ -156,6 +198,19 @@ impl ConfigBuilder { } fn build(self, path: &Path) -> Result { + if self.model.is_some() && self.claude_model.is_some() { + return Err(cfg_err( + path, + 0, + "'model' (OpenCode backend) and 'claude-model' (Claude CLI backend) are mutually exclusive", + )); + } + if self.model.is_some() && self.thinking.is_some() { + eprintln!( + "warning: {}: 'thinking' directive is only honored by the Claude CLI backend; ignored when 'model' (OpenCode) is set", + path.display(), + ); + } let mut periodics = self.periodics; for (pname, cmd, ln) in self.pending_guard_afters { match periodics.iter_mut().find(|p| p.name == pname) { @@ -172,6 +227,8 @@ impl ConfigBuilder { log_dir: self.log_dir, image: self.image, model: self.model, + claude_model: self.claude_model, + thinking: self.thinking, scope_rules: self.scope_rules, guards: self.guards, judge_every: self.judge_every, diff --git a/src/main.rs b/src/main.rs index f3df5a4..6cf204d 100644 --- a/src/main.rs +++ b/src/main.rs @@ -493,7 +493,7 @@ fn container_home(image: &str) -> String { } /// Build a docker command that runs the claude CLI inside a container. -fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command { +fn build_docker_claude_command(image: &str, claude_args: &[&str], extra_env: &[(&str, String)]) -> Command { let workdir = std::env::current_dir() .unwrap_or_else(|_| PathBuf::from(".")) .to_string_lossy() @@ -530,6 +530,10 @@ fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command { } } + for (key, val) in extra_env { + c.arg("-e").arg(format!("{}={}", key, val)); + } + if let Some(home) = std::env::var_os("HOME") { let home_path = PathBuf::from(&home); let claude_dir = home_path.join(".claude"); @@ -570,17 +574,28 @@ fn build_command(config: &Config, prompt: &str, resume: Option<&str>) -> Command "--include-partial-messages", "--dangerously-skip-permissions", ]; + if let Some(m) = config.claude_model.as_deref() { + claude_args.push("--model"); + claude_args.push(m); + } if let Some(sid) = resume { claude_args.push("--resume"); claude_args.push(sid); } claude_args.push("-p"); claude_args.push(prompt); + let extra_env: Vec<(&str, String)> = match config.thinking { + Some(t) => vec![("MAX_THINKING_TOKENS", t.max_tokens().to_string())], + None => Vec::new(), + }; match config.image { - Some(ref image) => build_docker_claude_command(image, &claude_args), + Some(ref image) => build_docker_claude_command(image, &claude_args, &extra_env), None => { let mut c = Command::new("claude"); c.args(&claude_args); + for (key, val) in &extra_env { + c.env(key, val); + } c } } diff --git a/src/templates/brute/yoke.conf b/src/templates/brute/yoke.conf index b9e16b2..30eee3c 100644 --- a/src/templates/brute/yoke.conf +++ b/src/templates/brute/yoke.conf @@ -33,6 +33,29 @@ # model openai/gpt-4o # model anthropic/claude-sonnet-4 +# ── Claude model (optional) ──────────────────────────────────────────── +# Override the model the Claude CLI uses for each iteration. Leave +# commented to use Claude Code's default (Opus). Useful for trading some +# reasoning depth for faster, cheaper iterations. +# +# claude-model claude-sonnet-4-6 +# claude-model claude-haiku-4-5 + +# ── Thinking budget (optional) ───────────────────────────────────────── +# Cap extended-thinking tokens per turn for the Claude CLI backend. +# Useful when running smaller/faster models (sonnet, haiku) and you'd +# rather they spend the iteration acting than reasoning. Sets the +# MAX_THINKING_TOKENS env var on the agent invocation. +# +# thinking off # disable extended thinking entirely (0 tokens) +# thinking low # 2k tokens +# thinking medium # 10k tokens +# thinking high # 32k tokens +# +# Ignored by the OpenCode backend (warns at config load). +# +# thinking low + # ── Sandbox ──────────────────────────────────────────────────────────── # Docker image to run the agent inside. Your working directory is # bind-mounted into the container at /workspace. Required unless you diff --git a/src/templates/loop/yoke.conf b/src/templates/loop/yoke.conf index 2c206d8..8b4d399 100644 --- a/src/templates/loop/yoke.conf +++ b/src/templates/loop/yoke.conf @@ -24,6 +24,29 @@ # model openai/gpt-4o # model anthropic/claude-sonnet-4 +# ── Claude model (optional) ──────────────────────────────────────────── +# Override the model the Claude CLI uses for each iteration. Leave +# commented to use Claude Code's default (Opus). Useful for trading some +# reasoning depth for faster, cheaper iterations. +# +# claude-model claude-sonnet-4-6 +# claude-model claude-haiku-4-5 + +# ── Thinking budget (optional) ───────────────────────────────────────── +# Cap extended-thinking tokens per turn for the Claude CLI backend. +# Useful when running smaller/faster models (sonnet, haiku) and you'd +# rather they spend the iteration acting than reasoning. Sets the +# MAX_THINKING_TOKENS env var on the agent invocation. +# +# thinking off # disable extended thinking entirely (0 tokens) +# thinking low # 2k tokens +# thinking medium # 10k tokens +# thinking high # 32k tokens +# +# Ignored by the OpenCode backend (warns at config load). +# +# thinking low + # ── Sandbox ──────────────────────────────────────────────────────────── # Docker image to run the agent inside. Your working directory is # bind-mounted into the container at /workspace. Required unless you diff --git a/src/templates/saga/yoke.conf b/src/templates/saga/yoke.conf index eeb2913..144f7e9 100644 --- a/src/templates/saga/yoke.conf +++ b/src/templates/saga/yoke.conf @@ -38,6 +38,29 @@ # model openai/gpt-4o # model anthropic/claude-sonnet-4 +# ── Claude model (optional) ──────────────────────────────────────────── +# Override the model the Claude CLI uses for each iteration. Leave +# commented to use Claude Code's default (Opus). Useful for trading some +# reasoning depth for faster, cheaper iterations. +# +# claude-model claude-sonnet-4-6 +# claude-model claude-haiku-4-5 + +# ── Thinking budget (optional) ───────────────────────────────────────── +# Cap extended-thinking tokens per turn for the Claude CLI backend. +# Useful when running smaller/faster models (sonnet, haiku) and you'd +# rather they spend the iteration acting than reasoning. Sets the +# MAX_THINKING_TOKENS env var on the agent invocation. +# +# thinking off # disable extended thinking entirely (0 tokens) +# thinking low # 2k tokens +# thinking medium # 10k tokens +# thinking high # 32k tokens +# +# Ignored by the OpenCode backend (warns at config load). +# +# thinking low + # ── Sandbox ──────────────────────────────────────────────────────────── # Docker image to run the agent inside. Your working directory is # bind-mounted into the container at /workspace. Required unless you