stash: changes to thinking behavior

This commit is contained in:
Zachery Aaron Shores-Chmielewski 2026-07-23 14:39:31 +04:00
parent 77f5138151
commit fa65ddb605
7 changed files with 143 additions and 266 deletions

View file

@ -1,64 +0,0 @@
# Context trim — design notes
## What's shipped
Worker sessions are now resumed across iterations with `claude --resume <sid>`,
and the session JSONL is surgically trimmed between rounds so Anthropic's
prefix cache stays warm without paying for irrelevant history.
- Agent declares a `KEEP: <paths>` line in `.loop/notes.md`.
- Yoke locates `~/.claude/projects/<cwd-slug>/<sid>.jsonl` and rewrites it
to retain only: bootstrap records, the initial user prompt, `Read`
tool_use/tool_result pairs for kept paths, and attachments. Drops:
`thinking`, intermediate `text`, every non-`Read` tool_use, `Read`s of
non-kept files, and the matching `tool_result`s. Re-links the
parent-uuid chain across the gaps; validates tool_use ↔ tool_result
pairing before commit; keeps a `.bak`.
- Judge (brute) and scoper (saga) always run fresh — independence per
behavioral-specification §2.1.
- `YOKE_DISABLE_SESSION_TRIM=1` is the escape hatch.
## What still needs adding
1. **Format-drift guard.** The Claude Code session JSONL is undocumented.
A future CLI release could rename a field, change content-block shape,
or move the file. The validator catches most damage post-trim, but
pre-trim we should fingerprint the format (e.g., known top-level keys
on bootstrap records) and bail if it drifts. Today we trust + bail on
validate-fail; a positive check would be safer.
2. **Recovery from `--resume` failure.** If Claude rejects the resumed
session (deleted, corrupted, version skew), the iteration aborts. We
should detect this from the spawn's exit/early stream error and
transparently retry once with no `--resume` (treat last_session_id as
stale).
3. **Sandboxed runs.** When `image` is set, the agent runs inside a
Docker container — the session JSONL lives in the container's home,
not the host's. Today `trim_worker_session` no-ops in that case
(silent). Either mount the session dir into the container, or run the
trim inside the container, or document the limitation.
4. **OpenCode backend.** Trim is Claude-specific. OpenCode users get
`--resume` benefits skipped (different session model). If OpenCode
becomes a first-class target, we need an analogous trim or a
reasoned-down equivalent.
5. **Fork policy.** v1 has no forking: the trim alone bounds growth.
But sessions still grow monotonically in the *kept* portion, and
long-running tasks will eventually want a hard reset. A
`context-fork-every N` directive (or agent-declared `RESET: TRUE`)
would let users break the conversation cleanly at stage boundaries.
6. **Periodic-agent isolation.** Periodics spawn fresh sessions but
share the project's session directory. If a periodic ever needed its
own resumable continuity (e.g., a reviewer agent that learns over
runs), today there's no separate session-id tracking for it.
7. **Saga handoff.** Across saga cycles the scoper is fresh and reads
`saga-log.md` to reconstruct context. A future variant could let the
*brute worker inside saga* keep its session across cycles when the
scoper produces a closely-related sub-plan — but only if the scoper
signals it (otherwise context bleeds between unrelated chunks).
Next steps involve creating some test vectors to help make behavior standardized

View file

@ -1,200 +0,0 @@
# Yoke Behavioral Specification
This document defines what yoke promises to its users. Every statement here is
a testable invariant over observables — files, exit codes, process behavior.
No statement references internal functions, line numbers, or implementation
details. These invariants survive refactors and rewrites.
---
## 1. The Loop
**Story:** You write a plan, a protocol, and some guards. You run `yoke run`.
An agent executes your plan iteratively. Each iteration, it reads the protocol,
does work, and updates notes. Guards check the work. When the agent writes
`STATUS: DONE` and all guards pass, yoke exits.
### Invariants
**1.1 — The spec is immutable from the agent's perspective.**
`protocol.md`, `plan.md`, and `yoke.conf` are backed up before the loop and
restored before every iteration. The agent can overwrite them during its turn,
but those changes do not persist to the next iteration.
**1.2 — Termination requires both signals.**
The loop only exits when `STATUS: DONE` appears in `notes.md` AND all guards
pass. Neither condition alone is sufficient. If guards fail but status is DONE,
the loop continues with feedback. If guards pass but status is not DONE, the
loop continues.
**1.3 — Guard feedback is visible.**
`guard-results.md` is written after every iteration. The agent sees it on its
next turn. No guard result is silently swallowed.
**1.4 — Boundary violations block guards.**
If the diff boundary check fails, all configured guards are skipped (not run).
The agent gets boundary feedback only. Guards do not run on invalid state.
**1.5 — Interrupts are clean.**
SIGINT kills the child process immediately. The loop does not exit
mid-iteration leaving partial state — it completes the signal check and exits
at the next safe point with code 130.
---
## 2. The Judge
**Story:** In brute mode, after the worker says DONE and guards pass, a
separate fresh agent (the judge) runs. It reads `judge.md`, tests the feature,
and writes `VERDICT: PASS` or `VERDICT: FAIL` to `verdict.md`. On PASS, yoke
exits successfully. On FAIL, the worker retries.
### Invariants
**2.1 — The judge is independent.**
It is a fresh agent invocation with no shared context from the worker. Its only
input is `judge.md` and the codebase state.
**2.2 — Verdict survives retries and restarts.**
`verdict.md` is never cleared automatically. On judge FAIL, the worker sees
the judge's feedback on its next iteration. On bailout, the verdict remains
on disk so the user (or agent on restart) can read why the loop failed.
Use `yoke clean` or `yoke stash` to reset.
**2.3 — Guard results survive retries and restarts.**
Same as verdict — `guard-results.md` is never cleared automatically. It
persists across brute retries and restarts so the worker sees what the
guards reported.
**2.4 — Notes status reset on retry, nothing else.**
On judge FAIL, only the first line of `notes.md` is overwritten to
`STATUS: IN_PROGRESS`. The rest of the file — the agent's prior iteration
notes — is preserved. All other files remain as-is. The worker starts with a
clean status but full context from both its own notes and the judge's verdict.
**2.5 — Bailout is exact.**
If `max-judge-failures` consecutive judge FAILs occur, yoke exits non-zero.
The count is exact — `max-judge-failures 2` means bailout on the 2nd
consecutive FAIL, not the 3rd.
**2.6 — Judge-every overrides cadence on DONE.**
If `judge-every` is configured and the worker signals DONE, the judge fires
immediately regardless of whether the iteration is on the cadence boundary.
DONE always triggers judgment.
---
## 3. The Stash
**Story:** `yoke stash` saves the current `.loop/` state. `yoke stash pop`
restores the most recent snapshot. `yoke stash checkout <hash>` restores a
specific snapshot. `yoke clean` auto-stashes before wiping.
### Invariants
**3.1 — Stash clears the working directory.**
`stash` snapshots all non-dotfile files in `.loop/`, then removes them.
After stash, `.loop/` contains only dotfile entries (like `.stash/`).
`stash` then `pop` (or `checkout`) restores the original contents — no
file is lost, truncated, or corrupted.
**3.2 — Auto-stash before destructive operations.**
Both `clean` and `checkout` auto-stash current state before modifying it. You
can always recover what was there before.
**3.3 — Mode tag is recorded.**
Each stash entry records the mode (loop/brute/saga) that was active when it
was created. This tag is preserved in the index and survives restore
operations.
**3.4 — Index is append-only.**
Stash never modifies or deletes existing index lines. New entries are appended.
The index is a history, not a mutable pointer.
**3.5 — Prefix matching is unambiguous.**
`checkout abc` matches any entry starting with `abc`. If multiple entries
match, yoke errors instead of guessing. No silent wrong restore.
---
## 4. The Saga
**Story:** Saga mode has a scoper agent that reads `specification.md`,
decomposes it into chunks, writes each chunk to `sub-plan.md`, and a brute
loop implements and verifies each chunk. When all chunks are done, the scoper
writes `STATUS: DONE` to `saga-notes.md`.
### Invariants
**4.1 — Saga completion checks saga-notes, not notes.**
The saga loop checks `saga-notes.md` for DONE. `notes.md` is local to each
brute chunk and is cleared between chunks. Checking `notes.md` would be
checking the wrong file.
**4.2 — Brute bailout triggers re-scoping, not abort.**
If brute fails `max-judge-failures` times on a chunk, control returns to the
scoper. The scoper can re-scope the same chunk differently. The saga does not
abort on a single chunk failure.
**4.3 — Sub-plan must be non-empty.**
If the scoper produces an empty `sub-plan.md`, the saga aborts. This prevents
a brute loop from running with no plan.
**4.4 — Chunk state persists and is logged.** `notes.md`, `verdict.md`, and
`guard-results.md` are NOT cleared between chunks. The scoper can read why
the previous chunk failed or succeeded. Before each chunk, the contents of
`notes.md` are appended to `saga-log.md`. Use `yoke clean` or `yoke stash`
for a full reset.
**4.5 — Saga log is append-only.** `saga-log.md` accumulates the worker's
notes from every completed chunk. It is never cleared or truncated during a
saga run. Each entry is labeled with its chunk number.
---
## 5. Config
**Story:** `yoke.conf` defines the rules of the loop — what files are
protected, what guards run, how the judge behaves. It is parsed once at
startup and applied consistently throughout the run.
### Invariants
**5.1 — Valid configs parse.**
Every legal combination of directives parses without error.
**5.2 — Invalid configs fail loudly.**
Unknown directives, malformed values, and missing required fields produce clear
errors — not silent defaults.
**5.3 — Scope rules resolve most-specific-wins.**
If `allow src/` and `no-modify src/main.rs` are both configured, `src/main.rs`
is protected and `src/other.rs` is allowed. Longer prefix wins.
**5.4 — Guard-after requires its periodic.**
A `guard-after` referencing a periodic that does not exist is a config error,
not a silent no-op.
**5.5 — Hooks are fire-and-forget.**
A `hook` command that exits non-zero or fails to execute produces a warning on
stderr but does not affect the loop's exit code, guard evaluation, or iteration
flow. Hook output is never written to any file the agent reads.
---
## 6. Mode Switching
**Story:** You can switch between loop, brute, and saga without losing
progress. Each mode's state is snapshotted when you leave it and restored
when you return.
### Invariants
**6.1 — Mode switch stashes current state.**
Switching from mode A to B stashes all of A's files via `yoke stash`. The
stash entry is tagged with mode A. Current state is always recoverable.
**6.2 — Mode switch always fresh-inits.**
After stashing, the target mode is initialized with fresh template files.
Previous sessions are not auto-restored. Use `yoke stash checkout` to
restore a prior session.

View file

@ -7,6 +7,26 @@ pub enum Backend {
OpenCode,
}
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum Thinking {
Off,
Low,
Medium,
High,
}
impl Thinking {
/// MAX_THINKING_TOKENS value to forward to the Claude CLI.
pub fn max_tokens(self) -> u32 {
match self {
Thinking::Off => 0,
Thinking::Low => 2000,
Thinking::Medium => 10000,
Thinking::High => 32000,
}
}
}
#[derive(Debug, Clone, PartialEq)]
pub enum ScopeTag {
Allow,
@ -34,6 +54,8 @@ pub struct Config {
pub log_dir: Option<String>,
pub image: Option<String>,
pub model: Option<String>,
pub claude_model: Option<String>,
pub thinking: Option<Thinking>,
pub scope_rules: Vec<ScopeRule>,
pub guards: Vec<String>,
pub judge_every: Option<u32>,
@ -51,6 +73,8 @@ struct ConfigBuilder {
log_dir: Option<String>,
image: Option<String>,
model: Option<String>,
claude_model: Option<String>,
thinking: Option<Thinking>,
scope_rules: Vec<ScopeRule>,
guards: Vec<String>,
judge_every: Option<u32>,
@ -107,6 +131,8 @@ impl ConfigBuilder {
log_dir: None,
image: None,
model: None,
claude_model: None,
thinking: None,
scope_rules: Vec::new(),
guards: Vec::new(),
judge_every: None,
@ -129,6 +155,22 @@ impl ConfigBuilder {
"log-dir" => self.log_dir = Some(value.to_string()),
"image" => self.image = Some(value.to_string()),
"model" => self.model = Some(value.to_string()),
"claude-model" => self.claude_model = Some(value.to_string()),
"thinking" => {
self.thinking = Some(match value.trim() {
"off" => Thinking::Off,
"low" => Thinking::Low,
"medium" => Thinking::Medium,
"high" => Thinking::High,
other => {
return Err(cfg_err(
path,
line_num,
&format!("thinking must be 'off', 'low', 'medium', or 'high', got '{}'", other),
));
}
});
}
"allow" => self.scope_rules.push(ScopeRule { tag: ScopeTag::Allow, prefix: value.to_string() }),
"add-only" => self.scope_rules.push(ScopeRule { tag: ScopeTag::AddOnly, prefix: value.to_string() }),
"no-modify" => self.scope_rules.push(ScopeRule { tag: ScopeTag::NoModify, prefix: value.to_string() }),
@ -156,6 +198,19 @@ impl ConfigBuilder {
}
fn build(self, path: &Path) -> Result<Config, String> {
if self.model.is_some() && self.claude_model.is_some() {
return Err(cfg_err(
path,
0,
"'model' (OpenCode backend) and 'claude-model' (Claude CLI backend) are mutually exclusive",
));
}
if self.model.is_some() && self.thinking.is_some() {
eprintln!(
"warning: {}: 'thinking' directive is only honored by the Claude CLI backend; ignored when 'model' (OpenCode) is set",
path.display(),
);
}
let mut periodics = self.periodics;
for (pname, cmd, ln) in self.pending_guard_afters {
match periodics.iter_mut().find(|p| p.name == pname) {
@ -172,6 +227,8 @@ impl ConfigBuilder {
log_dir: self.log_dir,
image: self.image,
model: self.model,
claude_model: self.claude_model,
thinking: self.thinking,
scope_rules: self.scope_rules,
guards: self.guards,
judge_every: self.judge_every,

View file

@ -493,7 +493,7 @@ fn container_home(image: &str) -> String {
}
/// Build a docker command that runs the claude CLI inside a container.
fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command {
fn build_docker_claude_command(image: &str, claude_args: &[&str], extra_env: &[(&str, String)]) -> Command {
let workdir = std::env::current_dir()
.unwrap_or_else(|_| PathBuf::from("."))
.to_string_lossy()
@ -530,6 +530,10 @@ fn build_docker_claude_command(image: &str, claude_args: &[&str]) -> Command {
}
}
for (key, val) in extra_env {
c.arg("-e").arg(format!("{}={}", key, val));
}
if let Some(home) = std::env::var_os("HOME") {
let home_path = PathBuf::from(&home);
let claude_dir = home_path.join(".claude");
@ -570,17 +574,28 @@ fn build_command(config: &Config, prompt: &str, resume: Option<&str>) -> Command
"--include-partial-messages",
"--dangerously-skip-permissions",
];
if let Some(m) = config.claude_model.as_deref() {
claude_args.push("--model");
claude_args.push(m);
}
if let Some(sid) = resume {
claude_args.push("--resume");
claude_args.push(sid);
}
claude_args.push("-p");
claude_args.push(prompt);
let extra_env: Vec<(&str, String)> = match config.thinking {
Some(t) => vec![("MAX_THINKING_TOKENS", t.max_tokens().to_string())],
None => Vec::new(),
};
match config.image {
Some(ref image) => build_docker_claude_command(image, &claude_args),
Some(ref image) => build_docker_claude_command(image, &claude_args, &extra_env),
None => {
let mut c = Command::new("claude");
c.args(&claude_args);
for (key, val) in &extra_env {
c.env(key, val);
}
c
}
}

View file

@ -33,6 +33,29 @@
# model openai/gpt-4o
# model anthropic/claude-sonnet-4
# ── Claude model (optional) ────────────────────────────────────────────
# Override the model the Claude CLI uses for each iteration. Leave
# commented to use Claude Code's default (Opus). Useful for trading some
# reasoning depth for faster, cheaper iterations.
#
# claude-model claude-sonnet-4-6
# claude-model claude-haiku-4-5
# ── Thinking budget (optional) ─────────────────────────────────────────
# Cap extended-thinking tokens per turn for the Claude CLI backend.
# Useful when running smaller/faster models (sonnet, haiku) and you'd
# rather they spend the iteration acting than reasoning. Sets the
# MAX_THINKING_TOKENS env var on the agent invocation.
#
# thinking off # disable extended thinking entirely (0 tokens)
# thinking low # 2k tokens
# thinking medium # 10k tokens
# thinking high # 32k tokens
#
# Ignored by the OpenCode backend (warns at config load).
#
# thinking low
# ── Sandbox ────────────────────────────────────────────────────────────
# Docker image to run the agent inside. Your working directory is
# bind-mounted into the container at /workspace. Required unless you

View file

@ -24,6 +24,29 @@
# model openai/gpt-4o
# model anthropic/claude-sonnet-4
# ── Claude model (optional) ────────────────────────────────────────────
# Override the model the Claude CLI uses for each iteration. Leave
# commented to use Claude Code's default (Opus). Useful for trading some
# reasoning depth for faster, cheaper iterations.
#
# claude-model claude-sonnet-4-6
# claude-model claude-haiku-4-5
# ── Thinking budget (optional) ─────────────────────────────────────────
# Cap extended-thinking tokens per turn for the Claude CLI backend.
# Useful when running smaller/faster models (sonnet, haiku) and you'd
# rather they spend the iteration acting than reasoning. Sets the
# MAX_THINKING_TOKENS env var on the agent invocation.
#
# thinking off # disable extended thinking entirely (0 tokens)
# thinking low # 2k tokens
# thinking medium # 10k tokens
# thinking high # 32k tokens
#
# Ignored by the OpenCode backend (warns at config load).
#
# thinking low
# ── Sandbox ────────────────────────────────────────────────────────────
# Docker image to run the agent inside. Your working directory is
# bind-mounted into the container at /workspace. Required unless you

View file

@ -38,6 +38,29 @@
# model openai/gpt-4o
# model anthropic/claude-sonnet-4
# ── Claude model (optional) ────────────────────────────────────────────
# Override the model the Claude CLI uses for each iteration. Leave
# commented to use Claude Code's default (Opus). Useful for trading some
# reasoning depth for faster, cheaper iterations.
#
# claude-model claude-sonnet-4-6
# claude-model claude-haiku-4-5
# ── Thinking budget (optional) ─────────────────────────────────────────
# Cap extended-thinking tokens per turn for the Claude CLI backend.
# Useful when running smaller/faster models (sonnet, haiku) and you'd
# rather they spend the iteration acting than reasoning. Sets the
# MAX_THINKING_TOKENS env var on the agent invocation.
#
# thinking off # disable extended thinking entirely (0 tokens)
# thinking low # 2k tokens
# thinking medium # 10k tokens
# thinking high # 32k tokens
#
# Ignored by the OpenCode backend (warns at config load).
#
# thinking low
# ── Sandbox ────────────────────────────────────────────────────────────
# Docker image to run the agent inside. Your working directory is
# bind-mounted into the container at /workspace. Required unless you