Planning & Task Decomposition
Part of the PersonalClaw research-learnings library. Source-agnostic; distilled 2026-07-13 from a 95-source competitive-research corpus.
Principles (the durable truths)
Section titled “Principles (the durable truths)”Control-flow ownership is the fundamental planning decision. The three-tier taxonomy — task (a single model call: classify/extract/summarize; bounded failure modes, predictable cost), workflow (multiple model calls in a predefined control flow: “the control flow is yours”), agent (a model using tools in a loop, deciding its own trajectory: “the control flow belongs to the model”) — is the industry’s convergent vocabulary. Labels matter because architecture follows them: forcing every piece of automation to declare its tier makes cost, governance, and accountability questions answer themselves. Many “agents” in production are workflows wearing a costume; classifying them correctly recovers accuracy, control, and cost.
Don’t build agents for everything. If you can pre-map the decision tree, build it explicitly as a workflow and optimize each node. Agents explore, and exploration is expensive; errors compound — each loop iteration multiplies the failure rate of the weakest capability in the trajectory. Coding is the canonical agent-worthy domain because it passes all four gate questions (ambiguous, valuable, models capable, verifiable via tests/CI); verifiability is credited for the abundance of successful coding agents.
Process discipline beats model capability. A prose-workflow library with no execution engine at all measured its thesis as: the win comes from triage/validation/handoff checkpoints, not the model — and those checkpoints also “absorb silent base-model quality drift.” Structured planning survives model swaps; vibes do not.
Planning must never become waterfall. Spec-driven development degenerates into “writing a spec is a gate to further progress” unless the planner supports a fast lane: ~10 minutes writing an inferior spec, agent builds ~20 minutes, examine its assumptions, refine the spec against the built artifact, repeat. Start the agent “maybe even earlier than might feel comfortable” — early flaws are cheaper to see than to anticipate. Refinement against artifacts beats up-front guessing; at the exploratory stage, throwing away the entire output and restarting from an improved spec is a legitimate, cheap move.
Interrogation exists to extract the human’s context advantage. The human stays in the loop not for “taste” but because they know things the model cannot infer (users, constraints, boundaries). Every question the planner asks should target exactly that gap, and every answer should persist so the same question is never asked twice — the loop’s job is to transfer context and shrink required human involvement over time. Corollary: “AI tokens are cheap; human tokens are gold” — minimize human interventions per run, not model calls.
Autonomy is a mode, and trust is earned per-template, not granted per-run. Multiple independent systems converge on graduated autonomy (draft → report-only → assisted-with-verifier → unattended) with evidence-gated promotion. A first run of any new pattern defaults to report-only; promotion requires measured accuracy over real runs. Autonomy is also demotable mid-run when confidence drops.
Triage first, always. Every serious multi-step workflow in the corpus opens with a cheap classification that sizes the work and selects the entry point — before any expensive planning or execution. Multiple independent systems converge on this shape.
Mechanisms (implementation-ready designs)
Section titled “Mechanisms (implementation-ready designs)”The 4-question workflow-vs-agent checklist
Section titled “The 4-question workflow-vs-agent checklist”Run before choosing a plan shape:
- Ambiguity — can you pre-map the decision tree? If yes → fixed workflow with single-call nodes; optimize each node for accuracy, control, cost.
- Value vs token spend — agents explore; a 10¢/task budget buys roughly 30,000–50,000 tokens, which is workflow territory. Worked example: a 1M-ticket/month support operation overspending 5× on tokens burns ~$1.5M/year.
- Capability reliability — de-risk the bottleneck capability in the trajectory first; if gaps exist, “reduce scope and try again,” because per-step failure compounds per iteration.
- Cost of error × error discoverability — high-stakes, hard-to-detect mistakes make autonomy a liability; mitigations (read-only access, human-in-the-loop, limited scope) work but cap scale.
The checklist output is a control-flow tier (task / fixed workflow / agentic loop), orthogonal to interrogation depth, and should be declared in the plan artifact (“this will run as a fixed workflow / as an agent” + estimated token band). The five named workflow patterns to compile to: prompt chaining (sequence + gates), routing (classify → dispatch, incl. cheap-model tiers on easy branches), parallelization (sectioning and voting variants), orchestrator-workers (runtime decomposition), evaluator-optimizer (generator + critic loop).
Complexity triage before the LLM planner
Section titled “Complexity triage before the LLM planner”A tiny local classifier decides LOW (answer inline / single action) vs HIGH (invoke the planner) in milliseconds at ~zero token cost. Proven design: two classifiers voting (zero-shot NLI + an adaptive prototype classifier over a distilbert-class backbone), seeded with 150 few-shot labeled examples; confidences normalized relative to each other; short inputs (≤8 chars) short-circuit to “talk.” The critical inversion: low classifier confidence escalates to HIGH (≥85% routing accuracy on a fixture suite).confidence < 0.5 → planner), while classifier exceptions fall back to LOW. The adaptive classifier can learn from user corrections at runtime without retraining (prototype-memory weight 0.8 vs neural head 0.2, EWC lambda 100.0 against catastrophic forgetting, prototype update every 50 examples) — user overrides (“just answer, don’t plan”) become labeled examples. Extend the tuple to (complexity, uncertainty, stakes, time_pressure): complex + high-uncertainty auto-escalates interrogation depth; critical stakes bias toward approval-gated autonomy; the tuple doubles as the bucketing key for outcome learning. Caveat from the field: classifier routing misroutes; always keep an LLM fallback above any classifier fast-path, with a deployment bar (
Tiered template matching (deterministic-first)
Section titled “Tiered template matching (deterministic-first)”Never rely on a single embedding-similarity step with hard-coded thresholds. The working ladder:
- T1 — inverted keyword index over a
keywords[]field (a zero-cost deterministic keyword→method map is a proven fallback tier on its own). - T2 — metadata scoring over tags/name/description/scenario and — crucially — embedded example outputs, because user intents resemble desired outputs more than descriptions.
- T3 — intent-shape pre-classification constraining candidates by category.
- T4 — embedding as tie-breaker only.
- T5 — LLM summarize-then-rematch that re-enters the deterministic scorer — never accept an LLM-emitted template id directly (fuzzy-resolve + warn). When this tier is used it must return typed JSON
{primary, confidence, suggested_alternates}and degrade to the deterministic tiers on any parse/API failure, so matching never hard-fails offline.
Every decision attaches a human-readable reason string rendered at review. Template metadata must carry: when_to_use (trigger-only), when_not_to_use + “not for X — use Y” negatives, examples[], lighter_path (trivial intents route to a direct answer or single subagent instead of a full run), and named starter parameterizations offered as one-click instantiations. Router tie-breaker policy, adoptable verbatim: ask at most ONE clarifying question, and only if template choice would materially change actions; on low-risk ambiguity pick the likeliest and state the assumption in the plan; an explicitly user-named template wins unless clearly unsafe, with overrides explained. If top candidates score within ~0.15, compose 2–3 templates as subworkflows at penalized confidence rather than forcing an arbitrary winner; matcher confidence clamps below 0.95.
Grounded spec generation + repair-not-regenerate
Section titled “Grounded spec generation + repair-not-regenerate”From-scratch plan generation is the single most failure-prone planner step, and grounding fixes it measurably. The recipe:
- Ship an offline node-taxonomy + exact action/tool signature reference with the planner (index first, then drill only into relevant pages — “orient-then-drill”), because exact-signature lookups “cut down on errors from hallucinated values.”
- Emit under a full JSON-schema constraint where the model supports it, typed
oneOf[Spec, {cannot_plan: reason}]. - Run strict validation (unknown node types, unbound bindings, missing terminal node = hard errors) with up to N repair retries.
- On invalid output, re-prompt with a failure-mode-specific correction note — never regenerate from scratch.
- End with a mechanical pre-output self-check (unique ids, gates have approvers, every foreach has a binding, terminal node exists).
- Prefer classifying the intent into a registry of proven graph shapes and slot-filling, with freeform whole-graph generation only as explicit fallback.
Measured effect of grounding + strict validation on a niche DSL: 4/5 vs 0/5 first-try-valid, 6 vs 14 total attempts, 0 vs 3 silent spec misses, 92% vs 63% quality — at +58% tokens but slightly less wall time. Score “silent spec misses” separately from validation failures; treat first-try-valid rate as a planner health metric. Grounding preambles extend this: for entity-centric intents, a deterministic identity-resolution first node (“use exactly this resolved identity; do not substitute unless a tool result explicitly disproves it”) kills hallucinated-identity failures; for brownfield projects, a cached repo-context bundle (depth-2 file tree + docs head ≤8k chars + package metadata, keyed by tree-hash, 7d TTL) grounds generated stages in the actual language/test framework/layout.
Plan review as typed patches over a persistent artifact
Section titled “Plan review as typed patches over a persistent artifact”The plan is a file, not a chat transcript: persist generated plans as markdown artifacts in a known per-project location; revision happens as inline, line-anchored comments on that artifact; the approved artifact is exactly what the executor receives (vendor-productized at scale). Revision mechanics that make this cheap and safe:
- Merge-by-id patches — the revising LLM emits only changed steps (same-id replaces, new adds, absent preserved), so untouched steps’ parameterization can never drift (~60 vs ~400 tokens per revision). Insertion-only semantics (“Phase N.5” steps that never rewrite originals) give revise-comments clean merge semantics and provenance.
- NO_UPDATE sentinel fast-path — between-step revision must emit either the literal sentinel (no parse, near-zero cost) or a typed mutation set, never a free rewrite of the spec.
- Frozen past, mutable future — a working precedent runs plan revision after every executed step (planner sees goal + last output + success/failure verdict + next task) with hard instructions “do not change past tasks; change next tasks,” and the plan may grow mid-run (step count recomputed each iteration). Enforce the invariant structurally, not by prompt.
- Drafts are TTL’d sketches auto-GC’d if never approved; approval atomically promotes the draft to a run/template; revisions stage into the draft under optimistic concurrency; committed execution only reads the committed spec.
- Synchronized views — plain-English per-step proposal cards + read-only graph canvas + authoritative JSON, streaming progressively while the planner runs; planner-inferred parameters are flagged “inferred — confirm?” distinct from derived-from-user-words (the trust anchor for approving LLM-generated graphs); a small-model naming call supplies
{title, description, per-step labels}with deterministic fallbacks. - Mid-flight template/strategy switches carry prior step outputs into the new template’s entry node instead of restarting (“acknowledge the switch briefly; never clear history”).
When binding a prior step’s output into a later step’s prompt, prefix with attribution (“According to step X: …”) — weak models handle attributed context markedly better than bare concatenation.
Rigor modes and structured interrogation
Section titled “Rigor modes and structured interrogation”Interrogation depth is a planner axis with cheap ends, not just an escalation ladder. rigor:fast = the anti-waterfall mode: skip interrogation, start immediately, auto-schedule a spec-refinement gate after the first artifact; each user-observed defect appends to the acceptance criteria (append-only ratchet — stopping criteria only widen). rigor:deep = a spec’d protocol, not free-form questioning:
- Every question ships WITH the planner’s recommended answer — this is what makes deep grilling fast instead of tedious.
- Facts-vs-decisions split — discoverable facts get looked up (codebase, knowledge store, memory); only genuine decisions are asked.
- Pacing adapts: ≥3 independent load-bearing decisions → one batched structured round (≤8 typed question objects, 2–5 options each + mandatory “Other”); dependent questions fall back to one-per-turn (multiple interdependent questions at once “is bewildering”; one-question-per-turn interviews beat form dumps for deep intake — a proven 7-phase domain-intake card paces exactly this way).
- A stress-test phase after scoping: 2–3 adversarial scenario probes derived from stated constraints (“your highest-conviction position drops 70% in a week — walk me through hour by hour”) to surface gaps between stated philosophy and likely behaviour, feeding contradictions back into the plan BEFORE the spec is finalized.
- Output adopts the Step-0 schema — confirmed requirements / inferred assumptions / open questions, with “never treat a guess as a requirement” and open questions as blockers.
- Every round includes a boundary/never-do question whose answers persist as a frozen prohibitions block injected into every worker’s context — negative space (what the user will NOT accept) is elicited as a phase equal to goals, because it is the one thing genuinely uninferable.
- All Q+A pairs persist to the decision log so no question repeats.
- An explicit shared-understanding confirmation gates spec emission.
The 5W1H question taxonomy (Who/What/Where/When/Why/How, 2 questions per category, then answer-or-ask per question) is a ready-made grill structure. UI-wise, a stepper widget (1–5 agent-authored multiple-choice questions, gated forward navigation, per-question custom-answer escape hatch, single Submit returning a typed answer record) beats free-text back-and-forth for parameterization.
Stage contracts and the planner altitude rule
Section titled “Stage contracts and the planner altitude rule”Every generated stage carries a sprint contract: scope / machine-checkable “done means” (expression, command, or artifact check) / EXCLUSIONS (“out of scope this phase” + regression risks). Per-stage approval approves the contract; revision edits it; the stage’s judge cites exactly it. Spec validation rejects any workflow lacking a machine-checkable stopping condition (goal / verification / stopping-condition is the minimal triple); steps with no derivable check are flagged “unverifiable — needs approval gate or human check.” Measured effect: sprint contracts turned 3–4 blind retry cycles (~45 min) into 1 (~15 min); “writing the done condition first caught more scope drift than any prompt change.” The planner emits a preflight step (credentials, network, tool availability) before work stages — kills the plan-approved-run-dies-at-step-1 class. Altitude rule: the planner is bold in scope but constrains deliverables, not implementation — granular technical detail in plans cascades errors downstream. Phasing rule: vertical slices — every phase crosses all affected layers and ends in an executable verification gate; LLM planners default to horizontal layer-by-layer phasing that defers end-to-end feedback (reject/repair such plans). Condition-based stopping is the essence of the agentic loop (“keep working until it satisfies a condition”); specs, evals, and test sets are interchangeable in that role, and authoring them is a key place to inject human knowledge. See verification-and-judging for the gate/judge side of these contracts.
Autonomy as a mode-switch at a single approval gate
Section titled “Autonomy as a mode-switch at a single approval gate”Approval is one Implement dialog that co-selects three things simultaneously: go/no-go, autonomy/permission mode, and executor+model+run environment — with the planning agent explicitly allowed to differ from the implementing agent (vendor-validated that these choices belong at the same gate). A four-tier permission ladder (Plan → Ask → Edit → Full Access) normalized across heterogeneous executors, plus an orthogonal effort knob (Low/Medium/High/Max), with per-executor-per-workspace memory of the last chosen mode so dialogs default to the previous choice. Plan mode itself must be guaranteed at the orchestrator layer — synthesized even for executors whose native runtimes lack it (plan written to a file; planning phase enforced read-only via an injected policy flag), deferring to native plan modes only as an optimization. Autonomy modes derive from typed plan nodes: type each node HITL vs AFK at plan time; unattended runs still hard-stop at HITL-typed nodes. A distinct frame-only mode (“frame decisions, never make them”) lets analysis run autonomously while every decision-type gate presents framed options and stops — an autonomy ceiling users demonstrably want for high-stakes personal domains. Approval requests follow a rich spec: what action / why it matters / what could go wrong / what happens if approved / if denied / whether modify is available; verbs approve / deny / modify / defer / always-allow-narrow-scope / always-deny-narrow-scope; every decision logged as a learning signal. A canonical risk-signal registry (destructive ops, external writes, credentials/payments, schedule creation) forces deep rigor and caps offered autonomy on any hit; templates declare an autonomy floor neither planner nor user can silently lower; a conflicting user request surfaces exactly ONE informed-consent question — never silent honor, never silent upgrade.
Earned trust and report-only first runs
Section titled “Earned trust and report-only first runs”The graduated ladder, convergent across multiple independent systems:
- L0 Draft — documented intent only.
- L1 Report-only — triage/analysis writes state, no auto-action. Mandatory week one: “never skip L1 for a new pattern on a production repo.”
- L2 Assisted — small auto-fixes with a separate verifier, isolated workspace, max-attempt cap ~3.
- L3 Unattended — requires ALL of: denylist, budget, run log, human gates, and demonstrated activity (“not just files on disk”).
Promotion is evidence-gated: measure triage accuracy at L1 before enabling L2; prove L2 attempt limits and the verifier for ~two weeks before adding higher-risk loops. Trust is tracked per skill/domain/template, not globally (“good at testing is not automatically good at deploys”), with a shadow → recommend → draft-with-approval → bounded-autonomy ramp for high-risk domains, and approval gates placed before dispatch, not after execution. A computable readiness score (0–100 from ~18 static + dynamic signals: verifier present, budget block, escalation path, attempt caps, demonstrated run activity; CI-integratable, exit-fail below 40) gates which autonomy modes are even offered. Operational demotion thresholds: triage false-positive rate >30% or budget >80% mid-period → slow down; cost > value for 2 consecutive weeks → kill. Cost-of-error drives defaults: steps with executable verification default unattended; high-stakes + hard-to-verify steps default per-stage approval + read-only tool posture. Unattended mode must encode “auto-decide replaces judgment, not analysis”: options + choice + rationale recorded per auto-decided step, with lint flagging analysis compression. See self-improvement-loops for how approval decisions feed learning.
Triage-first entry with tiered entry points
Section titled “Triage-first entry with tiered entry points”Every workflow opens with a 2–3 tier classification (bug: Obvious/Moderate/Complex; feature: Small/Medium/Large; review: Quick/Standard/Deep …) that determines the entry point and which steps are skipped — the smallest tier routes to a self-contained lightweight prompt, skipping the heavy subgraph entirely. Pair with a standing escalate-and-reclassify rule: if unexpected risk/ambiguity surfaces mid-run, stop, reclassify upward, and splice the previously-skipped stages ahead of the current position (a typed plan mutation, not an abandon). Triage output also sizes the plan: scale stage count and agent/persona set to classified scale (Micro = 1–3 stages single-agent; Sprint skips discovery; Full = complete pipeline). For stimulus-driven planning (commit/event/file triggers), step zero classifies the stimulus by user impact and may emit a skip with a one-line recorded rationale (“Skip. Test file rename/assertion only; no runtime output.”) — cheap ledger-only records for skips, full runs only where impact exists. “When not to use” is first-class template documentation (one-line fixes, throwaway prototypes, pure docs edits route to lighter paths).
Blocking vs non-blocking decisions
Section titled “Blocking vs non-blocking decisions”The planner classifies every decision point at plan time into two severities: blocking needs-input (pauses the run, enters the needs-input inbox — genuine forks, destructive-action approvals, anything whose answer feeds a downstream binding) vs non-blocking open decisions (never pause; collected and presented at run end under a structured “Open Decisions” heading, answerable retroactively — ambiguity that doesn’t change the execution path). Default rule: decisions that don’t affect downstream stage inputs are non-blocking. A concrete 3-condition interrupt taxonomy for unattended mode: interrupt only for (1) irreversible/high-risk actions, (2) uninferable credentials or product decisions, (3) conflicting requirements — all other ambiguity proceeds with a journaled assumption. Answering an open decision post-run can trigger a scoped re-run from the affected node. A related first-class exit: any executing step may return REQUEST_CLARIFICATION and transition to needs-input immediately rather than burning retry attempts. Blocked reports follow a fixed shape: exact blocker / what was attempted / evidence gathered / the smallest human decision needed.
Brainstorming & ideation method chains
Section titled “Brainstorming & ideation method chains”Ideation methods are workflow topologies, not prompts — each is a small fixed graph (fan-out → per-item transform → optional deepen) with statically predictable cost. Six proven method chains: big mind mapping (10 diverse initial ideas → each expanded ×5 → again; 1+N+5N calls, N+5N+25N nodes — flag as slow/expensive); reverse brainstorming / premortem (two-phase invert-then-flip: “think like a saboteur — how would we cause this failure,” then flip each into 5 counter-solutions); role storming (5–10 fixed personas — Curious Child, Skeptical Analyst, Visionary Futurist… — each embodied “deeply” per idea); SCAMPER (7 transformation operators applied per idea, exactly one output each: Substitute/Combine/Adapt/Modify/Put-to-other-uses/Eliminate/Rearrange — doubles as a typed mutation-operator vocabulary for plan/template diffs); six thinking hats (one perspective per hat per idea with the orthogonality constraint “stay strictly within this hat’s perspective” — a parallel-judge-panel shape); starbursting (5W1H question generation, 2/category, then a separate answer chain per question — a formalized scoping interrogation). Cross-cutting design: methods as data ({id, name, description, whenToUse, examples[]} registry driving both UI and matching); every method reduced to exactly two prompt templates (initial fan-out with fixed count, expansion deepening one item ×5) so the executor is method-agnostic; strict bullet output contracts (”- ” prefix, fixed counts, continuation-line folding) as a robust low-tech alternative to JSON schemas for list-producing steps; results as an idea TREE ({id, content, level, parentId, children[], methodUsed} with parent/level invariants) — flattening the tree into transcript text is a shipped, regretted regression. Sessions cost ~$0.01–0.02, and fan-out cost is computable from topology before approval. Multi-variant exploration is a planning primitive, not an edge case: ask for N different designs and use the implementations themselves as decision-making feedback.
Plan representation and durability
Section titled “Plan representation and durability”A minimal working plan format: flat JSON {plan: [{agent/executor, id, task, need: [dependency ids]}]} with results stored per-id and dependency outputs injected with attribution; plan generation retries parse failures with a correction prompt (up to 4 attempts). Classify every plan as plan_mode: fixed | dynamic | rolling — template-matched plans are fixed (“do not let a standardized business workflow become creative”; mutation requires explicit unlock), scratch-generated are dynamic, long-horizon are rolling with scheduled re-plan checkpoints. Durable decisions live in a SPEC.md-style file that survives context compaction, with human-provided decisions given persistence priority over agent-derived ones (gold vs cheap tokens). Template-creation pipeline: mine completed chat sessions into parameterized templates (observed tools, approval decisions as priors); discover-then-freeze — every LLM-generated spec for an unknown domain persists as a candidate template so similar intents load it instead of re-generating (prevents plan drift across runs); scrub concrete entities into {placeholder} slots when generalizing. A planner-level distillation detector closes the loop: agentic runs with low trajectory variance across executions should be proposed for distillation into fixed templates; fixed steps that repeatedly fail should be proposed for promotion to agentic stages (see self-improvement-loops).
Patterns & compositions
Section titled “Patterns & compositions”- The canonical planning pipeline: cheap classifier triage (LOW→answer inline) → tiered template match (with lighter_path off-ramps) → rigor-appropriate interrogation → grounded spec generation with repair retries → plan-as-artifact review with typed patches → single approval gate stamping autonomy+executor+environment → execution with frozen-past/mutable-future revision. Each stage has a cheap exit; the expensive stages only run when earlier ones escalate.
- Three planning entry modes: template-match (parameterize a proven graph), pattern-fill (user supplies an abstract pattern document — named phases and decision points without parameterization — and a short collaborative dialogue concretizes it; better than pure match or unconstrained generation for semi-structured inputs), and from-scratch LLM generation (grounded, schema-constrained, last resort). A fourth ambient intake: a watched scratchpad file whose actionable lines become PROPOSED plans in an inbox — never auto-executed.
- Interview → build → self-maintaining bootstrap: a “domain OS” shape — phased one-question-per-turn intake, pre-persist stress tests, scaffold materialization (raw records → rolling summary → one-pager), then standing rules (mandatory pre-response reads, append-only dated memory, policy checks that name the violated rule). One artifact bootstraps a durable domain assistant; the whole viral prompt-card genre hand-rolls this shape.
- Three nested feedback loops at distinct timescales: agent-iterating (minutes; spec+evals as stopping condition), developer steering (tens of minutes–hours; updates the spec after seeing implementations), external feedback (days–weeks; alpha users, A/B). Each outer loop’s output parameterizes the next inner loop; the loop tiers are different queue disciplines, and minimizing human-token spend per run is the optimization target.
- Planner/implementer/reviewer split: the planning agent, implementing agent, and reviewing agent may all differ (fresh session, optionally different model for the reviewer — “a more independent second opinion”); reviewer output is line-anchored comments the human triages, with the accepted subset dispatched back as follow-up instructions. See multi-agent-orchestration and verification-and-judging.
- Divergent-before-convergent: insert an ideation fan-out (role-storm or mind-map) before option-generation stages so parallel options are drawn from an explored space rather than the model’s first two guesses; six-hats lens constraints turn the evaluation side into orthogonal parallel judges.
Anti-patterns & failure modes
Section titled “Anti-patterns & failure modes”- Spec approval as a waterfall gate. Planners that only escalate rigor upward make planning heavyweight for exactly the exploratory tasks where starting early is cheapest. The fix is a fast tier plus revise-spec-from-artifact, not more up-front questioning.
- Agents wearing workflow costumes. Pre-mappable decision trees run as agentic loops burn 5× tokens for lower reliability. Audit every “agent” against the 4-question checklist; instrument token economics at p50/p95 per template so the value question is answerable at plan time.
- Whole-spec regeneration on revision or validation failure. Free rewrites drift untouched steps’ parameterization and cost ~7× the tokens of merge-by-id patches; regeneration from scratch on invalid output discards what was right. Repair with failure-specific correction notes; patch by id.
- Embedding-only template matching with hard-coded thresholds. Unauditable, offline-hostile, prone to hallucinated picks, and guaranteed to over-instantiate heavyweight runs for typo-class tasks. Deterministic tiers + negatives + lighter paths + reason strings close all four holes.
- Free-form “vibes” interrogation. Unstructured clarification either under-asks (guesses become requirements) or over-asks (asks discoverable facts). The recommended-answer + facts-vs-decisions + Step-0 schema mechanics are load-bearing, not polish. Multiple interdependent questions in one message bewilder; form dumps lose depth.
- Prompt-only frozen-past enforcement. Instructing the model “do not change past tasks” works until it doesn’t; the invariant must be structural (revision attempts on completed nodes rejected by the engine).
- Run-global autonomy with no risk model. Nothing prevents “run unattended” on a plan that books flights or deletes files; a run-global switch either over-interrupts or lets HITL-requiring steps run silent. Node-level HITL/AFK typing + risk registry + template floors fix this with almost no machinery.
- Skipping report-only for new patterns. The field’s most repeated safety rule; violated, it produces autonomous side effects from untested triage. Also its cousin: promotion by vibes (“it seems fine”) instead of measured accuracy over a defined period.
- Wedging unattended runs on low-stakes questions. Without blocking/non-blocking classification, runs stall on ambiguity that never affected any downstream input; without an open-decisions surface, those questions are simply lost.
- Setup-only success mistaken for completion. A self-correction loop that exits on “execution succeeded” alone accepts runs whose last action merely staged setup; the working heuristic was “success AND the last tool wasn’t bash ⇒ done” — encode deliverable checks, not exit codes. Builder self-report is systematically overconfident; see verification-and-judging.
- Flattening hierarchical plan/idea output into transcript text. A shipped rewrite dropped its tree projection for a flat chat stream and lost the product’s core value (the tree types sat unused). Keep hierarchical structure first-class in projections.
- Silent honor or silent upgrade of risky autonomy requests. When a user asks for unattended on a risk-flagged plan, surface exactly one informed-consent question.
Quantitative findings
Section titled “Quantitative findings”- Grounded planner vs ungrounded on a niche DSL: 4/5 vs 0/5 first-try-valid, 6 vs 14 attempts, 0 vs 3 silent spec misses, 92% vs 63% quality, at +58% tokens but slightly less wall time.
- Agent-vs-workflow economics: 10¢/task ≈ 30,000–50,000 tokens; a 1M-ticket/month operation overspending 5× on tokens burns ~$1.5M/year; each agent step reasons over only ~10–20K tokens of working context.
- Merge-by-id plan revision: ~60 vs ~400 tokens per revision vs whole-spec re-emission.
- Stage done-contracts: 3–4 blind retry cycles (~45 min) → 1 cycle (~15 min) in a measured case; writing the done condition first caught more scope drift than any prompt change.
- Complexity triage: confidence < 0.5 escalates to the planner (escalate-on-uncertainty inversion); adaptive classifier: prototype weight 0.8 / neural 0.2, EWC λ=100, prototype update every 50 examples; ~150 few-shot seed examples; ≤8-char inputs short-circuit; target ≥85% routing accuracy before deployment.
- Earned trust: report-only is mandatory week one for any new pattern; ~2 weeks of proven assisted runs before higher-risk promotion; self-correction attempt caps of 3–5; plan-parse retry cap 4; readiness score CI-fails below 40/100; triage false-positive rate >30% or budget >80% mid-period triggers slow-down; cost > value for 2 consecutive weeks triggers kill.
- Matching/plan-review thresholds: compose subworkflows when top templates score within 0.15; clamp matcher confidence below 0.95; at most ONE clarifying question at match time.
- Interrogation: batched rounds cap at ≤8 typed questions; stepper widgets carry 1–5 questions × 2–5 options + custom escape hatch; stress-test phase = 2–3 adversarial probes.
- Ideation fan-out topology: mind-map = 1+N+5N calls, N+5N+25N nodes (3 levels); starbursting = 1+N+6N calls (6 questions/idea via 5W1H); SCAMPER = exactly 7 children/idea; full sessions ~$0.01–0.02.
- rigor:fast cadence: ~10-minute inferior spec → ~20-minute agent build → refine against the artifact; an early agent run self-verified via a real browser over ~1 hour before returning.
Open questions
Section titled “Open questions”- Budget-aware planning enforcement. Budgets in tokens, money, AND time enforced by the engine (warn ~80%, pause at cap → needs-input) are a named open problem; workflows have this control, agents lack it, and no researched system enforces it inside agentic loop nodes.
- Where does the workflow↔agent tier boundary move as models improve? The checklist’s capability question is time-dependent; the distillation detector (low-variance agentic runs → fixed templates) is a mechanism, but promotion criteria in the other direction are unvalidated.
- How much interrogation can be replaced by retrieval? The facts-vs-decisions split assumes a queryable context store; as memory/knowledge coverage grows (memory-architectures, knowledge-pipelines), the residual “genuine decisions” set should shrink — no system yet measures question-count decay per user over time.
- Trust-score portability. Per-template earned trust is proven; whether trust transfers across similar templates (same domain, different graph) or across executors is unexplored.
- Human-token accounting. “AI tokens cheap, human tokens gold” implies a per-run metric counting approvals/steers/defect reports alongside model tokens, trended per template — described as doctrine, implemented nowhere in the corpus.
- Optimal batched-vs-paced interrogation switching. The ≥3-independent-decisions heuristic for batching questions is plausible but unmeasured against completion quality or user fatigue.