Memory Architectures

Part of the PersonalClaw research-learnings library. Source-agnostic; distilled 2026-07-13 from a 95-source competitive-research corpus.

Principles (the durable truths)

Injected memory needs declared authority, or the agent re-derives everything. A system with perfect per-turn injection of memory blocks still exhibited “memory-zero behavior” — the agent re-ran searches, re-curled its own vector store, and probed fact stores for answers already in its prompt — because injected memory “was not listed at all” in its source-of-truth hierarchy. The fix is a short conflict-hierarchy doctrine (terminal/tool output → injected memory → official docs → training priors) with explicit rules (“when injected memory contradicts your assumptions, injected memory wins”; “never treat a question as novel when the answer is already in your prompt”). Doctrine alone decays under time pressure, so pair it with a visible pre-action protocol or a measured adherence probe. The structural version of the same fix: inject memory harness-side (e.g. a proxy that prepends a recalled block to every request) rather than depending on the model choosing to call a recall tool — tool-based recall is unreliable by observation in multiple independent systems.

Relevance ranks; strength prunes. Decay/strength must be deliberately EXCLUDED from the retrieval ranking formula, because similarity × strength makes old-but-still-valid memories lose to newer irrelevant ones. Ranking should be relevance-dominant (lexical + vector, recency strictly secondary and capped); decay governs only eviction, archival, and curator review. This is the opposite of the naive design and is supported by ablations in the system that adopted it.

Typed, tiered memory beats one embedding store. Multiple independent systems converge on a working-→episodic→semantic→procedural ladder (one adds Resource and Knowledge-Vault types; another uses 5 sectors; another 7 layers). The types differ in schema, decay speed, retrieval pattern, and security posture — “how-to” steps, file catalogs, and credentials are three distinct retrieval surfaces, not tags on one row. Each tier should have a defined behavior when no LLM is available (one system’s pipeline gracefully stops at the episodic tier on a synthetic compression path and still scores 86.2% R@5 lexical-only).

Reinforce, don’t duplicate; supersede, don’t delete. A duplicate save should increment reinforcements and raise confidence instead of creating a row; unreinforced entries decay. Contradictions mark the old entry isLatest: false (versioning) or archive-with-back-ref rather than deleting — provenance stays intact and rollback is trivial. Contradiction handling keeps BOTH claims with citations plus a source-precedence ladder (user’s direct statements → compiled synthesis → raw evidence → external sources) — never silently pick one. Multiple independent systems converge on all three rules.

Deterministic first, LLM second. The highest-leverage memory mechanisms in the corpus are zero-LLM: write-time regex/heuristic graph extraction (which carried a +31.4-point P@5 lift, more than any embedding/reranker change), an embeddings-plus-NLP resolve cascade at store time, dependency-parse contradiction detectors, content-hash dedup. LLMs are reserved for background compression, synthesis, and judgment — on a cheap model tier, because for 24/7 background work “cost and speed matter more than raw intelligence.”

Every score must name its consumer. One system scored importance 0.0–1.0 at every ingest and then no code path ever filtered, sorted, or thresholded on it — dead metadata plus wasted extraction tokens. Any field written by extraction must be read by surfacing, curation, or retention, or be removed.

Capture must be non-blocking, hygienic, and record its no-ops. Lifecycle hooks do I/O only (parse transcript, write staging file, spawn detached worker); intelligence runs off-path. Capture applies quality gates (grounding evidence, notability, minimum substance) and explicitly logs “nothing worth saving” as a countable outcome — because fail-open recall/capture paths look identical to no-results, and one system’s vector-store injection was silently dead for ~9 days behind an except: return [].

Agent writes land in a transient zone by default. Propose-don’t-write enforced structurally: the agent’s save tool defaults to an inbox/staging folder that is not indexed as canon; promotion to canonical memory requires explicit governance. Least privilege by tool-set construction (the ingest agent physically cannot read; the query agent physically cannot write) beats policy text.

Mechanisms (implementation-ready designs)

M1. The tiered consolidation ladder

Four tiers with session-end consolidation: Working (raw + compressed observations, rolling window) → Episodic (session summaries: title, narrative, keyDecisions, filesModified, concepts) → Semantic (facts with confidence 0–1, accessCount, strength) → Procedural (named workflows with ordered steps, trigger condition, frequency count). Observations arrive via lifecycle hooks (SessionStart, UserPromptSubmit, Pre/PostToolUse, PreCompact, Stop, SessionEnd), pass a SHA-256 dedup window (~5 min) and a privacy filter, and are typed (15 observation types; decision and discovery are high-importance types that feed directly into semantic memory). Consolidation runs at session end when an LLM key exists; without one the pipeline stops cleanly at episodic. Richer variant: six typed components — Core (persona/human profile blocks), Episodic, Semantic, Procedural, Resource (file/document catalog), Knowledge Vault (sensitive static reference data) — each with its own schema, manager, and extraction prompt, classified in fixed priority order (Core > Episodic > Semantic > Procedural > Resource > Vault) with the bias “when uncertain, include additional types; specialists do final filtering.”

M2. Two-phase Extract→Decide formation

Never write extracted facts directly. Phase 1: LLM extracts candidate facts from the conversation. Phase 2: search existing memory for candidates related to each fact, then a second LLM call decides per fact: ADD / UPDATE / DELETE / NOOP, and only then apply. The candidate-gathering step between the phases is what prevents duplicate accumulation and drift. A fully deterministic variant runs at store time with zero LLM calls, in this exact order: (a) nearest-neighbor cosine < 0.85 → new; (b) subject guard — embed the first ~2 words of each sentence as a subject proxy; cosine < 0.60 → different entities → new (prevents “A uses X” merging with “B uses X”); (c) contradiction check BEFORE the reinforce check, because “dislike JavaScript” vs “love JavaScript” hits sim ≈ 0.92 but must replace; (d) sim ≥ 0.92 → reinforce; (e) else merge by entity-append. Related: topic-extract from the incoming content FIRST and use the topics as recall queries so extraction sees what is already known (“understand, then check what I know”) — also lets enrichment searches be sent existing knowledge so they return only the DELTA.

M3. Decay, heat, and promotion math

The corpus converges on importance-modulated exponential decay with semantic per-kind half-lives:

strength = clamp(importance × e^(−effective_λ × active_days) × (1 + recall_count × 0.2), 0, 1) with effective_λ = base_λ × (1 − importance × 0.8).
Per-kind half-lives carry meaning: failure ~11d (environments change; stale failures mislead) < assumption ~19d < fact ~24d < strategy ~38d. A sector variant uses multipliers over one base λ=0.02/day: episodic 1.5×, emotional 1.2×, reflective 0.8×, procedural 0.7×, semantic 0.5×. A simpler tiering: half-life 90d if importance ≥ 0.3 else 30d, archive below decay 0.1.
Active-days clock: count only days the user actually used the system (idempotent daily activity row; distinct-day count), so an idle vacation doesn’t decay valid memories.
Access-modified decay: effective_decay = base_rate^(1/(1 + 0.1 × access_count)) — more access, slower decay.
Reinforcement: +0.3 on access, halved to +0.15 if reinforced within the last hour (diminishing returns), capped at 1.0, every boost journaled (before/after strength, boost, type). Bump recall counts once per SESSION (in-process hit set flushed on >30 min idle), not per query — otherwise one chatty session inflates heat.
Hebbian recall propagation: a direct hit at sim ≥ 0.75 boosts its depth-1 graph neighbors (+0.2 node strength) — recalling a memory keeps its neighborhood alive.
Pruning needs TWO signals: strength < 0.2 AND importance < 0.3 (importance is a decay-immune override). Chain-aware sparing: before evicting, check depth-1 neighbors; one strong linked neighbor → keep alive. Review escape-hatch: decayed-but-high-confidence (≥ 0.7) items are routed to human review instead of silently archived — a third curator verdict beyond keep/archive. Exemptions: human-sourced and procedural entries never auto-archive. “Archive” is a soft payload flag; nothing is deleted or moved.

M4. Typed entity graphs and edge vocabularies

Deterministic write-time linking (zero LLM): on every write, regexes (markdown links, wikilinks, typed-link blockquotes) plus a fixed heuristic type cascade (FOUNDED → INVESTED → ADVISES → WORKS_AT → MENTIONS) create edges (from, to, link_type, context, source); new entity mentions create stubs; import is idempotent via SHA-256 content hash; a 17K-page full extract completes in seconds and costs $0 in tokens. Ablation: this graph layer alone contributed +31.4 points P@5 — more than hybrid search itself. Enforce the “unlinked mention is a broken brain” rule with an orphans lint (zero inbound links → link or flag, never auto-delete). Alternative deterministic edge builders: SVO dependency triples with verb-weighted edges (cause 0.8 > use/prefer/build 0.7 > like/hate/know 0.6 > have 0.5; negated predicates prefixed not_; edge_weight = sim × verb_weight), plus entity-bridge edges at fixed weight 0.55, calibrated so a bridge hit is competitive with the weakest direct hit; +12pp on multi-hop retrieval. LLM-extracted variant: 13 node types / 14 edge types including rejected and succeeded_by. Richest variant: a reified temporal graph where statements (SPO triples) are first-class NODES carrying validAt/invalidAt, one of 12 aspect labels, and provenance chains to source episodes — contradiction detection becomes a graph query (same subject+predicate, different object). Sparse-graph discipline: each new memory links only to its 1–3 most similar neighbors; link weight = 0.6×embedding similarity + 0.25×metadata overlap (keyword/tag Jaccard) + 0.1×temporal proximity + 0.05×salience. Cheapest rung: no edge table at all — append {linked_to, relationship} JSON onto BOTH endpoint rows, so every read carries its neighborhood inline.

M5. Graph-walk recall that never displaces direct hits

Round 1: vector/hybrid retrieval. Round 2: BFS (depth 2) from round-1 seeds, strictly additive — graph hits scored W_vector × min(edge_weight, 0.74) so they can compete for top-k but can never trigger further recall propagation; graph-expanded hits get their own injection floor (≥ 0.30) and cap (3), because unconditionally injected graph neighbors “dragged in low-relevance noise from co-occurrence edges.” Principled second-pass variant: top-5 pass-1 hits seed Personalized PageRank over typed edges (requires 1.0, related 0.7, wikilink 0.5, tag co-occurrence 0.1–0.3, strongest-edge-per-pair only) with three guard rails — seeds get zero self-boost, dangling seed mass redirects through the teleport distribution, non-seeds normalize by the best non-seed. Fusion of arms: RRF with k=60 (score = Σ 1/(60 + rank_i); weights need not sum to 1), per-stream scores exposed for debugging, and diversification before budget-trim (max ~3 results per source session so one verbose episode can’t monopolize the budget). Intent-adaptive weight profiles (lexically detected debug/brainstorm/default) modulate the signal mix — debug weights lexical precision + graph connectivity, ideation weights importance + recency. See knowledge-pipelines for the document-side retrieval stack.

M6. Voice-vs-world dual storage

Split memory on WHOSE truth it is: Voice aspects — the user’s directives, preferences, habits, beliefs, goals — are stored WHOLE (non-decomposed statements) in their own store/namespace; World/graph aspects — identity, events, relationships — decompose into SPO triples in the graph. The two answer different queries (“how does the user think?” vs “what is factually true?”) with different retrieval strategies, and decomposing a preference destroys its meaning. A sibling design adds a holder-attribution layer: every claim carries a holder (world | system | person:<id>) and a weight in 0.05 increments (no false precision), with caps — retweet-only amplification ≤ 0.55, self-reported facts 0.75 not world-consensus — plus a “so what” test (a claim must be load-bearing for some future query). Prerequisite for multi-person memory without belief laundering.

M7. Self-model with reinforcement promotion

A compact private diary the system maintains about ITSELF: after each turn it observes route taken, tools run, success/failure; it keeps a bounded self-model — narrative story, behavioral principles (max 6, promoted only after seenCount ≥ 2 AND confidence ≥ 0.72), current focus (4), working theories (4), initiative candidates (5), retrospections (6), recent turns (10). Periodic LLM reflection produces a patch. Only a compact snapshot is injected into planning/recovery prompts — never the full diary. Initiative candidates feed the proactive/pulse layer (see automation-and-triggers). Promotion-by-repeated-reinforcement above a confidence threshold yields emergent personalization without explicit configuration; the bounded caps prevent the self-model from becoming another unbounded store.

M8. Memory slots — bounded, editable registers

A distinct primitive from searchable memory: N named, size-capped, always-injected, user-editable registers. Proven set of 8: persona, user_preferences, tool_guidelines, project_context, guidance, pending_items, session_patterns, self_notes; sizeLimit default 2,000 chars (hard cap 20,000), project/global scope, readOnly flag, and appends FAIL LOUDLY over the limit (bounded by construction). A reflection hook appends TODOs into pending_items and patterns into session_patterns. Sibling designs: character-limited core-memory blocks where an overflowing write returns a descriptive error (not an exception) so the agent condenses and retries; a curated always-injected file pair (~800-token agent notes + user profile) injected as a frozen snapshot per session — never mutated mid-session, explicitly to preserve the LLM prefix cache, while writes persist to disk immediately; an index file hard-bounded at 200 lines / 25,000 bytes with truncation diagnostics, holding one-line pointers (“an index, not a memory body”).

M9. Readable, editable memory vaults

Memory as plain markdown files with YAML frontmatter, git (or a folder) as system of record, database as rebuildable index — “the model only remembers what gets saved to disk; there is no hidden state,” and you can “git diff what your agent learned overnight.” Key file conventions that recur independently:

Compiled truth + append-only timeline per entity page: synthesized current understanding on top, dated evidence entries below; consolidation rewrites ONLY the compiled section, evidence is immutable — solves “memory either goes stale or grows unboundedly.”
L0/L1/L2 progressive disclosure in one file: an l0 one-liner in frontmatter (always available as catalog), L1 operational summary (50–150 words), L2 full detail split by body markers. The retrieval engine loads a TIER, not a memory: L2 threshold = max(top_score×0.9, 0.3), L1 = max(top_score×0.65, 0.15), at most 3 items at L2 — thresholds relative to the best hit so a weak query never earns a detail dump.
Usage metadata in a sidecar (usage.json), never in the content files — heat tracking must not churn git history or human edits.
Index-guided retrieval, no RAG: maintain a one-line-summary index table; the LLM reads the index to pick 3–10 articles and synthesizes with wikilink citations. At 50–500 articles this beats cosine similarity (“embeddings match words, the LLM matches concepts”); switch to hybrid retrieval as a pre-filter around ~2,000 articles (~2M tokens).
Tolerant parsing: unknown enum variants + defaults everywhere; never reject a memory file for one bad field — a strict frontmatter enum once silently dropped 23 of 25 memories from an index.
Frontmatter dedup signature: SHA-256 over normalized content|type|category; a duplicate write REFRESHES the existing file. Soft-delete only (disabled: true); supersedes lists for lineage. Memories >1 day old get an injected staleness note (“point-in-time observations” needing verification).

M10. Consolidation cadences (sleep-like)

Multiple independent systems converge on an idle-time “dream” pass; the load-bearing controls are the same everywhere:

Queue mechanics: the simplest correct shape is a consolidated=0 flag-cursor — batch-select WHERE consolidated=0 LIMIT 10, flip on success; skip the run entirely (zero LLM spend) unless ≥2 items pend (cheap SQL precondition); the pending count doubles as a health metric. Idempotency keyed (file_path, content_hash); cooldown_hours: 12 with the timestamp written ONLY on successful runs.
Event-driven variant — consolidate around the write: after each store, retrieve neighbors of the NEW memory; if ≥5 cluster above ~0.62 sim, LLM-summarize that neighborhood (“preserve EVERY distinct detail — names, numbers, dates, decisions”; 1–3 sentences), insert the summary at the cluster’s max importance, archive originals with a summary_id back-ref (reversible), leave the cluster untouched on LLM failure. Cost scales with writes, not store size. Measured: 444 memories → 16 summaries.
Phased dream cycle: fixed-order phases (lint → backlinks → sync → synthesize → extract → patterns → embed → orphans); a cheap-model verdict filters routine transcripts before fanning out one stronger-model subagent per worthwhile transcript; a patterns phase writes a pattern page only at ≥3 supporting reflections (min_evidence: 3). A three-phase variant separates authority: light (stage/dedupe, no writes) → REM (theme summaries, reinforcement signals) → deep (the ONLY phase that writes the canonical file), with promotion requiring ALL gates (minScore, minRecallCount, minUniqueQueries) over a 6-signal ranking (relevance .30, frequency .24, query diversity .15, recency .15, consolidation .10, richness .06).
Non-destructive compression: consolidation reduces original strength ×0.5 (or archives with back-refs) rather than deleting; the summary links to its originals. Insights carry parent_ids lineage and parents carry reflection_count with eligibility < 3 — without lineage caps, synthesis re-synthesizes its own output. Exactly ONE insight per batch keeps the derived layer reviewable, and consolidation output must be FED BACK into query context (two-layer retrieval: raw memories + accumulated insights), not stored write-only.
Containment: autonomous consolidation runs under a write-scope allow-list (path prefixes it may touch), consolidation job names are unsubmittable by remote/untrusted callers, and the run takes a backup + mtime-diff + rollback lock. “Even on prompt-injection success, the subagent cannot write outside that list.” See security-and-guardrails.
Two-stage extract→compile with cost asymmetry: per-session flush is cheap and dumb (no tools, 2 turns, append to an immutable daily log; $0.02–0.05); compilation is expensive and agentic (full corpus in context, writes articles directly; $0.45–0.65/log), scheduled cron-lessly (after local hour ≥ 18, if today’s log hash differs from last compiled — activity-piggybacked + hash-idempotent).

M11. Screen-capture-to-memory

Continuous observation pipeline: capture screenshots at ~1 fps → discard idle frames (change detection) → accumulate up to 20 per batch → compress via subprocess (avoids GIL contention in the async loop; max 1920×1080 q85) → dedup via a cloud-file mapping table → batch-dispatch as one multi-modal LLM call → topic-extract first and use topics as recall queries against existing memory (prevents duplicate/contradictory writes) → fan out the SAME batch to per-type extraction agents, each with a different lens prompt (episodic asks “what happened when?”, procedural asks “what steps were followed?”, semantic asks “what concepts?”). Unprocessed context sits in a raw staging store with a 14-day TTL enforced by nightly cleanup. Context-window discipline matters here: warn at 75% of window, summarize down to 10% pressure, keep last 5 messages. Same fan-out-different-lens composition applies to any rich ingest (meeting transcripts, long documents).

M12. Contradiction and supersession

Three deterministic detectors cheap enough to run on every write (zero LLM): polarity flip (curated positive/negative verb sets + dependency-parse negation children), negation flip (verb→negated map over root/clause verbs plus whole-sentence negation asymmetry when ≥2 content lemmas are shared), number conflict (3–4-digit regex; differing sets flag only if ≥4 non-stop words shared). Escalate to an LLM only on detector hits. Graph-native detection: statements with same subject+predicate but different objects (or same subject+object, different predicates). Resolution policy: contradiction → replace proposal (or version supersession isLatest:false), never silent overwrite; where truth is ambiguous, keep both claims with citations and apply the source-precedence ladder (user statement > compiled truth > timeline evidence > external). A batched “micro-reflection” variant adjusts confidence in place by contradiction severity (−0.05/−0.1/−0.2; +0.05 if consistent, clamped [0,1]) with an audit note, under the doctrine “consolidate existing data, never generate new knowledge,” rate-limited by a persistent budget table (max 5 runs/hour).

M13. Injection, budget, and compaction survival

Single salience pool, one budget: all recall candidates from every source enter ONE pool: salience = (0.55×query_overlap + 0.45×base_score) × 0.85^rank × source_prior, priors deliberately near 1.0 (facts 1.10, curated 1.05, sessions/vector 1.00 — query relevance should dominate, not source identity); cross-source corroboration boost min(corroborations × 0.15 × base, 0.50); relative prune at 35% of max salience; near-dup suppression at 0.82 containment; hard budget of ~6 items across ALL sources. Degrade an item to its L1/L0 tier before dropping it (M9).
Gating calibration: a global repetition gate at 0.6 overlap suppressed ALL injection from turn 2 in single-topic sessions — 0.85 (near-literal repetition only) is the working value; social-closer messages (“ok”, “thanks”) skip retrieval but zero-cost local sources stay exempt; short queries must NOT be length-gated (“fix the BM25” benefits most from injection); stable facts inject first-turn-only.
Make the budget cut visible: return/append a compact L0 catalog of near-miss items “you can request later” — converts a hard budget into progressive disclosure. Log every not-loaded candidate with score + reason; a nudge_threshold detector over near-misses (scored close but never surfaced in ≥3 of last 10 requests) proposes importance bumps. Position by attention shape: rules at the absolute top, scored content in the middle, the catalog/index last.
Compaction survival: long sessions compact multiple times and intermediate context dies before session end. Two nets: a PreCompact capture hook (byte-identical to session-end capture, with a 60s same-session dedup window against double-fire), and a SessionStart re-injection hook with source-differentiated labels (compact / resume / startup / clear each worded differently) that fetches recent N≈12 memories via plain SQL (no embedding-model load; fail-silent).
Retrieval feedback contract: facts carry trust_score (Bayesian prior 0.50) + retrieval_count + helpful_count; derive “helpful” MECHANICALLY (entity cited/used in the turn) rather than asking the model to call a feedback tool — voluntary feedback contracts fail silently and every score stays 0.50 forever. The minimal measurement loop: log every volunteering event with per-arm confidence (alias 0.9 / exact title 0.8 / fuzzy 0.6, +0.05 recency bonus, gate at 0.7, cap 3), count “used = retrieved-after-volunteered,” tune thresholds from per-arm precision.

M14. Capture hygiene and pipeline ergonomics

Quality gates proven to keep junk out: a grounding filter (durable capture only if BOTH a past-tense decision regex AND an outcome-evidence regex match, AND response >200 chars AND request >50 chars); session scoring (weighted depth×2 + decision×3 + recall-usage×2 + links×2 + engagement×1; sessions < 0.2 never captured); a notability gate before creating entity pages (will-you-interact-again / does-it-matter — “when in doubt, DON’T create: a junk page degrades search; a missing page can always be added later”); a system-injection filter (cron/orchestrator preambles like [SYSTEM: excluded — scaffolding must not become memories); skip meta-observations about the session itself; capture the user’s original phrasing VERBATIM (“the user’s language IS the insight”); prefer capturing large tool RESULTS (≥800 chars) over model prose — tool outputs carry the work detail. Pipeline shape: per-session cursor with optimistic advance (advance BEFORE storage succeeds — duplicates are worse than loss), exchanges written to a pending file, a detached worker posts them (inline fallback if spawn fails), a recursion sentinel env var so the extraction session’s own lifecycle hooks don’t re-fire capture (documented infinite-flush and Stop-hook-recursion incidents), and an audit trail: deletes require a reason and every read/write/delete can ride a ~20-line hash-chained ledger (row_hash = sha256(prev | fields), genesis anchor, verify-walk, ids-only so the audit log is not itself a leak vector, prune as the only sanctioned deletion path with a 90-day floor).

M15. Degradation ladders and backfill discipline

Every retrieval path needs a defined fallback cascade, each rung fail-open into the next: hybrid (dense + lexical, RRF) → dense-only (sparse arm failed) → lexical (vector store offline → filesystem grep) → plain SQL keyword (vault inaccessible) → empty. Two traps: fused (RRF) scores need their OWN threshold (~0.15) — reusing the cosine threshold (0.55) silently drops all hybrid results; and each fallback tier in use should be reported as a health signal so degraded recall is visible, not silent. When adding new metadata fields to an existing store, backfill only points MISSING the target fields (no overwrites), use conservative defaults (importance flat 0.5; confidence by source class: curated 0.85 / session 0.70 / unknown 0.75), resolve timestamps payload → file mtime → now(), and keep the job in dry-run until verified — missing decay fields otherwise default everything to strength 1.0 and the sweep becomes a silent no-op.

M16. Maturity, promotion, and curation provenance

Knowledge/memory items carry a 3-state maturity field — seedling → growing → evergreen — giving synthesized output an explicit “not yet trustworthy” phase that retrieval can weight. The raw→curated→indexed promotion pipeline: a raw intake dir is “source material, not curated knowledge”; a periodic curation agent reads the schema constitution + master index + logbook, decides per item (concept? entity? comparison? skip?), writes typed pages, updates the index, and appends every curation session to an append-only log.md (“if a page looks wrong, check log.md to see which session created it and why”) — provenance-for-curation, distinct from data provenance. Continuous ingestion runs hourly on SHA-256 diff detection with a state file {path, hash, ingested_at} preventing re-embedding. A deterministic promotion trigger worth copying: ≥3 same-type journal entries (decisions/ideas) → propose one permanent memory — synthesize when a countable pattern accumulates, not on a blind schedule. Auto-refreshing environmental context (typed probes — system resources, workspace, activity, network — each a class with a TTL and a generate() method) is indexed into the same store so “what’s true about my environment right now” and long-term memory share one retrieval surface, with a context-first policy for environment questions.

Patterns & compositions

Capture → stage → compile → inject, with cost asymmetry at each hop. Cheap instant capture into an immutable append-only staging tier; expensive batched compilation into a curated vault (M9/M10); bounded injection at session start (index + recent log, hard-capped ~20k chars). Compiled entities cite staging entries (sources: frontmatter) — provenance all the way down.
Classifier-then-dispatch. A meta agent classifies incoming content into memory types first, then dispatches the full input to type specialists that extract independently — cleaner than one monolithic extraction prompt, and the specialists do final filtering (M1, M11).
Deterministic skeleton, LLM flesh. Graph edges, dedup, resolve verdicts, contradiction flags, and queue mechanics are deterministic; the LLM only summarizes clusters, synthesizes insights, and judges edge cases. The system stays valuable with zero AI available (a stated four-tier degradation doctrine: no AI / external / local / managed).
Memory ops as governed proposals. Dedup-merge candidates, decay archivals, importance bumps, and consolidation summaries are emitted as typed PENDING suggestions with impact + evidence + estimated token saving — never auto-applied; a curator/human promotes. Detector battery worth copying: compress-oversized-summary, downgrade-detail-tier, promote-importance-from-usage, merge-by-tag-overlap (>60% same-ontology), archive-unused (30d unserved AND importance <0.7), remove-decayed (<0.05 AND importance <0.3), nudge-threshold. See self-improvement-loops.
Namespaced sharing over new tables. Team/shared pools as user_id = "pool:<id>" rows in the SAME memories table reuse embedding, dedup, decay, and audit for free; every record carries the producing agent id with shared|isolated recall scoping.
Memory health as a product surface. Composite 0–100 health score (coverage 25% + efficiency 25% with an IDEAL BAND of 50–80% budget utilization — >80% penalized as over-stuffing + freshness 20% + balance 15% + cleanliness 15%), daily snapshots, pending-consolidation backlog as a dashboard number, and a mandatory list → dry-run preview → prune flow for forgetting. See product-surfaces-and-ux.
Memory as a path to fine-tuning. Curate training_value on high-value memories → export training pairs → distill into a personal local model (aspirational tier in one system; see local-models-and-inference).

Anti-patterns & failure modes

Blending decay/strength into the ranking formula — old-but-valid loses to new-but-irrelevant (M3’s core finding; measured, not vibes).
Write-only scores. Importance scored at every ingest, consumed by nothing — dead metadata + wasted tokens.
Voluntary feedback contracts. “The agent MUST call fact_feedback when it uses a fact” — it doesn’t; every trust score sits at its prior forever. Derive usage mechanically.
Fail-open recall with no watchdog. except Exception: return [] hid a broken import for ~9 days of dead injection — silent degradation looks identical to no-results. Every recall source needs a last-success/hit-rate health signal.
Path-keyed file dedup. A processed_files(path PK) table re-ingests renamed duplicates and never re-ingests edited files; key on (path, content_hash).
Read-all-then-synthesize with a hard cap, globally. LIMIT 50 newest-first means silent amnesia past 50 memories. Read-all is a legitimate scope-local strategy (per project/run) only, and the cap must be an explicit signal (“N older memories not consulted”).
Two writers, one file. A second subsystem’s write_text() clobbered another’s entries every session end; writer-per-file resolves it.
Strict parsing of user/agent-authored files. One unknown enum value silently dropped 23/25 memories from the index.
Over-aggressive repetition/length gates. A 0.6-overlap injection gate muted all recall from turn 2; a 20-char minimum-query cutoff kills exactly the short technical queries that benefit most.
Hook recursion storms. An LLM call inside a Stop/SessionEnd hook re-fires the hook (one product ships that feature disabled by default for this reason); the extraction session itself triggering capture is the same bug — sentinel-guard both.
Junk-entity accretion. Auto-capture without a notability gate and brain-first lookup (search before create) degrades retrieval permanently.
Temporal boosts without evals. A +0.25 in-time-window boost measured ZERO effect on a long-memory benchmark (fired on 6% of queries; the temporal-question wins came from lexical keyword overlap). Ship no surfacing heuristic without its measured delta; remove 0pp heuristics.
Synthesis without lineage caps re-synthesizes its own output on long runs (parent_ids + reflection_count < 3 eligibility is the fix).
Split-brain state updates. A decay job that wrote strength only to the graph backend while the SQL table diverged silently; keep one canonical store per field.
Cron/system scaffolding captured as memories — orchestrator preambles become “facts” unless explicitly filtered.

Quantitative findings

Typed-edge graph layer ablation: +31.4 points P@5 vs the same stack graph-disabled (P@5 49.1% / R@5 97.9% on a 240-page corpus; lexical-only, vector-only, and hybrid+RRF-no-graph all ~18 P@5). Ingestion $0 in tokens; 146,646-page production deployment.
SVO/entity-bridge graph on multi-hop QA: 71.5% vs 59.5% similarity-only (+12pp).
LongMemEval-S: 95.2% R@5 / 98.6% R@10 / 88.2 MRR with a published BM25-only ablation at 86.2/94.6/71.5 (i.e. lexical-only is already strong); a sibling system reports 89.4% R@5; a reified-graph system reports 88.24% LoCoMo average.
Context cost: token-budgeted injection ≈ 1,900 tokens/session vs 22K+ for full-context memory files (~92% savings).
Temporal time-window boost: 0pp on LongMemEval (honest null result).
Embedding-model tradeoff: a QA-tuned embedder hit 84.8% on passages but 55% on conversational summaries — benchmark on your actual corpus shape.
Decay palette: failure ~11d / assumption ~19d / fact ~24d / strategy ~38d half-lives; importance-tiered 30d/90d; sector multipliers 1.5/1.2/0.8/0.7/0.5 over λ=0.02/day; archive < 0.1; prune < 0.05.
Resolve cascade thresholds: new < 0.85, reinforce ≥ 0.92, subject-guard ≥ 0.60; contradiction-before-reinforce because opposites hit sim ≈ 0.92. Consolidate-around-write: ≥5 neighbors at ≥0.62. Near-dup dedup report at cosine > 0.92; Jaccard dedup at 0.84.
Fusion: RRF k=60; per-stream weights (BM25 0.4 / vector 0.6 / graph 0.3) need not sum to 1; vector adds ~+8pp recall over BM25-only.
Consolidation compression: 444 memories → 16 summaries; per-op costs: flush $0.02–0.05, compile $0.45–0.65/log, query $0.15–0.25.
Injection calibration: repetition gate 0.85 (0.6 muted everything); per-source thresholds vector ≥0.55 top-2 / curated top-3; RRF-fused scores need their own threshold (~0.15 — reusing the 0.55 cosine threshold silently drops all hybrid results); graph-hit floor 0.30 cap 3; volunteer gate 0.7 cap 3 (hard 5).
Bounds: slots 2,000 chars default (20,000 hard); index file 200 lines / 25,000 bytes; L2 items ≤3 per assembly; self-model principles ≤6, promoted at seenCount ≥2 + confidence ≥0.72; pattern pages at ≥3 supporting reflections; index-read retrieval fine at 50–500 articles, switch to retrieval pre-filter ~2,000.
Screen capture: ~1 fps, batch 20 frames, compress to 1920×1080 q85, raw staging TTL 14d; context warn 75% / target 10% post-summarization.

Open questions

Is decay necessary at all once ranking is relevance-only and pruning is proposal-gated? The corpus proves decay must not rank; whether time-based strength beats pure usage-based curation (near-miss + unused detectors) is unmeasured.
Temporal reasoning is not first-class anywhere. “What was true last week but isn’t now?” — even the reified validAt/invalidAt graph doesn’t expose it as a retrieval primitive; no system benchmarks it.
Edge-vocabulary standardization. Every graph system invents its own 6–14 edge types; no convergence on a canonical vocabulary, and no system uses multi-hop traversal as a PRIMARY retrieval strategy (acknowledged weakness even in the strongest graph system).
Where exactly the index-read → RAG crossover sits for mixed memory (not article) corpora; the 500/2,000 numbers come from one article-shaped vault.
Self-model safety. Behavioral-principle promotion has caps but no drift benchmarks; nothing measures whether an injected self-model snapshot improves outcomes or entrenches early habits.
Screen-capture economics and privacy. 1 fps multimodal extraction has no published cost/benefit numbers, and privacy filtering beyond exclusion regexes is unsolved.
Multi-writer convergence. Attribution (agent ids, holders, pools) exists, but no system demonstrates conflict-free memory formation from multiple concurrent agents writing about the same entities; see multi-agent-orchestration.
Verification of consolidation quality. Acceptance gates exist for skill/prompt optimization (median-of-3 judges + epsilon; see verification-and-judging) but no system gates whether a consolidation summary actually preserved the details it claims to.
Retrieval eval harnesses are rare but load-bearing. The credibility of every headline number above rests on a small versioned corpus + query set with per-arm ablations; only two systems in the corpus published one. Building the harness before tuning any surfacing knob appears to be the real prerequisite — see verification-and-judging.