Building a Local Memory Layer for Codex and Claude on Top of Obsidian

A build log: agent-wiki-compiler - a generated, safe, local memory wiki for agents

TypeScriptNode.js CLILocal-onlyNo LLMNo embeddingsDeterministicObsidian

01 · MOTIVATION

The problem: powerful agents, short memory

I use Obsidian as long-term memory for software projects: architecture decisions, known bugs, project structure, open questions. Each project gets its own folder of handwritten notes that accumulate over time.

In parallel, Codex and Claude have become a central part of my workflow. They're excellent within a session - but between sessions they lose all context. Every new conversation starts from zero, and I find myself re-explaining the same decisions and the same constraints over and over.

The obvious fix - handing an agent access to the vault - was a non-starter for me. The vault holds far more than project notes, and I don't want an agent scanning all of it, nor writing into it freely. What I wanted was something in the spirit of Karpathy-style LLM Wiki ideas - a knowledge layer built for the model - but simpler, local-only, and with hard safety boundaries: a generated wiki, scoped to a single project, built from the handwritten notes without ever touching them.

02 · WHAT WE BUILT

agent-wiki-compiler

agent-wiki-compiler is a CLI written in TypeScript that lives in its own separate repository at /Users/dorluzgarten/projects/agent-wiki-compiler. It has no runtime dependencies at all - just Node.js. It reads the Markdown notes of a single project from DorBigBrain/02 Projects/<ProjectName>, extracts summaries and indexes from them deterministically, and writes generated output exclusively to DorBigBrain/02 Projects/<ProjectName>/Agent Wiki.

The extractor itself is deliberately simple: it derives a summary from the note's title and first paragraphs, and collects bullet lines found under headings recognized by keyword - Decisions, Architecture, Known Issues, Prompts, Open Questions, and so on. The category is inherited down the heading tree, so a bullet under ## Update - 2026-06-15 nested inside # Decisions still counts as a decision.

AGENT_CONTEXT.md

The agent's entry point: a project overview, the list of compiled notes, and any open questions.

ARCHITECTURE_MAP.md

Lines collected from under headings that talk about architecture and project structure.

DECISIONS_INDEX.md

An index of every decision - lines found under headings matching "Decision" and its variants.

BUGS_INDEX.md

Known bugs and issues - lines under Bug / Issue / Known Issue headings.

PROMPTS_INDEX.md

Notes related to prompts, harness setup, and agent configuration.

SOURCE_SUMMARIES/index.md

An index of all source summaries, one summary file per note.

SOURCE_SUMMARIES/*.md

A deterministic summary for each source note, with a stable filename derived from its relative path.

manifest.json

The list of source files, output files, and generation time - for transparency and inspection.

03 · ARCHITECTURE

Three diagrams, the whole picture

The architecture is built around a single principle: a one-way flow from the handwritten notes to the generated memory, with a single, narrow write boundary.

Diagram 1 - the high-level flow

The handwritten notes are the only input. The compiler reads them read-only, generates the Agent Wiki, and the agents consume the generated wiki instead of scanning the vault.

Diagram 2 - the safety model

One allowed source, one allowed write target. Every other combination - the whole vault as source, output outside Agent Wiki, overwriting handwritten files, or writing without --yes - is blocked before a single byte is written.

Diagram 3 - repository separation

Three separate repositories: the site, the tool, and the vault. The tool reads project notes from the vault and writes only derived memory into the Agent Wiki folder inside that same project.

04 · SAFETY FIRST

Safety-first: the tool is guilty until proven otherwise

The vault is years of accumulated knowledge. A tool that writes into it is a risk, so every design decision started from the assumption that the tool might be wrong - and the system has to stop it before the mistake touches any files. These are the rules enforced in code, not just by convention:

The safety rules

The CLI is built and run outside the vault, in its own separate repository.
Work always starts with a dry-run: the tool prints what it would generate without writing anything.
A preview of the output is first written to a folder outside the vault.
Writing into the vault requires an explicit --yes flag; without it the planned writes are shown and nothing is written.
Inside the vault, only one exact location is accepted: <vault>/02 Projects/<ProjectName>/Agent Wiki.
Every generated Markdown file starts with the marker .
An existing file is overwritten only if it carries that marker; a handwritten file at a generated path aborts the whole run before anything is written.
The source folder is read-only - the tool never creates, edits, or deletes anything inside it.
The whole vault can never be used as the source; you must point at a single project folder.
Dangerous source/output combinations are blocked: output equal to source, output inside source, or source inside output.
No cloud LLM provider: no OpenAI, no Claude, no Gemini. No embeddings and no vector database.
The entire extraction is local and deterministic - the same input always produces the same output.

The last check runs again at the level of the individual file: right before every write, the tool verifies the path sits inside the output folder. Even if a bug in the logic tried to write outside it, this layer stops it.

05 · DEVELOPMENT PHASES

From an isolated CLI to controlled vault writes

Development progressed in graduated phases, each one widening permissions only after the previous phase had proven itself:

1
Phase 1 - a basic CLI, outside the vault
Build the skeleton in a separate repository: a compile command, Markdown scanning, deterministic extraction, and writing to an arbitrary output folder. At this stage the vault isn't writable at all - it's registered as a protected root the tool refuses to write into.
2
Phase 2 - preview against a real project
Run the tool against a real project folder inside the vault (read-only), with output going to a preview folder outside the vault. This is how you evaluate extraction quality on real notes with zero risk to the notes themselves.
3
Phase 3 - explicit vault-write mode
Only once the output looks good, add a mode that writes into the vault: exclusively to the project's Agent Wiki folder, exclusively with the --yes flag, and exclusively over files that carry the generated marker.
4
Phase 4 - update the global AGENTS.md and CLAUDE.md
So future agents know how to use the tool safely, the global instructions were updated: when it's allowed to run (only on explicit request), prefer dry-run, never run against the whole vault, and never commit/push note changes without approval.
5
Phase 5 - future work (intentionally not done yet)
LLM provider support, a local model via Ollama, richer concept and entity extraction, better deduplication, an Obsidian plugin wrapper, and integration with project automation. All of it was planned - and deliberately left out of the first version.

Worth stressing: Phase 5 was not done, and that's a deliberate choice. The first version should be boring, predictable, and safe - cleverness can come later.

06 · COMMANDS

What it looks like in practice

Three commands cover the entire workflow - always in the same order:

1. Dry run - print what would be generated without writing anything

agent-wiki compile \
  --source "/Users/dorluzgarten/projects/DorBigBrain/02 Projects/TelegramRadar" \
  --output "/Users/dorluzgarten/projects/DorBigBrain/02 Projects/TelegramRadar/Agent Wiki" \
  --dry-run

2. Actual write - only with an explicit --yes

agent-wiki compile \
  --source "/Users/dorluzgarten/projects/DorBigBrain/02 Projects/TelegramRadar" \
  --output "/Users/dorluzgarten/projects/DorBigBrain/02 Projects/TelegramRadar/Agent Wiki" \
  --yes

3. Inspect after writing - the tool never runs git itself

cd ~/projects/DorBigBrain && git status

Every run prints a summary: the mode (dry-run or write), the source and output folders, how many Markdown files were found, what was filtered out, and how many output files were planned or written.

07 · HOW AGENTS USE IT

The agent's working contract

The tool is only half the solution; the other half is the rules agents get in the global AGENTS.md and CLAUDE.md. These are the rules:

Rules for agents

The handwritten Obsidian notes are, and always remain, the source of truth.
The Agent Wiki folder is derived, generated memory - not a place to edit by hand.
Agents do not run the tool automatically after every task.
It is run only when explicitly asked to compile or refresh the wiki.
Always prefer --dry-run before an actual write.
Never commit or push vault changes without an explicit request.
If the generated output contradicts a handwritten note, the handwritten note wins.

08 · A PRACTICAL EXAMPLE

TelegramRadar as a test case

The first project to go through the full pipeline is TelegramRadar - a multiplatform SwiftUI app. Its memory folder in the vault holds just two handwritten notes: Project Overview.md and Decisions.md, each of which has accumulated dated ## Update sections over time.

From those two notes the compiler produced a complete wiki: a project context file with an overview and the list of notes, an index of 22 decisions (from design decisions like a specific light-mode background tone to structural decisions), an index of 6 known bugs and issues, and a source summary for each note with a deterministic filename. All in one Agent Wiki folder that an agent can read in seconds instead of parsing long notes with a full update history.

A repeat run produces the exact same paths - the summary filenames are derived from each note's relative path plus a short hash of it - so refreshing the wiki is a clean diff in the vault's git, not a pile of new files.

09 · WHAT THIS GIVES ME

Focused memory instead of scans

Codex and Claude get focused project memory instead of starting every session from scratch.

There's no need to scan the whole vault - just one small, focused Agent Wiki folder.

Memory can be refreshed per project, with a single command, whenever you want.

Fewer repeated explanations of the same project in every new conversation.

Project memory stays durable over time, across sessions and across tools.

It creates a clean bridge between Obsidian - the human memory - and the agents.

10 · LIMITATIONS

What it still isn't

In full honesty

The current extractor is deterministic and simple - it relies on headings and keywords, not semantic understanding.
This isn't a true LLM wiki yet; there's no grasp of context, only structural extraction.
Subtle ideas that don't sit under a matching heading can be missed.
Some lines may be classified into the wrong category.
The generated output should be reviewed, at least the first few times.
The tool is deliberately conservative - better to miss information than to pollute the memory or risk the notes.

11 · FUTURE DIRECTION

Where this could go

Ideas for later

Local model-based extraction via Ollama, without giving up offline operation.
A stronger Markdown parser instead of line-based analysis.
A richer concept graph across notes and projects.
Backlinks between the generated pages.
A dedicated per-project agent onboarding file.
Maybe later: an Obsidian plugin that wraps the compiler.
Maybe: a built-in command inside Codex/Claude workflows.

But even if none of these ideas happen, the tool already does the important work: it turns Obsidian from a personal notebook into project memory that agents can consume - without giving up control, and without risking the source of truth.