Personal RAG Without the Drag (It's Actually Agentic Search)

Most conversations about RAG start in a place that makes sense for companies and very little sense for personal notes. They typically begin with chunking strategies, embedding models, vector databases, rerankers, freshness guarantees, permission boundaries, eval tooling, dashboards, orchestration, and with someone always bringing up GraphRAG.

Those are real, genuine concerns when you're building retrieval over large, messy, multi-user corpuses.

My problems are smaller than that - I have an Obsidian vault with raw captures, cleaned-up notes, controlled tags, and topic wikis. I want Claude, Codex, or another assistant to use that context without me having to paste 5+ into the prompt every time.

In all fairness, my title was a bit of a click-bait, what I am actually building has a more precise name than RAG. Anthropic calls the broader discipline context engineering, and the specific strategy here is what their engineering team calls just-in-time context loading: an agent keeps lightweight references, in this case file paths, and loads the real content at runtime instead of pre-processing everything up front. RAG is one way to implement that. Pointing an agent at a well-organized folder is another, and for my single Obsidian vault - it works amazingly.

The Vault Is the Architecture

The useful part of my vault is organized around four folders (Figure 1):

Figure 1. The vault's four working folders, in Obsidian.

There's also 90-System/templates/, which defines the shape of the notes, but the working knowledge lives in those first four folders: 00-Raw is intake, 10-Notes is the normalized layer, 20-Tags is the controlled vocabulary, 30-Wikis is synthesis.

If that's starting to sound like Databricks' medallion architecture, bronze, silver, gold, that's not an accident - raw, notes, and wikis map cleanly onto it, with tags off to the side as a reference layer. 'Obsi-dallion,' if you want a name for it. Figure 2 shows the resulting pipeline:

Figure 2. Raw captures to enriched notes, controlled tags, topic wikis, and assistant context.

00-Raw Is Not the Knowledge Base

The raw folder stays deliberately boring: voice notes, meeting transcripts, pasted thoughts, inbox fragments - captures that land before they're trusted. A transcript is not a note. It hasn't been cleaned, titled, tagged, linked, or summarized, so it doesn't carry the same retrieval weight as an enriched note or a wiki page. My assistant treats 00-Raw as something to check, not somewhere to start - raw material is allowed to be messy, just not allowed to pass itself off as finished knowledge.

10-Notes Is Where Capture Becomes Data

A skill turns each raw capture into a note: read whatever landed in 00-Raw, rewrite it into a fixed shape - frontmatter first, then body sections. You can see how this is basically what vector database pipelines do to every chunk before it gets embedded - structured extraction is an old idea, LLMs just made it cheap enough to run on personal notes too. The frontmatter is the schema (type, date, source, project, tags, status); a meeting note looks like this (Figure 3):

Figure 3. A meeting note's structured frontmatter.

The body is just as important as the metadata, shown in Figure 4:

Figure 4. The same note's body: Summary, Key points, Decisions, Action items, and Transcript.

The frontmatter makes the note filterable without opening the file. Headings matter too: ## Summary and ## Transcript hold different kinds of information, so a search that ignores headings can just as easily hand back a wall of transcript when a two-sentence summary would do. And ## Related links the note to others by hand - which, if you squint, is a graph node, minus the graph database.

I don't make my assistant infer structure the note can just state. If I already know the date, project, decisions, and related notes, that belongs in the Markdown - not something to rediscover with embeddings.

20-Tags Is a Vocabulary, Not a Hashtag Dump

Each tag is its own Markdown file with a small definition (see Figure 5):

Figure 5. A tag file: a one-line definition plus auto-populated backlinks.

That one-line definition changes the role of tags: not labels sprinkled across notes, but a controlled vocabulary a tag has to earn its way into. The same skill that writes notes assigns tags in the same pass, biased toward doing nothing - check existing tags first, and only mint a new one if nothing covers the concept, it's genuinely central, and it'll plausibly apply to future notes too. On any close call, reuse the closest existing tag instead.

Tags are the cheapest useful filter in the system: a training question starts from the training tag, a client question from the client tag. When a topic has both a tag and a wiki, the tag is the boundary and the wiki is the current synthesis - tags are schema, not a vibe.

30-Wikis Is the Synthesis Layer

This is where the vault stops being a pile of notes. A wiki page is a living briefing over a cluster of source notes - close to what Andy Matuschak calls an evergreen note: one that gets rewritten as understanding improves instead of staying frozen at capture time. It has a stable shape (Figure 6):

Figure 6. A synthesized wiki page for a topic.

Most personal RAG setups skip this layer and expect the model to synthesize everything at answer time. That wastes the work already done in the vault - a wiki page is already the compression of many notes into something a human can read.

A second skill builds wikis after notes exist to cluster: four meeting notes on a topic with no wiki, and it's ready for one; existing wikis refresh the same way as new notes land. Under four, a hub page isn't synthesis, it's one conversation with extra formatting.

For retrieval, wikis come before source notes - current state first, evidence second, raw transcript only to verify. You can think of it as a reranker I computed once, by hand, instead of scoring it fresh on every query. Figure 7 shows the full order:

Figure 7. The assistant's lookup order: tag definitions first, raw transcripts only when needed.

That order is the point: retrieval can't clean up a bad note, fill in missing context, or check that anything is true - it just finds text and hands it to the model. A half-finished capture gets answered just as fluently as a well-sourced one, same tone, same confidence, no visible seams, so the order decides what surfaces first.

Retrieval Is a Pointer, Not a Pipeline

Once the vault has these layers, there's no retrieval system left to build. Claude and Codex already read files, grep, and follow links - that's the job. All they need is to be told where to look and in what order:

Vault lives at <path>.
Check the tag definition first, then the topic wiki, then source notes.
Only open 00-Raw when the wiki and notes do not have the answer.
Cite file paths or Obsidian links.

That's the entire integration - the same list from paragraph one, minus everything on it. No hosted vector database, no local index, no sync platform, no dashboard, no separate memory product, no dedicated retrieval skill. My assistant's own file tools are the retrieval layer, running directly against the Markdown that's already the source of truth. Nothing gets built, so nothing falls out of sync.

This mechanism has a name too: agentic search, sometimes agentic retrieval - an agent using its own tools, grep, glob, read, iteratively, instead of a fixed embedding-lookup pipeline. Not a fringe choice, either. Claude Code shipped with RAG and a local vector database early on, then dropped it once agentic search proved to work better; Boris Cherny, who leads Claude Code, has said it "outperformed everything. By a lot." He's separately said agentic search avoids RAG's issues around security, privacy, staleness, and reliability - the exact concerns that make RAG heavy to run. Amazon Science found agentic keyword search reaching 94.5 percent of RAG's faithfulness with no vector store at all. The vault structure in this post exists to make that kind of search cheap: tags, wikis, and headings are what let grep-and-read replace an index in the first place.

This isn't hypothetical - it's how I work day to day. My standing instructions point Claude at the vault path, same pattern as the block above, and it checks there whenever a task could use the context. The vault owns the knowledge; pointing my assistant at it is the only retrieval layer I need.

Why Keyword Search Comes First

Keyword search sounds outdated, but it fits how this vault is built. Embeddings exist for when you don't know the exact words to search for - they turn text into numbers and pull back results that mean the same thing, even if the wording doesn't match. Useful, but most personal questions don't need it. I already know the note title, project, client, tag, date, or heading I'm after (Decisions, Action items, Open questions). If the words are already in my head, plain keyword search finds them faster and more reliably than a similarity search would. That's also where a lot of practitioner writing lands: you probably do not need a vector database for your RAG, yet.

So the first instruction to my assistant isn't "find semantically similar chunks." It's closer to a filtered lookup:

Find notes tagged training.
Prefer 30-Wikis.
Then search 10-Notes.
Return the Current state, Open questions, Decisions, and Sources sections.
Cite file paths or Obsidian links.

A much, much smaller problem to solve. If keyword search ever falls short, embeddings can slot in later as one extra step, not a whole new system: run both searches, combine the results, rank them, and hand the best ones to the model.

keyword results
  + embedding results
  -> merge
  -> rerank
  -> cite notes

Embeddings are an upgrade on top of a search that already works. They're not a substitute for having the structure in the first place - a fuzzy search still needs something well-organized to search over.

Vector databases exist to answer one question well: what means something like this, across more content than anyone could hold in their head. That's the actual job they're built for, and it's a hard one. It's just not my job - I already have the handle, a tag, a title, a date, so reaching for similarity search on top of that isn't a scale problem. It's a purpose mismatch.

There's a related instinct worth resisting: just retrieve more, dump the whole vault in, and let the model sort it out. More context does help - until it doesn't. Chroma's research on context rot found that even strong models get 30 to 50 percent less accurate well before the context window is actually full, because burying the right answer in a pile of loosely related notes isn't the same as handing it over directly. That's the real reason the lookup order matters, not just to save tokens: it keeps the wiki's answer from getting lost under three raw transcripts nobody needed to read.

What This Gives My Assistant

This setup gives my assistant more than text - it gives it a trust model. 00-Raw means probably-unclean source material. 10-Notes means an enriched record with summary, metadata, and links. 20-Tags means a controlled concept boundary. 30-Wikis means current synthesis over a topic.

That lets it answer differently depending on what it finds: start at the wiki if one exists, flag a status: draft note as such, say plainly when an answer only has individual notes behind it, treat 00-Raw material as unprocessed, and show disagreement between notes instead of smoothing it over. It's the same instinct behind eval tooling on a real RAG system, just done by hand instead of by a labeling pipeline: know how much to trust an answer before you hand it over.

That's the real payoff. My assistant isn't retrieving more context - it's retrieving context with shape.

Where This Breaks

This system has real sharp edges:

Obsidian's Graph view is the obvious place to check that last one, but I've found it's mostly decoration (Figure 8):

Figure 8. Obsidian's Graph view. Looks nice, doesn't do much.

Looks good in a screenshot, which is roughly all it's good for - I don't navigate by it, and it plays no role in how my assistant answers anything. The graph that actually works is the one enforced by ## Related links and the lookup order, not the one rendered on screen. Which is also my answer to GraphRAG from the intro: at this scale, the useful part of a knowledge graph is link discipline, not a database to stand up and query.

That's personal RAG without the drag. Not less serious. Just right-sized.

The Point

Retrieval is only as good as what's underneath it. Capture into 00-Raw, promote into 10-Notes, keep 20-Tags controlled, synthesize into 30-Wikis - only then let assistants search, and let embeddings show up later if keyword search, tags, and links actually hit a wall.

The goal was never a miniature enterprise search platform for one person, just more useful notes. Ask a question, get back the right tag definition, the wiki page, two source notes, and an answer grounded in those files - that's enough. The source of truth stays in Markdown, the folders say what kind of knowledge each file holds, and my assistant gets better context for free.

Small, local, inspectable, boring. That's the RAG I wanted.

I'm building this out as an actual Obsidian plugin, Nous - inbox, notes, tags, wiki threshold, and all. Early and rough in places, but if you'd rather see it as code than a blog post, that's where it's happening.