I Replaced My LLM Wiki With a Pure Python Compiler
A pure Python pipeline that compiles messy text notes into a linked, linted markdown wiki — no LLM calls, no embeddings, no external APIs.
TL;DR
- A pipeline that compiles a folder of raw, messy text notes into a linked, linted markdown wiki. No LLM calls, no embeddings, no external APIs, standard library only.
- The pipeline has four stages: a regex extractor, a graph builder that detects cross-references, a section-aware rewriter that preserves anything you write by hand, and a linter that checks its own output.
- Two real bugs emerged while building this: a graph builder that scaled badly, and a linter that silently undercounted orphan pages. Both are documented here as they actually happened, along with the fixes.
- The full pipeline was benchmarked at three corpus sizes on two different machines (Linux and Windows), and the deterministic outputs matched exactly across both.
- Full code, all 17 tests, and unrounded terminal output are included below so you can rerun everything yourself.
Why I Wrote This
I tried building a Karpathy-style LLM wiki. Agent loops. Recursive LLM calls. Embeddings for everything.
The input was a folder of local markdown files I already had, sitting on my own disk.
And partway through, it hit me: I was paying tokens to reorganize text I already owned.
So I replaced the entire pipeline with a pure Python compiler.
This article walks through that system in full: turn a folder of raw, inconsistently formatted text notes into a linked, linted markdown wiki, with zero LLM calls, zero external APIs, and zero third-party dependencies. Every benchmark number below is real, was run on two different machines (a Linux container and my own Windows PC), and I’ve included the two real bugs I hit while building it.
If you’re searching for a pure Python markdown wiki compiler, a deterministic alternative to agent-based knowledge base tools, or a practical breakdown of building a local-first RAG alternative, this is that article.
Complete code: https://github.com/Emmimal/wiki-compiler/
The Compiler Mindset
Here’s the reframe that the rest of this article is built on:
An agent decides what your wiki might look like. A compiler guarantees what it must look like.
I wanted this wiki to be predictable. Unlike an LLM which varies its output, a compiler gives you the same result every single time you run it. That consistency is essential for personal reference notes. The system is structured so that markdown pages act like object files — generated from source and rebuilable at will — with hand-edited content kept separate from machine-generated sections. Here is how the pipeline is set up:
The pipeline breaks down into four stages. Each one handles a single deterministic task and can be tested on its own. No step relies on a model to make a judgment call.
Why Zero Dependencies Matters Here, Specifically
Everything in this codebase runs on the Python standard library alone. No sentence-transformers, no vector database, no HTTP client for an embedding API. That’s not a purity test for its own sake — it’s a direct consequence of the problem this pipeline solves.
Once you strip away the LLM calls, what’s actually left to do is text parsing, string manipulation, and graph traversal over an in-memory dictionary. Those are exactly the kinds of problems re, os, and plain Python data structures were built for. Reaching for a heavier dependency here wouldn’t buy correctness; it would just buy install friction and one more thing that can break for reasons unrelated to your actual notes. If you’ve ever had a pip install hang on Windows because a compiled wheel for a machine learning library wasn’t available for your Python version, you already know why “it just runs” is worth protecting.
The Problem With Agent-Driven Wikis
The idea of using an LLM to build and maintain a personal wiki isn’t new. It gained serious traction after Andrej Karpathy described the pattern in a widely shared post, explaining that he was spending less of his token budget generating code and more of it building structured, persistent knowledge bases out of his research notes. He followed up with a public “idea file” laying out the architecture in more depth, and explicitly compared the process to compilation: raw sources go in, a structured, cross-referenced wiki comes out, and the LLM is the thing doing the compiling [1][2].
That compilation framing is exactly right. The LLM doesn’t need to be the compiler.
Here’s the practical problem. If your raw source is already local, already text, and already deterministic, routing it through a probabilistic system to organize it introduces three costs that a parser or compiler simply doesn’t have:
Cost: Every time you add a new document, an agent-driven wiki re-reads content, decides what changed, and rewrites pages. That’s token spend on organizational work, not synthesis. It adds up fast once your source folder has hundreds of files instead of a dozen.
Latency: Every read-decide-write cycle is a network round trip if you’re using a hosted model, and a real compute cost even if you’re running something local. For work that’s fundamentally about restructuring existing text, that latency has no reason to exist.
Non-determinism: This is the one that actually caused problems. Running the same folder through an early agent-based prototype twice produced two different link structures. Nothing had changed in the source files — the model just made slightly different judgment calls both times about what counted as related. For code, that’s occasionally charming. For the thing you’re using as your source of truth, it’s a problem.
None of this means LLMs are the wrong tool for knowledge work in general. It means they’re the wrong tool for the specific part of the job that’s actually deterministic: taking known inputs and producing a known, reproducible structure. That part is a parsing problem, not a reasoning problem.
Step 1: The Regex Metadata Extractor
Real note folders are a mess. Some files use a # Header, some use a bare uppercase line, and some have no header at all. Metadata like “created:” or “aliases:” might be missing, or hidden in the middle of the file. Any extractor expecting consistent formatting breaks the moment it hits a real-world file.
This extractor checks for a # header first. If that fails, it looks for a bare capitalized line. If it still finds nothing, it defaults to the filename. It scans for metadata fields wherever they happen to exist rather than requiring them to be in a specific location.
This means the pipeline doesn’t crash on a malformed note; it just works with what it finds. It is the most straightforward part of the project, but it cleans up the bulk of the mess, which is why getting this stage to actually hold up required the most time.
Step 2: The Graph Builder
This stage handles the links between notes. Building it revealed a significant performance problem.
The first version ran a separate regex for every entity against every other file — an O(n²) approach. With 100 files, it was fine. With 1,000 files, it took 4.4 seconds. At 5,000 files, it took 107 seconds. The assumption that not calling an API meant the code would be fast by default turned out to be wrong. The algorithm matters more than the absence of network calls.
The fix was to replace the pairwise regex matching with a word-indexed phrase matcher. Each file is tokenized once. As the tokens are traversed, a dictionary lookup checks only for entity names that start with the current word. Instead of testing every entity name against every single position, only the candidates that could actually match are checked. This turns a process that grew quadratically into one that scales far more efficiently as the corpus expands.
The results changed drastically. The 1,000-file run dropped from 4.4 seconds to under 50 milliseconds. The 5,000-file run went from 107 seconds to less than a second. These specific numbers are from early testing in a Linux development environment; the benchmark section further down has the Windows figures.
| Approach | 100 files | 1,000 files | 5,000 files |
|---|---|---|---|
| Naive pairwise regex (first version) | ~46 ms | ~4,400 ms | ~107,000 ms |
| Combined regex alternation (intermediate) | ~12 ms | ~597 ms | ~14,000 ms |
| Word-indexed phrase matcher (final) | ~2 ms | ~33 ms | ~492 ms |
The middle row is worth noting, because it was the first attempted fix and it failed. Combining every entity name into one massive alternation pattern (Name1|Name2|Name3...) reduced the number of regex objects, but the underlying engine still had to check the full list at every single character position it scanned. With 5,000 entities, that is still effectively quadratic behavior wearing a linear disguise. The word-indexed matcher was the only version that actually fixed the complexity, rather than hiding it behind a faster constant factor.
Here is what the resulting graph looks like for three real entities from the test corpus:
