LLM Wiki: The Compounding Knowledge Base, Explained

Andrej Karpathy shared a gist called “LLM Wiki” and suddenly everyone had one.

An LLM wiki is a knowledge base written for a language model to read and maintain. It's a set of small, cross-linked documents an AI agent can navigate, cite, and keep current, instead of static docs that rot in a drive nobody opens.

That's the whole idea. The interesting part is why it works, and how to build one that stays useful past week two.

Why an LLM wiki, and why now

Every company already has a knowledge base. It's called Slack, email threads, six half-finished Notion pages, and three people's heads. An agent can't act on any of it in a meaningful manner. Data an LLM can't read is data that doesn't exist as far as your AI is concerned.

The LLM wiki addresses data accessibility and quality issues in an elegant way. It fixes the input, not the model. Give an agent a clean, linked, current body of knowledge and its answers stop being generically plausible and start being specifically correct about your domain. One of the biggest barriers to AI adoption is that agents are not reliable enough to trust. The wiki is a particular way to fix that.

Two things made this practical: models got good enough at following links and maintaining structure, and long context stopped being a luxury. The technique isn't really new, everyone always knew a wiki (or some shared knowledge base) was a good idea. But agentic LLMs with long context and discipline to call tools (to fetch and manipulate data) make the idea practical now.

How an LLM wiki actually works

There are a handful of moving parts.

1. Small pages, one idea each. A page is a concept, an entity, or a source summary, not a 40-page handbook. Small pages are what a model can load, reason over, and update without clobbering everything around it.
2. Cross-links between pages. Every page points to related pages. The links are the retrieval path: the agent follows them the way you'd follow a train of thought, pulling in only what's relevant.
3. An index the agent reads first. One catalog page listing everything. It's the entry point — the map that tells the agent where to look before it looks.
4. A maintenance loop. New information comes in as an immutable source; the agent summarizes it, then updates every page that idea touches. Knowledge compounds instead of piling up. This is the hard part most people would skip with manually curated wikis.

Notice what's not here: a vector database, an embeddings pipeline, a semantic-search layer. Not because retrieval doesn't matter — a wiki is already a form of retrieval-augmented generation. The model retrieves knowledge by following links and an index, rather than by embedding similarity. Semantic search is a different retrieval method under the same RAG umbrella, and it earns its keep when you scale to a large, unstructured corpus no one can hand-curate or link. Start with link-based retrieval for the knowledge worth structuring; add vector search when the corpus outgrows what links can navigate. The two compose.

How to build an LLM wiki: the minimum that works

Here is a setup you can stand up this afternoon:

• A sources/ folder for raw inputs you drop in and never edit.
• A wiki/ folder the agent owns — concept pages, entity pages, source summaries, each in lowercase-kebab-case.md.
• An index.md the agent keeps current, listing every page with a one-line summary.
• A short instruction file telling the agent how to ingest a source, when to create versus update a page, and to always fix the index.

Then use it. Every time the agent answers a question worth keeping, it files the answer. Every source you add updates the pages it touches. Within a few weeks the wiki knows things no single document ever stated — because the agent connected them.

That compounding is the real payoff. A static knowledge base is worth what you put in. An LLM wiki is worth what you put in plus every connection the model draws between the pieces.

Where the LLM wiki fits

An LLM wiki alone isn't the final form of agentic AI. It's one component of a working AI engine, the knowledge layer that feeds your agents. Pair it with the harness that runs those agents and the operating layer that lets them act across your systems, and the wiki stops being a clever note-taking trick and becomes infrastructure.

In the bigger picture, “should we build an LLM wiki” is the easy question. Yes, you need knowledge bases and wikis work well for that. You should build toward generally useful AI agents that move your business forward. Your AI should be a competitive advantage, not technical debt. A knowledge base your agents actually read is where that starts.

Challenges and lessons learned

Someone has to actually read the wiki

Actually LLMs must read it, but humans must remember to point the agents to it. It's not useful if it's not.. used. We're no longer solving a technical problem, we're solving a human problem. Employing LLMs to build and leverage knowledge bases in business context is a social challenge that tends to require a cultural shift and change management to fully realize the benefits.

The wiki is not a dumping ground

It's critical to consider the audience of the wiki when collecting information. I've found it necessary to maintain several wikis for different scopes: company knowledge, personal domain, project-specific, etc. For the practitioners: I like to map each to an agent skill, either global or local, so it's clear what goes where from context. I also like to publish wikis via subagents to keep information search token spend from polluting the task context.

Sharing is caring, but hard

The wiki is a dream come true for individual users. It works well for small tech savvy teams with a git repository backing it up. In addition to the cultural shift, sharing wikis within a larger organization is a key technology question. Done improperly, writes start to conflict, inconsistencies accrue, and technology selection becomes politics. Even markdown as a format can be questioned when scaling up to larger user bases. Role-based access control, governance, budget, ROI become top of mind questions.

This is the line between a clever tool and infrastructure

A Markdown folder is an afternoon project. An LLM wiki a whole organization trusts (permissioned, governed, consistent under concurrent writes) is an architecture problem. That's the shift we spend our time on: taking something that works for one engineer and making it work for the business.

If you're past the weekend-project stage and want the org-scale version to just run, that's what we build. Talk to us about your AI engine →

Frequently asked questions

What is an LLM wiki?

A knowledge base written for a language model to read and maintainl Small, cross-linked documents an AI agent can navigate, cite, and keep current, rather than static docs a human has to update. It overcomes the biggest burden of knowledge bases: maintaining information relevance.

How is an LLM wiki different from RAG?

It isn't a different thing. An LLM wiki is a form of RAG (retrieval-augmented generation). What differs is the retrieval method: a wiki retrieves by explicit links and an index the model follows like a human would, while vector-based RAG retrieves chunks by semantic similarity from an embedding store. Links suit curated knowledge worth structuring; semantic search suits large, unstructured corpora you can't hand-curate. Same goal, different methods, and production systems often use both.

Do I need a vector database to build one?

No. Start with a folder of Markdown files, an index page, and an agent instructed to keep both current. Add infrastructure only when a real limit forces it.