What is an LLM wiki?
An LLM wiki is a knowledge base that a large language model helps write and maintain from your source material. You collect notes, documents, articles and research. The model organizes information from those sources into connected pages and updates them as new material arrives.
In Andrej Karpathy’s proposed approach, the wiki is a persistent collection of Markdown files. It gives useful summaries and connections a place to live beyond an individual conversation. It is a pattern you can implement with different tools, rather than one specific application.
For a product team, this might mean connecting customer interviews, an onboarding decision and a project brief. A teammate could then ask why the team chose a particular approach and find an explanation linked to the original evidence.
The distinction between a source and a summary matters. A meeting note records what was captured. A wiki page contains the model’s interpretation of that material. Keeping both makes it possible to check a claim, understand its context and correct a misleading summary.
The goal is to make accumulated knowledge easier to use. Its reliability still depends on the quality of the sources, clear instructions and review of important changes.
How an LLM wiki works
An LLM wiki turns collected material into connected pages that remain available beyond a single conversation. In Karpathy’s proposed architecture, it has three layers.
- Raw sources are the original materials you collect, such as articles, research papers, meeting transcripts and notes. The model reads these files without rewriting them, preserving a reference you can return to when checking a summary.
- The wiki is a collection of Markdown pages the model creates from those sources. It contains summaries, topic pages, comparisons and connections. As new material arrives, the model updates relevant pages and links related information.
- The schema is the instruction document that defines how the model should maintain the wiki. It describes page structure, naming conventions and workflows for adding sources, answering questions and reviewing existing content. In a file-based setup, these instructions can live in CLAUDE.md or AGENTS.md.
The workflow repeats as your knowledge grows. Add a source, have the model integrate it, then ask questions across the connected pages. Useful answers can become new wiki entries. Periodic reviews help identify outdated claims, conflicting information and missing links.
Keeping these layers separate makes errors easier to investigate. If a summary misrepresents a decision, you can compare it with the original record, correct the wiki page and improve the instructions that guided the update.
What changes when a team runs it
Karpathy presents the LLM wiki as a pattern for personal knowledge management, but he explicitly includes team use. The idea extends naturally to shared research, meeting notes, customer calls and product decisions. His proposal also mentions human review and Git for version history and collaboration.
What changes is the coordination. One person can often remember why a page changed and spot when a summary misses the point. With several teammates and agents contributing, that context needs to be recorded.
Concurrent changes. What happens when two agents update the same page from different sources? The team needs a way to detect conflicts and reconcile changes without silently losing either contribution.
Visibility and review. Can teammates see what changed, which sources informed it and whether anyone has checked it? A newly generated summary should have a clear review status.
Questions that need a person. If two documents disagree, where does the unresolved question go? Someone needs to own the answer so an agent does not quietly choose an interpretation.
Access to shared knowledge. Who can read the source documents, edit the wiki or approve a decision? Those boundaries also need to apply when an agent retrieves information on someone’s behalf.
Where an LLM wiki can go wrong
A wiki can look organized and consistent while drifting away from its sources. In The Hidden Flaw in Karpathy’s LLM Wiki, Anand Lahoti highlights the risk of future answers relying on stored AI summaries without checking the original evidence.
A summary can become an unchecked source
The problem can develop through a few small steps.
- A summary loses a condition. The model captures the general idea but drops a limit, exception or qualification.
- A later answer relies on that summary. The assistant retrieves the wiki page and treats its wording as authoritative.
- The interpretation spreads. Other pages reuse the claim, carrying the mistake into future answers.
- Internal checks miss the issue. Pages agree because they inherited the same error. Their consistency does not establish accuracy against the source.
Consider a fictional onboarding decision. The team approves letting users explore a demo before connecting their tools. A summary reduces this to “connecting tools is optional.” A later assistant recommends removing the connection step, even though the decision still requires it for working with real customer data.
Preserving the original helps only if the system returns to it when the details matter. A citation should let someone check the supporting passage, including any conditions the summary omitted.
A growing wiki does not guarantee shared understanding
There is another risk even when the summaries are accurate. More documentation does not automatically mean that people understand it.
In An LLM Wiki Won’t Compound Your Knowledge, Chris Lettieri distinguishes automated knowledge organization from the work of learning. His argument emphasizes capturing your own thinking, including questions, observations and connections, alongside the material you collect.
For a team, that distinction has a practical consequence. A polished summary of a meeting does not establish that everyone agreed with it. A decision page may record what the team chose without explaining which alternatives were rejected or what uncertainty remains.
Imagine that a product manager and a developer read the same onboarding summary. One understands “optional connection” to mean users can postpone it until after the demo. The other understands it to mean users never need to connect anything. Both have access to the wiki, but they leave with different expectations.
The information worth capturing therefore includes more than the outcome.
- The reasoning. Why did the team choose this approach?
- The alternatives. What did people consider and reject?
- The uncertainty. Which assumptions still need testing?
- The agreement. Is this a suggestion, an interpretation or an approved decision?
AI can help organize these details and surface unanswered questions. People still need to resolve disagreements and confirm that the record reflects their intent.
These are risks to manage, not inevitable outcomes. Karpathy’s pattern already separates originals from generated pages and includes human involvement. The challenge is preserving those distinctions as more people, sources and agents contribute.
LLM wiki, RAG and chat memory
These approaches help an assistant use information beyond its training, but they play different roles. An LLM wiki maintains connected knowledge pages. Retrieval augmented generation, or RAG, brings relevant stored information into an answer. Chat memory preserves context from earlier interactions. They can work together within the same system.
| Approach | What it keeps | How it helps | What needs attention |
|---|---|---|---|
| LLM wiki | Persistent pages with summaries, explanations and links to sources | Makes accumulated knowledge easier to browse and reuse | Generated pages can lose detail, become outdated or carry forward an incorrect interpretation |
| RAG | A searchable collection of content and an index used to retrieve it | Supplies relevant passages when the assistant answers a question | Retrieval can miss important evidence, and the model can misinterpret what it finds |
| Chat memory | Conversation history, summaries or selected facts and preferences | Maintains continuity within a conversation or across sessions | Retained context may be incomplete or outdated; its scope depends on the implementation |
These approaches can work together. Chat memory provides conversational context, the wiki explains a topic, and RAG retrieves supporting passages from wiki pages or original sources.
For an onboarding question, that could mean remembering the pilot context, finding the decision summary and checking it against the approved record. What matters is whether the answer reflects current evidence and can be verified.
How to keep summaries connected to the evidence
Treat every AI summary as an interpretation that someone should be able to check. A few practical rules help keep it grounded.
- Preserve the originals. Store source documents separately from generated summaries.
- Link to supporting passages. Make it easy to find the exact wording, including conditions and exceptions.
- Label the status. Distinguish observations, AI interpretations, proposals and approved decisions.
- Review affected pages when things change. Record which sources informed a summary and revisit it when those sources are superseded.
- Verify important details before acting. Check dates, numbers and commitments against the relevant original record.
When sources disagree or leave a question unanswered, keep that uncertainty visible. A useful summary should help the team identify what still needs a human decision.
Frequently asked questions
Is an LLM wiki better than RAG?
It depends on the task. A wiki gives readers a maintained overview of a topic. RAG retrieves material for a particular question. For example, understanding how your onboarding strategy evolved benefits from connected pages, while checking an exact requirement calls for the relevant source passage. You can combine both by using wiki pages to navigate a subject and retrieval to check the underlying evidence. Neither approach guarantees accuracy.
Do I need Obsidian to build an LLM wiki?
No. Obsidian provides a convenient interface for reading Markdown pages, following backlinks and viewing their connections. The agent that creates and updates those pages is a separate part of the setup. You can use another Markdown editor or a web interface instead. For example, Documentation.AI describes a team implementation where its browser editor and published site take the place of Obsidian. The essential parts are the sources, wiki pages and instructions governing updates.
Can a team share an LLM wiki?
Yes, but sharing the files is only part of the setup. Teams also need a way to review edits, resolve simultaneous changes and control access. Documentation.AI describes one concrete workflow in which agent edits go onto a branch, a person checks the preview and diff, and approved changes are merged. Other implementations can use different tools, but should make it clear who changed a page and whether the change has been reviewed.
Does an LLM wiki replace the original documents?
No. The generated pages should remain separate from the original evidence. Anand Lahoti illustrates why with a payment agreement whose summary drops the exact discount and payment window. A later answer based only on that summary cannot reliably recover the missing terms. Keep the original document accessible, link summaries to supporting passages, and return to those passages when a question depends on precise wording, numbers or exceptions.