Karpathy LLM wiki foundation

The Karpathy LLM wiki foundation is a memory pattern in which sources are compiled once into a wiki and the wiki is then queried. A compiled wiki is what is queried,¹ whereas RAG re-reads raw sources on every question.²

Overview#

Sources stay immutable, and the model writes the wiki.² A schema states how to ingest, query, and maintain it.² Useful answers are filed back, so later questions start from the wiki.²

Mechanism#

1. Immutable sources are indexed, and wiki pages are compiled from them.²
2. On a question, the index is read, then the pages, and the answer cites them.²
3. Useful outputs are filed back into the wiki.²
4. The wiki is linted for stale claims, orphans, and missing links.²

Applications#

The pattern applies when memory should be a maintained wiki instead of a fresh read of the sources on every question.

Limitations#

The pattern is not a product to clone, and fine-tuning weights is not a substitute for first making the wiki the thing that is queried. RAG is not the primary memory when a compiled wiki already holds the answer.²

See also#

• Ephemeral wiki compile-lint – a per-question ephemeral wiki, lint, loop, and report
• Thin harness, fat skills – thin harness, fat skills
• Autoresearch loop – the human iterates the prompt, the agent iterates code, and a metric gates keep or discard

Further reading#

• https://x.com/polydao/status/2055237756318781502
• https://x.com/polydao/status/2042203352054771748

References#

Footnotes#

1. https://x.com/karpathy/status/2039805659525644595 ↩

2. https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹