LLM Wiki V2 Knowledge Compiler

Problem

Many RAG setups retrieve and then discard: each answer is re-derived, while entities, concepts, relationships, and updates are not accumulated. llm-wiki treats source material as a compilation target: documents, articles, code, conversations, and ledger records become maintainable wiki pages and a typed knowledge graph. It is useful when knowledge needs to be queried repeatedly, traced, and revised, not just prompted once.

How It Works

• Compile sources: /wiki-compile defaults to Agent compile, strips sensitive data such as API keys, tokens, passwords, and emails, renders PDF/PPT/PPTX pages to images, then uses OCR or vision analysis when available; other documents can use MarkItDown.
• Structured pages: generates pages with YAML frontmatter and sections such as Key Facts, Overview, Questions, Details, Relationships, and Source Context, while updating index.md and log.md.
• Knowledge graph: writes entities.json and edges.json, using relationship types such as uses, depends_on, contradicts, and fixed_by.
• Query and ledger: /wiki-query combines metadata, BM25, graph, and ledger streams; /wiki-ledger manages structured tables in DuckDB and includes table data in search results.
• Quality maintenance: /wiki-lint checks contradictions, broken links, and orphan pages; Doctor diagnoses uncompiled sources, OCR misses, and search quality; Dream can modify content automatically with Git snapshots and quality-gated rollback.

Boundaries

The default path does not require an external model API, but deep PDF or PPT reading needs OCR or vision capability, and unreadable sources require user-provided readable content. Dream is an unattended writer and is safest when Git snapshots and rollback are available. Vector embeddings and rerankers are optional experimental tools, not the default retrieval path.