LLM Wiki Knowledge Base System

The Specific Problem It Solves

When dealing with large volumes of documents, traditional retrieval-augmented generation (RAG) systems hit a fundamental bottleneck: knowledge is ephemeral. Every query starts from scanning raw documents anew, meaning the system fails to accumulate insights, contradictions, and synthesized understanding derived from multiple sources. As the corpus grows beyond hundreds of documents, maintaining consistency and traceability becomes an immense challenge.

How It Works: Core Capabilities & Key Steps

This skill’s core philosophy is to transform an LLM from a document retriever into a Wiki building and maintenance agent. It compiles raw documents into a persistent Markdown knowledge base through an incremental, structured process. Its operations revolve around three key steps:

1. Knowledge Ingestion (Ingest Operation)

This is the foundation of building the knowledge base. The skill parses new source documents and automatically executes a checklist:
- Extraction & Tagging: Identifies key entities and concepts, creating or updating dedicated Markdown pages for them.
- Cross-referencing: Automatically adds links using [[page-name]] syntax to weave a knowledge graph.
- Indexing & Logging: Updates the index.md and appends to the log.md, ensuring the system remains navigable and auditable.

It supports two modes: Interactive Mode for crucial sources (where it pauses to discuss emphasis with the user) and Batch Mode for secondary sources (processing autonomously).

2. Knowledge Query & Write-back (Query Operation)

Queries against the Wiki are not simple retrieval. The LLM synthesizes answers from the already compiled knowledge, such as performing concept comparisons or relationship analyses. Crucially, valuable new analyses (e.g., comparison tables, discovered connections) can be saved back into the Wiki as new pages, preventing insights from being lost in chat history.

3. Health Check (Lint Operation)

To ensure long-term quality, the skill includes a Lint function. It periodically scans the Wiki to check for issues like incomplete page metadata, broken links, or inconsistencies, generating a structured health report to catch problems early.

Applicable Scenarios & Caveats

Where It Fits:
- Ideal for building and maintaining long-lived, evolving topic-specific knowledge bases (e.g., technical documentation, research corpuses, project wikis).
- A best practice is to keep the raw/ source directory immutable; the Wiki serves as the mutable knowledge layer, facilitating version control (Git commits are recommended).
- For large wikis (>100 sources), integrating the qmd search engine or using Obsidian’s graph view for navigation is advised.

Key Considerations:
- This is a background compilation process, not a real-time chat system. Its value manifests in the accumulation and refinement of knowledge over time.
- The human's role is to curate sources, direct analysis, and ask insightful questions. The LLM handles the mechanics of summarizing, cross-referencing, filing, and bookkeeping.
- Initial configuration (via WIKI.md) and regular execution of the Lint operation are critical for maintaining system health.