RAG Knowledge Curator

Problem it addresses

Enterprise RAG systems often fail because retrieved chunks are noisy, fragmented, or hard to trace. The issue is frequently not the model, but the ingested corpus: navigation text, headers, repeated paragraphs, garbled characters, and ad-like noise can pollute vector indexes. Missing metadata such as topic, version, audience, confidentiality, and source makes retrieval results harder to explain. rag-knowledge-curator turns unstructured documents into a cleaner, auditable, RAG-ready dataset before they enter the vector store.

How it works

The skill runs a pre-ingestion curation workflow:
- Text cleaning: removes garbled text, headers, repeated paragraphs, ad-like noise, and invisible characters while preserving meaningful content.
- Chunking: splits text by chunk_strategy, keeping context boundaries intact and avoiding broken business logic, table explanations, or code blocks.
- Metadata extraction: tags chunks with topic, entities, version, audience, confidentiality, and source for filtering and traceability.
- Quality scoring: rates completeness, accuracy, timeliness, and readability, and notes why a chunk scores low.
- Versioned output: produces a summary, chunk previews, labels, scores, and curation recommendations that can feed vector database pipelines and human review.

Limits

It is best suited for curating documents, SOPs, technical manuals, and product guides. It does not replace vector database deployment, permission control, real-time data synchronization, or business review. For highly structured databases, log streams, or OCR output, it needs external tools for extraction and validation.