Vector Database Master

Problem

Semantic search, RAG, and multimodal retrieval often get stuck on practical choices: which of Milvus, Qdrant, Chroma, Pinecone, or pgvector fits the workload? How should HNSW, IVF_FLAT, or IVF_PQ be configured? A standalone demo may work, but cluster deployment, benchmarking, ef, M, partitioning, connection pooling, and monitoring can become hard to control.

How it works

The skill organizes vector database practice into an execution path:
- Selection: compares deployment mode, performance, filtering, ops burden, scale, and community for five vector databases.
- Indexing: explains precision, speed, and memory tradeoffs for IVF_FLAT, IVF_PQ, and HNSW.
- Deployment and benchmarking: provides Milvus Docker Compose validation, cluster architecture references, and 1M/10M/100M baselines.
- RAG and multimodal search: covers LangChain integration and text-image retrieval in one Collection.
It also lists production pitfalls such as Embedding dimension mismatch, bulk writes, HNSW M, Partition Key, ef tuning, dynamic schema, monitoring, archival, persistence, and connection pooling.

Boundaries

It is useful for taking vector retrieval from prototype to production, especially with Milvus. The material is practice-oriented and script-indexed; performance numbers depend on hardware, vector scale, and data distribution, so they should not be treated as fixed SLAs.