FAISS is not suitable for production. The dedicated vector search solutions solve all the issues you mentioned: you just store the metadata along with vectors in JSON format. At least, with Qdrant, it works like this: https://qdrant.tech/documentation/concepts/payload/
Thanksthat makes sense and it never even crossed my mind . FAISS has been great for prototyping but I'm definitely hitting the limits around metadata, updates, and operational overhead.
One thing I’m exploring now is Qdrant in embedded mode, since the tool has to run in fully air-gapped environments (no internet, no external services, distributed on a portable SSD). The embedded version runs as a simple file-based directory, similar to SQLite:
from qdrant_client import QdrantClient
client = QdrantClient(path="./qdrant_data") # local-only, no server
If that model works reliably, it would solve several problems FAISS creates for my use case:
incremental updates instead of full index rebuilds
storing metadata as payloads instead of a 1.5GB pickle
much easier filtering (e.g., per-source, per-customer, per-tool)
better concurrency under load
I’m still benchmarking, but curious about your experience:
Have you used Qdrant’s embedded mode in a production/offline scenario? And if so, how does it behave with larger collections (500k–1M vectors) on consumer hardware?
Not dismissing FAISS — just trying to pick the right long-term architecture for an offline tool that gets updated via USB and needs to stay lightweight for the end user.
miniCOIL is a contextualized per-word embedding model. It generates extremely small embeddings (8dim or even 4dim) while still preserving the word's context for each word in a sentence.
I can answer how it would be in Qdrant if interested. The index will take around 70GB RAM. New vectors are first placed in a non-indexed segment and are immediately available for search while the index is being built. The vectors and the index can be offloaded to disk. Search will take some milliseconds.
reply