SF(0404): Searching vector embeddings at scale with Weaviate

Tech Talk 3: Searching vector embeddings at scale with Weaviate
Speaker: Dan Dascalescu @Weaviate
Abstract: With the proliferation of Large Language Models (LLMs) such as GPT, LLaMA, Chinchilla or Claude, comes the need for storing and indexing vector embeddings - the numeric object representations output by ML models. Efficient search among embeddings for the top results matching a query is performed with Approximate Nearest Neighbor (ANN) algorithms, which usually are implemented in specialized vector databases running separately from the objects database. This separation between objects and their embeddings leads to several problems, such as increased difficulty of performing filtered vector searches, or the need to keep the databases in sync for C(R)UD operations. Weaviate is an open source vector database (with a SaaS offering) that solves this problem by storing both objects and vectors, allowing to combine vector search with structured filtering. It supports question answering, text/image/audio or vector search, and piping the results through 3rd party APIs such as OpenAI for generative search. You can bring your own vectors, or you can configure Weaviate to use OpenAI, Cohere, or any Hugging Face transformer to vectorize your data
ML Meetup in San Francisco - 4/4/2023
Website: www.aicamp.ai/...