Search-Based RAG with DuckDB and GLiNER

Поделиться
HTML-код
  • Опубликовано: 27 янв 2025

Комментарии • 5

  • @ranjit9427
    @ranjit9427 6 месяцев назад +2

    Can we combine both Text Search and Vector Search? And can u create a complete implementation video?

    • @learndatawithmark
      @learndatawithmark  6 месяцев назад +1

      We can! This would be hybrid search which I need to read a bit more about. Once I've done that, I'll make another video - thanks for the idea :)

    • @saqqara6361
      @saqqara6361 5 месяцев назад

      This is one of the most promising approach in RAG -> hybrid search with a re-ranking

  • @user-wr4yl7tx3w
    @user-wr4yl7tx3w 6 месяцев назад +1

    why would you do full text search over rag?

    • @learndatawithmark
      @learndatawithmark  6 месяцев назад +4

      You'd use Full-Text Search (FTS) if you were searching for specific terms that you know are in the text. But it will only find those terms or any variants close to those words. So if you know exactly what you're looking for it should (in theory) work better than doing vector search, where you'd be matching an embedding of your search query/keywords against embeddings of paragraphs of text.
      Of course this technique won't find other bits of text that have the same semantic meaning.
      But your question makes me curious to compare the two techniques on the same dataset to see if/how the query results differ.