Open Source LLM Search Engine with LangChain on Ray

Waleed, Head of Engineering at Anyscale, explains how to use LangChain and Ray Serve to build a search engine using LLM embeddings and a vector database.
Blog: www.anyscale.com/blog/llm-ope...

[@khalidal-reemi3361]

Thanks Waleed.
You clarified many concepts and made it doable .

[@shafiemukhre]

Great overview on how to use Ray with LangChain!

[@waleedkadous3165]

Thank you! Can't wait to share Part 2 with you.

[@ruixifan2346]

Great overview!

[@waleedkadous3165]

Thank you very much!

[@hsubyron2277]

This is amazing

[@anyscale]

Thanks a lot!

[@user-qb8cy2iw5k]

Has LocalHuggingFaceEmbeddings been replaced with another class? Because I am facing this error:
ImportError: cannot import name 'LocalHuggingFaceEmbeddings' from 'embeddings'

[@anyscale]

There's also an embeddings.py file. You can find that here: github.com/ray-project/langchain-ray/blob/main/open_source_LLM_search_engine/embeddings.py

[@labloke5020]

It would be more helpful if you concentrate on a specific use case like embedding and creating an app using local LLM using Ray. How it could scale and please do not use LangChain - there is no such thing as Faiss vector store.

[@anyscale]

Details of the FAISS vector store and its integration can be found here: python.langchain.com/en/latest/modules/indexes/vectorstores/examples/faiss.html. You do not have to use LangChain, but honestly LangChain is pretty useful.

[@ikjb8561]

Great video. Do you have any script that performs the search? Will this code work with CPU only? I am getting this error.
_ = BeautifulSoup(
Loading documents ...
Time taken: 0.00011205673217773438 seconds.
Loading chunks into vector store ...
Traceback (most recent call last):
File "lchainray.py", line 41, in
db = FAISS.from_documents(chunks, embeddings)
File "/home/osboxes/.pyenv/versions/myenv/lib/python3.8/site-packages/langchain/vectorstores/base.py", line 317, in from_documents
return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
File "/home/osboxes/.pyenv/versions/myenv/lib/python3.8/site-packages/langchain/vectorstores/faiss.py", line 502, in from_texts
return cls.__from(
File "/home/osboxes/.pyenv/versions/myenv/lib/python3.8/site-packages/langchain/vectorstores/faiss.py", line 453, in __from
index = faiss.IndexFlatL2(len(embeddings[0]))
IndexError: index 0 is out of bounds for axis 0 with size 0