Haystack Open-source framework for question answering system Python Google Colab Demo

In this video I demonstrate how Haystack can be used to build search systems. I use Haystack on MIMIC clinical data to build a simple search and question answering engine
If you like such content please subscribe to the channel here:
www.youtube.co...
If you like to support me financially, It is totally optional and voluntary. Buy me a coffee here: www.buymeacoff...
Relevant Links:
haystack.deeps...
colab.research...
haystack.deeps...

[@FutureSmartAI]

Hi Rithesh,
Thanks. Nice video and a good overview of concepts. Recently I am also exploring Haystack use cases.

[@RitheshSreenivasan]

Thank You Pradip!

[@mjacfardk]

great teacher, thank you brother

[@RitheshSreenivasan]

Thank You

[@aptitudecloud8120]

Hi Rithesh, Thanks for the video.

[@RitheshSreenivasan]

Thank You

[@chandankumar189]

How we can implement the google collab code of QnA model in a web application?

[@RitheshSreenivasan]

You need to create and expose a REST api. There are different ways of doing it

[@abhishekprakash9803]

what about my document i want i asked question from my document ,,,model is able to genearte long answers

[@RitheshSreenivasan]

Have a look at haystack documentation to see if there is a parameter that can be modified for long answers

[@aptitudecloud8120]

Hi Rithesh,
How to fine tune and deploy this model on aws or on any cloud service?

[@RitheshSreenivasan]

Please read haystack documentation

[@keerthanam3299]

Can I know how we can collect the datasets for the nlp on a specific topic?? I am exploring haystack for nlp now. Pls do help

[@RitheshSreenivasan]

There is a website index.quantumstat.com/

[@parthgulati4914]

Where can I get this dataset?

[@RitheshSreenivasan]

Have a look at this video:ruclips.net/video/uYj-uP6CQnA/видео.html

[@rohitchatterjee2327]

Very nice thank you

[@RitheshSreenivasan]

Thank You!

[@sonalikoli384]

very helpful, concept clearing video for me. thanks for it. but i want to know two things here plz help me in it. actually i am facing error in unzipping the file , i tried it many times. and i also want to know how can i do the same process in pdf file actually i need to insert the pdf file instead of document file , i am following the same process that you explained but getting error. please tell me .

[@RitheshSreenivasan]

I am not sure of pdf support in haystack. Check the documentation . Otherwise you need to use python libraries to parse pdf into text

[@sonalikoli384]

@@RitheshSreenivasan thanks ritesh. please tell me the unzipping also i got error.

[@RajdeepBorgohainRajdeep]

Hi Rithesh,
You videos are very informative,
I am currently working in this domain but we have prefer not to use full managed systems like haystack.
The question that I want to ask you is that, When choosing the deep learning model for the embedding what do you consider apart from the available benchmark.
I our use case we have choose mpnet v2
It will be much helpful if you can share your thoughts :)
Much love,
Rajdeep

[@RitheshSreenivasan]

I have generally used BERT based embeddings

[@RajdeepBorgohainRajdeep]

@@RitheshSreenivasan but Does BERT able to handle huge request

[@RajdeepBorgohainRajdeep]

The reason we choose mpnet v2 is because of the performance by trading the speed(QPS).

[@RitheshSreenivasan]

Use whatever works for you. If the text is related to the domain on which the embeddings were trained then you might get fewer out of vocabulary words

[@pchandu1995]

is this free? can i run this in my local and create Q and A?

[@RitheshSreenivasan]

Yes it is open source. Check system requirements in their GitHub page

[@sallu.mandya1995]

Thanks

[@RitheshSreenivasan]

Thank You