In the abstractive summarization use-case, usually a lot of focus is given to the LLMs being used and its performance. Limitations of LLM including context length and ways to overcome this issue are often overlooked. Its important to make sure that our application is scalable when dealing with large document sizes. Thank you for this great and insightful video.
Thank you Vijay Gandhi, for your insightful comment! You've raised an excellent point about the importance of considering the limitations of LLMs in the context of abstractive summarization, especially regarding their context length and scalability issues when dealing with large documents. Indeed, one of the significant challenges in using LLMs for abstractive summarization is their inherent limitation in processing long texts due to the maximum token limit imposed by these models. This constraint can be particularly problematic when summarizing lengthy documents or articles, where the full context might not fit within the model's capacity.
Hey, Abonia..Thanks for the amazing content. I just had one issue though: on executing the 'map_reduce_outputs' function, I had the ConnectionRefusedError: [Errno 61]. Hope someone know what it is
hello! Thanks for the video. I am from Brazil. What would you recommend for large documents, averaging 150 pages? I tried map-reduce, but the inference time was 40 minutes. Are there any tips for these very long documents?
Thanks for you kind words and glad this helped. Implement a strategy that combines semantic chunking with K-means clustering to address the model’s contextual limitations. By employing efficient clustering techniques, we can extract key passages effectively, thereby reducing the overhead associated with processing large volumes of text. This approach not only significantly lowers costs by minimizing the number of tokens processed but also mitigates the recency and primacy effects inherent in LLMs, ensuring a balanced consideration of all text segments.
Really useful info mam , keep up the good work
It's my pleasure.
In the abstractive summarization use-case, usually a lot of focus is given to the LLMs being used and its performance. Limitations of LLM including context length and ways to overcome this issue are often overlooked. Its important to make sure that our application is scalable when dealing with large document sizes. Thank you for this great and insightful video.
Thank you Vijay Gandhi, for your insightful comment! You've raised an excellent point about the importance of considering the limitations of LLMs in the context of abstractive summarization, especially regarding their context length and scalability issues when dealing with large documents.
Indeed, one of the significant challenges in using LLMs for abstractive summarization is their inherent limitation in processing long texts due to the maximum token limit imposed by these models. This constraint can be particularly problematic when summarizing lengthy documents or articles, where the full context might not fit within the model's capacity.
Hey, Abonia..Thanks for the amazing content. I just had one issue though: on executing the 'map_reduce_outputs' function, I had the ConnectionRefusedError: [Errno 61].
Hope someone know what it is
@@alvaroaraujo7945 Hello , thanks for your kind words.
It may be related to your ollama serve.Are you sure Ollama is running ?
hello! Thanks for the video. I am from Brazil. What would you recommend for large documents, averaging 150 pages? I tried map-reduce, but the inference time was 40 minutes. Are there any tips for these very long documents?
Thanks for you kind words and glad this helped.
Implement a strategy that combines semantic chunking with K-means clustering to address the model’s contextual limitations. By employing efficient clustering techniques, we can extract key passages effectively, thereby reducing the overhead associated with processing large volumes of text. This approach not only significantly lowers costs by minimizing the number of tokens processed but also mitigates the recency and primacy effects inherent in LLMs, ensuring a balanced consideration of all text segments.
@@AboniaSojasingarayar Video was great and very useful.. can you make the small video on this clustering method using embedding ?
@@VirtualMachine-d8x Sure will do, happy to hear from you again. Thanks for the feedback.
Did you use OpenAI API key here?
Here we use open-source Mixtral from ollama.But, yes we can use OpenAI models as well.