How to scrape through captchas, geo blockers and rate limits (crawl4ai + Local Deepseek + Proxy)

Поделиться
HTML-код
  • Опубликовано: 7 фев 2025

Комментарии • 16

  • @leonardogrig
    @leonardogrig  День назад +3

    Link to the code: leonardo467.gumroad.com/l/xezswi

  • @out-of-sight
    @out-of-sight 23 часа назад +1

    Finally someone did this 🎉

  • @JhonniMartins
    @JhonniMartins День назад +2

    +1 Sub, I don't think I ever saw this type of content with such quality of information and easy to follow along, would love if you linked the code for the test website with anti-scraping features, that would be a the cherry on top of the cake, Good work

    • @leonardogrig
      @leonardogrig  22 часа назад

      Welcome aboard! Thanks a lot for your feedback, it means a lot to know my content is actually helping!

  • @brahimsliti156
    @brahimsliti156 День назад +1

    Thanks man you are pushing amazing content

  • @DataTribe-c5f
    @DataTribe-c5f День назад +1

    Teu conteúdo é ouro! Só aula cara! Muito obrigado por compartilhar!

  • @FarrokhRostamiKia
    @FarrokhRostamiKia День назад +3

    Thanks, man! Really helpful. Which model would you recommend for production? Have you tried the Craw4i with Gemini 2.0? With the new pricing, it seems like a solid competitor.

    • @leonardogrig
      @leonardogrig  День назад +1

      Thanks for the feedback! I've been using either gpt-4o, deepseek or one of groq's llms are awesome too (and free).
      Haven't tried gemini for a while now though, I'll test it today!

  • @grupik_pl
    @grupik_pl 7 часов назад +1

    amazing video

  • @liormechubad2489
    @liormechubad2489 20 часов назад +1

    Thanks, man! Using the cookie with changing proxy can't be risky and being banned in some websites?

    • @leonardogrig
      @leonardogrig  18 часов назад

      That’s a great point, and I overlooked it. Rotating proxies cause IP changes that could get flagged when reusing cookies. Logging in at each access helps, but with residential proxies, it’s very unlikely you’ll face bans or rate limits.
      If you want a persistent IP, just change your proxy settings to a sticky session.

  • @axeor1166
    @axeor1166 День назад +1

    Great video!

  • @codelucky
    @codelucky 14 часов назад

    Can you possibly do this on Quora?