Web Scraping + Reverse Engineering APIs

Поделиться
HTML-код
  • Опубликовано: 24 дек 2024

Комментарии • 21

  • @cguser
    @cguser 7 месяцев назад +8

    finally a talk on Web Scraping! good to see you again wesbos and scott!

  • @pedrogorilla483
    @pedrogorilla483 7 месяцев назад +4

    Awesome! On the same line, I’d love an episode on reverse engineering scrambled or minified webapps 😏

    • @WesBos
      @WesBos 7 месяцев назад

      good idea - I think there is also one on how to find objects of data in the JS heap

  • @chamithjanaka6040
    @chamithjanaka6040 7 месяцев назад

    Love you both from Sri Lanka...🇱🇰 ❤

  • @bingerminn
    @bingerminn 7 месяцев назад +1

    Awesome! I was using puppeteer to scrape a site and converted it to pinging their api directly. So much faster and no random errors when a element fails to load. Where would you host your scraping scripts that run everyday, hour or minute? I used a package to run it as a service on windows.

  • @parkerrex
    @parkerrex 13 дней назад

    Great sode fellas

  • @qnoox
    @qnoox 7 месяцев назад +1

    love this podcast and this episode since i’m also an scrape OG/ automation panda :) side question will the video format of the podcast ever pan into visual snapshots; when talking about something like when mention console then pan into a snapshot of that or if a website is mentioned than a print screen of that like wes did once during the this video; i know this will add in more work during editing but it would be extra coolness if it was included as a standard; thanks keep up the awesomeness 🎉👍;

    • @jayfiled
      @jayfiled 7 месяцев назад

      Yeah, I jumped off the audio version and onto RUclips hoping to see something in action. But I think that would slow down the time to upload, CJ probs has something in the mix no doubt.

  • @KevinMacKenzie61
    @KevinMacKenzie61 7 месяцев назад

    Is there a course you recommend for this?

  • @paullvindquist
    @paullvindquist 7 месяцев назад

    I never thought I’d hear XPath mention on a podcast. It’s really too bad XML became a 4 letter word. There was actually some cool things you could do with it that you can’t do with JSON. It also having a DOM for one thing.

  • @jayfiled
    @jayfiled 7 месяцев назад

    How would you alert if something was available? I want instant, attention ambushing feedback if my scraper finds something.
    If i run a cypress script in headless to check a site for tickets, say, and it found one, i want a desktop alert somehow. Browser alerts work if i run it manually, but if I schedule it on mac, then it runs in the background and i dont get any alerts.

    • @Andres-it2du
      @Andres-it2du 5 месяцев назад

      I'm kind of late to this question, but telegram bots are where useful for this!

  • @siya.abc123
    @siya.abc123 7 месяцев назад +3

    Lol I've been watching every episode since CJ joined and yet I'm not subscribed 😅
    Time to change that

    • @WesBos
      @WesBos 7 месяцев назад

      yeahhh buddy

  • @stolinski
    @stolinski 7 месяцев назад +1

    Working on a scraper rn.

    • @jayfiled
      @jayfiled 7 месяцев назад

      Public repo? Link us up

    • @jayfiled
      @jayfiled 7 месяцев назад

      Oh it's you Scott, hahah. I had a rush of enthusiasm to work on it with a fellow listener but now I feel silly.

  • @Cyber-Bison
    @Cyber-Bison 5 месяцев назад

    Have you or anyone else extracted data from an interactive chart?

  • @Stoney_Eagle
    @Stoney_Eagle 7 месяцев назад

    If someone scrapes for indexing and links to your site to consume it I am totally cool with it, but if someone scrapes to bypass the site I'm not.