Try this SIMPLE trick when scraping product data

Join the Discord to discuss all things Python and Web with our growing community! / discord
using the schema.org standards we can easily scrape product data for lots of different pages.
If you are new, welcome! I am John, a self taught Python developer working in the web and data space. I specialize in data extraction and JSON web API's both server and client. If you like programming and web content as much as I do, you can subscribe for weekly content.
:: Links ::
My Patrons Really keep the channel alive, and get extra content / johnwatsonrooney (NEW free tier)
Recommender Scraper API www.scrapingbe...?fpr=jhnwr
I Host almost all my stuff on Digital Ocean m.do.co/c/c7c9...
I rundown of the gear I use to create videos www.amazon.co....
Proxies I recommend nodemaven.com/...
:: Disclaimer ::
Some/all of the links above are affiliate links. By clicking on these links I receive a small commission should you chose to purchase any services or items.

[@graczew]

Good stuff as always. I'm still waiting for some more about how you you keep your scrapers run.

[@JohnWatsonRooney]

Thanks mate. Yes definitely I want to do some more infrastructure type videos

[@return_1101]

Hi, Mr. Rooney! I'm your big fan! Nice video!

[@carlos-ferreira]

Thank you for sharing you knowledge!
I haven't been able to watch all your videos, but do you have a video about crawling a page, filling some forms and downloading a pdf by clicking a button?
Sorry for any mistakes. English isn't my fislrst language

[@JohnWatsonRooney]

Not specifically, however if you look at my automation video, i use playwright to do a similar task, that will work for you. Videos called “automate your job with Python”

[@dramarama359]

Python along with Selenium WebDriver might help you in this task. You could "tell" which clicks to perform using XPATH or HTML selectors, input the data you wish into the form and download the pdf. It interacts with buttons, links and other interactive elements on a web page. The entire process will occur within the browser window, so you will be able to observe its progress in real-time

[@user-lg6dl7gr9e]

Love your content!
Btw i tried to join your discord but the link isn't working

[@anthonyj.587]

Same here

[@dolamuoludare4383]

Hi john , seems like the proxy variable in the code is an environment variable, if it is, how did you derive the proxy value?

[@akshai333]

Hi, can you please create a video explaining how to work with shadow root elements? Specifically, I would like to learn how to access shadow root elements from a webpage and how to interact with them.
Thanks

[@thghtfl]

Man, great content! How do you activate you virtual environment with just act command? Couldn't fins anything on that

[@JohnWatsonRooney]

Thanks! Ah yes that is a custom bind in my terminal shell

[@Tony.Nguyen137]

Where exactly is the json ld inserted? Before the product or after the product? What if my page shows 10 different products? Do i need for each product a json ld script

[@awais.shorts]

Hi Sir, how we can scrape a webpage or website which is showing status code 403.
(Not by saving html) kindly another method.

[@aljohame]

I think I have a good subject to have a video on ..
Where can I contact you

[@learnngo-jr5xu]

What theme do you use if you don’t mind me asking

[@LuicMarin]

Someone should make a tool that parsers schema automatically and extracts all the data.

[@AmrCode1998]

Hello. How get money in scrapping

[@chunman6735]

can i work without proxy? i am use vpn

[@JohnWatsonRooney]

You can yes, most vpns are known and on a block list though

[@rubyachu2958]

Please share a video amazon ae scraping with free proxies

[@MakeDataUseful]

Avoid free proxies!

[@lordlegendsss7776]

Why ​@@MakeDataUseful

[@bakasenpaidesu]

..