Web Scraping with Python and BeautifulSoup is THIS easy!
HTML-код
- Опубликовано: 26 дек 2023
- Do you want to scrape structured data from a website with Python and BeautifulSoup? Then you're in the right place! In this video I show you the easiest way to scrape data from multiple pages.
BONUS: At the end of this video you'll learn how to avoid your IP address from being exposed by using a proxyserver.
💭 Join my Discord community and get help from many other Python developers:
/ discord
⭐️ Link to OxyLabs Proxyserver (to hide your IP) ⭐
🔗 oxylabs.go2cloud.org/SH31
⭐ The website we scrape ⭐
🔗 books.toscrape.com/
⭐ Check your IP Address here ⭐
🔗 ident.me/
⭐️ Download the source code ⭐
🔗 tomstechacademy.com/code-exam...
📘 My favorite Python books:
- Python Crash Course: amzn.to/3vIFtJr
⭐️ Follow me ⭐
💼 LinkedIn: / thomas--janssen
✅ Instagram: / tomstechacademy
❗ Some of my videos contain affiliate links. When you click on these kind of links, I get a small percentage in case you make a purchase via the website. Be aware that this doesn't cost you anything - you never pay more than the standard retail price.
#python #webscraping #pythonprogramming
Hy, Thomas you've done a great job, I learned a lot of imp things from you by watching only this video.
Plz make a playlist on web scraping from basic to advance so that a new person can understand better.
Once again amazing video
Thank you so much for your valuable information and the easy-to-follow explanation style.👏👏
You're very welcome Abdullah 😃 is there anything else you would like to learn?
Great video, really very helpful, thanks!
Thank you Ebin! 😀
Very good explaination.. Tyx.. Hope you make some more videos on data scraping using BeautifulSoup
Thank you for the kind words, there’re more data scraping videos in the pipeline 😀
Awesome tutorial 👍
Thank you for the kind words! Happy coding :)
Good video. Many thanks.
Thank you for your comment Code Cobber! Glad my video was useful for you 😀
Since the items are added to a set, how do you ensure that the columns always come back in the same order?
Hi, the items in the dictionary (e.g. Title, Link, Price) will always be in the same order
I followed the same code for a different website except I left out the if and else codes because I wanted to scrape only the first page. Everything worked out except my output which came back in the same column, and it is very disorganized. How can I fix this in my code?
Hi, best way to help you is if you share the website + your code on my Discord, then I can help you right away :)
Join my Discord here: discord.gg/WHJWFNDXXX
Great video and flow. I did this for another website but when i run the code at the end the scraping of pages should be 235 but it doesn't stop it keeps scraping pages? How do i stop it at the amount of pages i would like to scrape from the website.
Hi, there’s a link to my Discord server in the description of this video. If you share your script and the website you want to scrape there, I’m happy to help you with your script 😀
thank you so much
Thank you for your comment, glad my video is helpful for you :)
Thank you for this video. What is the risk of web scraping a Google scholar page (only one page) with an unhidden IP address?
If you only scrape 1 page and only do it one time (so not for example every hour) I would say the risk of your IP being banned is pretty low.. In case you have any doubt, just google "free proxy server" - there're a lot of proxy servers you can use for free. The challenge with free proxy servers is that they aren't reliable - but if you only scrape one page that's not a big deal.. Good luck with your project :)
@@TomsTechAcademyThank you 🙏🏽❤️
does it work in similarweb
I can't tell you whether this method works for specific websites. It should work for most websites, although in case you need to log in or the website uses lazy loading, you need a bit more work to get it up and running..
Brilliant. Thank you.
You're very welcome!
Nice explanation, Very Good, I have a doubt , Kindly provide your email pls....
Thank you for your comment, glad my video is useful for you 😀 you can ask any questions in the comments, this way others who have the same question can also benefit from the answer.