I just remebered my first data scrapping project using BeautifulSoup looking at this video. Thank you very much Alex for sharing this video. It's a great start for anybody looking to scrape data from various sources and easy to grasp when compared to tools like Selenium.
I went over the documentation for BeautifulSoup but watching your video is reinforcing everything I read. I will be watching the rest of your videos on this, thanks!
Very important, DO NOT forget to use a timer otherwise you will find yourself causing DOS on the website you're scraping from or your IP being blocked by the site. Load the page in a browser and copy the page source when building your scraper.
Hey man, I don't quite understand what you are referring to in the second paragraph, wouldn't that be the same as: url = '...' page = requests.get(url) ?
A caveat worth mentioning is to read the terms of use for any website or data source to ensure web scraping or other automated non-API retrieval method is allowed.
Lol if you are going through the effort of scraping, you probably don't care. My government is too incompetent to have ANY API to give access to vital public information, they instead have it on outdated websites that aren't even mobile friendly. Sooooooo..... Actually one of the most highly used apps in my country scrapes data, it's been years and the government still hasn't built an API, they print out a basic excel like table of the data. They infact tell us to download that very app.
Hello you mentioned in your video that beautiful soup is a "snapshot" of the web page. Can we use beautiful soup to automatically update the value of something in real time? For example stock prices or currency exchange rate or price of bitcoin...etc.
Hello I am new here , I actually first found the last video in this list then I heard the word previously so I searched and found the list but also you are referring to previous videos so I now am following alright but I was wondering is this part of another bigger playlist ?
For some reason when I run requests.get(url) it doesnt give me any status code whatsoever, I have also tried using the .status_code and nothing gets returned to me. I also checked and both Beautifulsoup and requests are installed. Is there anything I can try to fix this?
Hello I am an aspiring data analyst I have been watching your "how to build a resume videos"; in one of them, about three years ago, your were reviewing resumes and I wanted to know if you would be willing to review mine (not video or anything), thank you for your hard work ^_^
Merci Alex pour tes supers vidéos ! Voici un post à venir qui pourraient intéresser plusieurs abonnés et autres, c'est le fait de pouvoir trouver comment cliquer sur les choix de demandes de consentements comme sur le site Mappy, qui contient une masse d'infos de Pros et surtout leur email..., ais il y a des fenêtres qui sont bloquantes et difficiles à contourner ("Accepter & Fermer" + "Continuer sans accepter" + "Connexion" ...), merci pour ton écoute? Roby
i assigned a variable to it like that: ' soup = BeautifulSoup(website.text, 'html')' which you then have to return in the end as well. Seemed to solve the error for me
i am getting this error ImportError: cannot import name 'beautifulSoup' from 'bs4' (C:\Users\ASUS\anaconda3\lib\site-packages\bs4\__init__.py) can you tell how to measure it.
This is simply amazing! I have spent hours trying to figure out how to do a basic scrape and finally this video did it. I truly appreciate you sir!
Like I said in my last comment, I'm really enjoying this mini series on webscraping. You're a PHENOMENAL instructor. As always, THANK YOU ALEX!!
I just remebered my first data scrapping project using BeautifulSoup looking at this video. Thank you very much Alex for sharing this video. It's a great start for anybody looking to scrape data from various sources and easy to grasp when compared to tools like Selenium.
We all remember our first time :D
@@AlexTheAnalyst That's what she said
@@ozonehephzibah6949 I got it haha
I went over the documentation for BeautifulSoup but watching your video is reinforcing everything I read. I will be watching the rest of your videos on this, thanks!
Very important, DO NOT forget to use a timer otherwise you will find yourself causing DOS on the website you're scraping from or your IP being blocked by the site.
Load the page in a browser and copy the page source when building your scraper.
how to change every request ip address
Hey man, I don't quite understand what you are referring to in the second paragraph, wouldn't that be the same as:
url = '...'
page = requests.get(url)
?
A caveat worth mentioning is to read the terms of use for any website or data source to ensure web scraping or other automated non-API retrieval method is allowed.
Lol if you are going through the effort of scraping, you probably don't care. My government is too incompetent to have ANY API to give access to vital public information, they instead have it on outdated websites that aren't even mobile friendly. Sooooooo.....
Actually one of the most highly used apps in my country scrapes data, it's been years and the government still hasn't built an API, they print out a basic excel like table of the data. They infact tell us to download that very app.
Yeah and then obey it, right? ... lol
Thanks Alex for this second video on the web scrapping series!
I have been waiting for the next video since you started the Web scraping series. Thank you Alex 😊
Hope you enjoyed it!
@@AlexTheAnalyst 101% I enjoyed every minute. Anticipating the next! Thank you!
Alex thanks for making beautifulsoup as simple as possible
This should be the 2nd video of the playlist
it should be "html.parser" not just "html".
its really simplified. Thanks
So easy to understand. Thank you teacher
Would've been helpful if the first video was part of the playlist, or if it's even still up.
Thank youu for your videoss , you're a life saver
thank you so much.
Hello you mentioned in your video that beautiful soup is a "snapshot" of the web page. Can we use beautiful soup to automatically update the value of something in real time? For example stock prices or currency exchange rate or price of bitcoin...etc.
I love the courses but it is only beginner, so how am I supposed to go from beginner to advanced
When I use this method, it doesnt give me all the site data. I'm guessing its because its a subscriber only webpage. Is there a way to bypass this?
BeautifulSoup(r.text, 'html.parser')
add .parser else a warning will appear
Thankyou Alex!
Please Alex can you do a video on how to install beautifulsoup using Anaconda prompt
crystal clear explanation 😎
Hello I am new here , I actually first found the last video in this list then I heard the word previously so I searched and found the list but also you are referring to previous videos so I now am following alright but I was wondering is this part of another bigger playlist ?
hey alex, how do i get that page thing so i can code there?
Hello. I have installed requests module but then also error occurs as no module named requests.why this happens?
Thanks.
Thank you
For some reason when I run requests.get(url) it doesnt give me any status code whatsoever, I have also tried using the .status_code and nothing gets returned to me. I also checked and both Beautifulsoup and requests are installed. Is there anything I can try to fix this?
same here, let me know if you find a fix
Hello I am an aspiring data analyst I have been watching your "how to build a resume videos"; in one of them, about three years ago, your were reviewing resumes and I wanted to know if you would be willing to review mine (not video or anything), thank you for your hard work ^_^
Anyone help me
ImportError: cannot import name 'BeautifulSoup' from 'bs4' (unknown location)
in jupyter type BeautifulSoup (b and s in caps)
I know it's been 6 months, but fk it. You must install the lib via pip
Merci Alex pour tes supers vidéos ! Voici un post à venir qui pourraient intéresser plusieurs abonnés et autres, c'est le fait de pouvoir trouver comment cliquer sur les choix de demandes de consentements comme sur le site Mappy, qui contient une masse d'infos de Pros et surtout leur email..., ais il y a des fenêtres qui sont bloquantes et difficiles à contourner ("Accepter & Fermer" + "Continuer sans accepter" + "Connexion" ...), merci pour ton écoute? Roby
i might sound a complete noob , but can i do this on vs code?
Yes, ofc you can 😅
@@Alien-by3gb hehhe am not a noob anymore bro
ask me anything
You can program even in notepad or Ms word if you want
I got the import error in 1.05 can someone help me
can you interact using beautiful soup like clicking or entering data in an input tab
No but you can use selenium for that
I have an issue I get an error at 4:13
i assigned a variable to it like that: ' soup = BeautifulSoup(website.text, 'html')' which you then have to return in the end as well. Seemed to solve the error for me
What series is this video part of?
It's the Web Scraping Series - it'll be a mini series in the Python series :)
I'm getting fail to fetch data. could please anyone tell me what to do?
did you find a solution for this.....if not i can help you with it
I'm getting errors
When I do requests.get(url)
I'm not getting any response at all, how did you fix this
i am getting this error
ImportError: cannot import name 'beautifulSoup' from 'bs4' (C:\Users\ASUS\anaconda3\lib\site-packages\bs4\__init__.py)
can you tell how to measure it.
Capital letter B
in jupyter type BeautifulSoup (b and s in caps)
were you able to solve it am facing the same problem even after the caps B and S
can someone hire me?! I know sql, tableau and excel
i get probelm. i don't know about my text. this text is wrong "no module named ' request" i hope so can help me. thank you
make sure you type it 'requests' . *followed by 's'