Darshil, very enthusiastic and excited I feel while listening your beautiful classes. Please explain different Hadoop and spark projects. That immensely help us to clear the interviews and should be good performer in job role. As You said , your data engineer course will lauch , please explain concept and how concept will be used in project. Very excited for your course. Your course and projects should meet all our needs. Thankyou darshiil. All the best.
@@rajeshramisetti83 Yes, that's the goal! I will try to make everything as much as easier as I do it on RUclips Not only Data Engineering but I will try to cover everything I can from multiple domain Cloud/Devops/ and many more
hey did your code work? because seems like i am getting blocked by amazons anti bot measure (which stops any bot from scraping their data ) could you help me out ?
Thank you Darshil for this amazing video. With the help of this I made Scraping from stock market data based on weekly or monthly gainer and exported it as CSV. it is helpful for swing trades or investment. Thank you for teaching us great things.
Hi, I was working on something similar at the moment. Could you tell me where to get live data from and what kind of data do I need exactly to begin with. This would really help me alot.
Hello, thank you very much for this video. In my case the line [ links = soup.find_all("a", attrs={'class':'a-link-normal s-underline-text s-underline-link-text s-link-style a-text-normal'})] it gives me emptiness and i don't understand the reason. Can you help me? please
It was a perfect learning example of the data mining technique of web usage recently learned about it and saw it practically in your video . Your videos are fantastic for anyone looking for a career in data. Thanks for all things that you provide
Bro thanks for sharing the content as alwys it's amazing ... Can you please post a video about a sample real time data engineer projects you came across and how infrastructure decided like memory etc and what steps involve from development to production. To help us to understand a full picture of actual things happen in the organisation. Thanks much
Hi bro, You are dropping very useful and productive tutorials on yt. Generally i watch python videos specially web scraping. But i didn't find any playlist on this topic,please drop more videos on this topic and make a playlist.
Thanks for walking through the project. Curious to know how can you automatate this process? And what needs to be done for user-agent if we want to automate this process.
I think many people, including me are getting 503 error when trying to send request to Amazon. I tried different domains but it does not work. Any other suggestion to overcome this error?
I am trying to extract the text in for loops but it shows that de find object is a nonetype object that has not the attribute text. But if I choose just one element of the list, i can obtain the text. It just does not work in a loop
hello i love the video it so informative , i just have a little request , how to go from page to another page , like if scraped the page one , how to go to the next page for scrapping
Thanks Darshil for such an amazing content. Just a query, since Twitter API is not free these days, is there an alternative to complete the Twitter Data Pipeline Project in that case ?
Thank you Darshil. How to prevent from being blocked while scraping the data as I think I have been blocked by one of the websites that I have been scraping. Any help?
Hmm, anyone having an issue with the data retrieved from the FIND_ALL() function? It retrieves the HREF links from A tags, but it doesn't do it in order & it skips the first A tagged, HREF element. Hmm
Respones is the output I got and after I tried to access amazon content and it is not allowing to get the contents, eventhough I used user agent. How much time we need to wait for amazon webpage to give access?
Hi, this tutorial is very easy to understand, thanks for making this tutorial. I want to ask whether the use of single (' ') or double quotation (" ") needs to be considered here?
The video was too good suggestion- If you could have written the code in the video instead of reading it in second half, it could be cherry on the cake
When I try to use this program to get an update on the price of the saved items in my cart, the scraper i built using the code above doesn't work. It always gives an empty list. What could I be doing wrong?
Similar, I have a requirement to get the cost of ola and uber for certain list of lat lon ,I am facing issue to connect to the ola and uber website due to authentication and how can I pass the lat lon to the from and to location of website to extract cost. Help me, if possible
Hi Darshil, This one was very informative like your other videos. I like them because they are accurate and to the point and also project-based. I've one doubt related to web - scraping, like some of the modern web pages don't use visual pagination with the page numbers, instead they have like a show more button which renders the new results. In this case it doesn't change the webpage so how can we scrape the new results.
Hey darshil, while executing getting an error ' 403 forbidden ' so pls tell me what to do in that case. Or anyone who encountered this issue how to solve it pls tell me guys..
Hello! I can see you are interested in learning how to scrape websites. I can help you get better at it. Let me know if you’d like more details or if you have any questions!
Don’t forget to hit the like button and tell me what you want see more?
Darshil, very enthusiastic and excited I feel while listening your beautiful classes. Please explain different Hadoop and spark projects. That immensely help us to clear the interviews and should be good performer in job role. As You said , your data engineer course will lauch , please explain concept and how concept will be used in project. Very excited for your course. Your course and projects should meet all our needs. Thankyou darshiil. All the best.
Please don't forget to relase projects.
@@rajeshramisetti83 Yes, that's the goal!
I will try to make everything as much as easier as I do it on RUclips
Not only Data Engineering but I will try to cover everything I can from multiple domain Cloud/Devops/ and many more
one real time End To End Project please
awesome tutorial...
but how we can run to scrape one entire category or 1000 pages
Amazon block our bot
what to do at that time!!!
thank you
I always feel difficulty in doing web scraping, but you man makes it so easy and comfortable
Just loved it!!
I'm not able to do this, when I'm extracting the HTML code from amazon, it is not having any anchor tag, instead it is saying "something went wrong"
Not the 50 videos in a playlist but more data than the whole playlist appreciated man ♥
hey did your code work? because seems like i am getting blocked by amazons anti bot measure (which stops any bot from scraping their data ) could you help me out ?
Please don't quite any content. Because, i am learning from your content. Upload the content like learning with projects.
Thank you Darshil for this amazing video. With the help of this I made Scraping from stock market data based on weekly or monthly gainer and exported it as CSV. it is helpful for swing trades or investment. Thank you for teaching us great things.
Hi, I was working on something similar at the moment. Could you tell me where to get live data from and what kind of data do I need exactly to begin with. This would really help me alot.
Hi Darshil, i tried this but for some reason, I am unable to pull in the product title? Any tips. Thank you.
Hello, thank you very much for this video. In my case the line [ links = soup.find_all("a", attrs={'class':'a-link-normal s-underline-text s-underline-link-text s-link-style a-text-normal'})] it gives me emptiness and i don't understand the reason. Can you help me? please
Same problem
It was a perfect learning example of the data mining technique of web usage recently learned about it and saw it practically in your video
. Your videos are fantastic for anyone looking for a career in data. Thanks for all things that you provide
Hey I need help while scraping do you mind sharing email id can we have google meet?
@@vishaljeshnani1352 Is it done or you still need help
Thumbnail madhi tar लइच खुश झाला भो 😂😂..
Thank for this amazing tutorial. You made it so simple and easy to apply.
Bro thanks for sharing the content as alwys it's amazing ... Can you please post a video about a sample real time data engineer projects you came across and how infrastructure decided like memory etc and what steps involve from development to production. To help us to understand a full picture of actual things happen in the organisation. Thanks much
Your content and the way you are teaching, it's awesome brother.
Keep teaching us 😊...
bro u r a LIFESAVERRRR i am SO THANKFUL for this video and code
Hi bro,
You are dropping very useful and productive tutorials on yt. Generally i watch python videos specially web scraping. But i didn't find any playlist on this topic,please drop more videos on this topic and make a playlist.
That is really tutorial. We need such kind of practical Python tutorial.
I can easily understand from your explanation . thanks
Thank you soo soo much bro, loved the way you explained, I've made a web scrapping for flipkart by just watching your video, thanks alot!!!!!
Great Zahid, I am also going to do the same 😉
Thanks for your effort.. it is such a great learning watching your videos😊
i think now amazon has changed api or something that does not accepting scraping requests as it detects scarling
Thanks man for your help, I could understand everything right from the start. You explained it very easily 😃
Thanks for walking through the project. Curious to know how can you automatate this process? And what needs to be done for user-agent if we want to automate this process.
I think many people, including me are getting 503 error when trying to send request to Amazon. I tried different domains but it does not work. Any other suggestion to overcome this error?
got any solution
@@onkarHINDU Did you got any solution for the problem?
Hi bro , try using retry mechanism I have done the same and it is returning me the data.
@@omkarpatil9717 How to use retry mechanism? can you show how you have done
This one video is better a lot of courses
thank you so much bhai for giving me needed content
This tutorial was amazing, can you do a tutorial to extract data from multiple pages, say 20 pages in amazon?
Useful. Pls, Continue like this. Thank you.. Really Appreciated.
Amazing tutorial on webscraping and data preperation!
I found exactly what i needed! Thank you!
this video really helped me , thank you so much 😍
Awesome, thanks 👍
Best ever and easiest tutorial.
Good job , amazing Tuto !!
Thx Teacher
Really helpful keep making more videos like this ✨
I am getting error on accept language.. telling invalid syntax
i am not getting where is anchor tag. when i check the code their only class a code. how to get a
I am getting error in if_name_=_main_
NameError: name is not refined
Can anyone pls help me to resolve this error
Is Amazon not letting you to scrape data.
I am trying to extract the text in for loops but it shows that de find object is a nonetype object that has not the attribute text. But if I choose just one element of the list, i can obtain the text.
It just does not work in a loop
I’m so glad I found you omggg
Not working, can't pull the data out, ig amazon has changed something, though we can see html tags but can't pull the data
Hi , you can pull the data
@@omkarpatil9717 No they have applied preventive measures, you either need to use scrapy playwright or selenium to do it
Thankyou Darshil
hello i love the video it so informative , i just have a little request , how to go from page to another page , like if scraped the page one , how to go to the next page for scrapping
this is awesome and hopefully you do the pagination also and this will be the best tutorial for web scraping.
Thanks Darshil for such an amazing content.
Just a query, since Twitter API is not free these days, is there an alternative to complete the Twitter Data Pipeline Project in that case ?
Hey, I'm getting the response 503. What should I do?
same
@@aakif6182 Did you got any solution for the problem?
Yeah because the url response might not be providing details at that time due to load on webpage. You can try retry mechanism there and check
same problem
Thankyou bhaiya ❣️
Thank you Darshil. How to prevent from being blocked while scraping the data as I think I have been blocked by one of the websites that I have been scraping. Any help?
have u use any proxy and ip fingerprinting use for real human beahivour for scraping large number of data?
im not getting anything. what could be the problem?
You didn't get html contents or you go 503 error then try aiohttp library
Hmm, anyone having an issue with the data retrieved from the FIND_ALL() function? It retrieves the HREF links from A tags, but it doesn't do it in order & it skips the first A tagged, HREF element. Hmm
Yes I've been having this issue also
You are a chad. Have someone ever told you that? 💯
Dada ek number
i am encountering some error when i try to get text i.e. the productTitle is it because of captcha, please help
Great tutorial.. for future tutorials, can you reduce the size of your face when writing code, it is tooo big!
bro your jawline ☠
How would you go about using proxies to avoid captchas and rotating proxies as well?
Respones is the output I got and after I tried to access amazon content and it is not allowing to get the contents, eventhough I used user agent. How much time we need to wait for amazon webpage to give access?
I am also having same issue can anyone help?
@@itsdevilish9856 Did you got any solution for the problem?
please did you get any solution?????????? please please give meeeterad
@@juanignacio4353 Yes
Try retry mechanism here you would get the response
Thank you
I am getting response 503 after so many tries, can anyone show/tell me that where em I doing wrong.
Did you got any solution for the problem?
You can try retry mechanism here by sending request again and again
thanks
i have a task that i need to get the prices of 3000 products , but the price div is different for differnt link so i am not getting the price
you're the best
Hi, this tutorial is very easy to understand, thanks for making this tutorial. I want to ask whether the use of single (' ') or double quotation (" ") needs to be considered here?
doesn't matter in python
The video was too good
suggestion- If you could have written the code in the video instead of reading it in second half, it could be cherry on the cake
It becomes repeatable that's why I did not do it
@@DarshilParmar Ohk Understood. Thank you. Loved your content. looking forward to more videos like this
How can I scrap product price details of all products in amazon? is it possible?
At 8:09, I get an error that says "requests" is not defined. Can anyone help with this?
😅 Nice explanation I'm using scrapy actually but thinking about how to get data everything from the link and is it gonna block my ip address and ..
Thanks a lot brother! Very helpful!!
how to to get reports from amazon seller central and amazon ads using API or we can scrape them.
Bro is it possible to scrap purchase data of these websites ? , i mean purchase count of a particular product.
When I try to use this program to get an update on the price of the saved items in my cart, the scraper i built using the code above doesn't work. It always gives an empty list. What could I be doing wrong?
How do you scrape from the other next pages also
Similar, I have a requirement to get the cost of ola and uber for certain list of lat lon ,I am facing issue to connect to the ola and uber website due to authentication and how can I pass the lat lon to the from and to location of website to extract cost.
Help me, if possible
awesome
How much time is required to scrape atleast 5 pages or how quick is scarpping
Captcha if I use your header
Great
I am unable to get data using .find method
same here
Hi Darshil,
This one was very informative like your other videos. I like them because they are accurate and to the point and also project-based.
I've one doubt related to web - scraping, like some of the modern web pages don't use visual pagination with the page numbers, instead they have like a show more button which renders the new results.
In this case it doesn't change the webpage so how can we scrape the new results.
You will have to use Selenium
@@DarshilParmar Ok thanks.. I'll try once
bro could please tell me how to get product href link and feature image href links in directory
Thank you for this video bro make on how data engineer work on daily basis
cant find the CLASS in ANCHOR tag, ID is there instead of class, where you are extracting the liinks ?
PLEASE HELP !!!
They keep changing structure
@@DarshilParmar so how am i going to extract the links now ?
I am getting few errors here can you pleaese help me through google meet?
Hi Darshil can you make a video on data pipeline and how to dockeriz the entire pipeline
I will add this in my list, thanks for suggestion
❤
header --- user agent
I'm getting response 503 😥 . Please help !
Are you still getting 503 error?
Did you got any solution for the problem?
When I try to print the links I am getting [] as output
Same issue for me have you found any solution bro..?
Please have a look at discord link I guess its expired I am not able to join
I am getting none in all title, prize, ratings data
Hey darshil, while executing getting an error ' 403 forbidden ' so pls tell me what to do in that case.
Or anyone who encountered this issue how to solve it pls tell me guys..
IP might be blocked, maybe try diffrent internet or IP
At first you went Slow and Explained Everything , But when converting your Data to Dataframe , it was too Fast
Can anyone tell how to scrape the description
I am not getting any output code is running but 🌝
Iam not getting any output all empty lists
Hello! I can see you are interested in learning how to scrape websites. I can help you get better at it. Let me know if you’d like more details or if you have any questions!
@@japhethmutuku8508 why are some of us getting empty lists
How to manage college study with coding
can u share the code
i got response 503 and what can i do next?
i just got the same error i had to use VPN so i can access because in my case the error is caused due to geographical restrictions
Did you got any solution for the problem?
yes I have used different method@@thedailymotivational
@@anilprajapat Can you share what solution you have done?
@@riyamodi8154 i did get any solution properly because amazon can only scraping data for one time after that they applied restrictions on request
Can we do this in vs code
Why not