Thank you so much brother. Your video saved lots of my time and helped. Infect I have zero coding knowledge. Just followed your instructions its worked perfectly. Keep it up these good works. All the very best.
@@AminBoutarfi Worked awesome the other day but today I'm getting this error: result = next(iter(done)).result() playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("xpath=(//div[@role=\"article\"])[1]") ============================================================ any solution?
Hi Amin, firstly, thank you for this lesson, I feel more confident with pursuing the language. I do have an issue though , So I noticed the code is different on the github, around the listings area, I managed to get python to start chrome, open maps, input search box, it scrolls once then closes. I don't have any errors that i can see, I may just be missing a value.
Yes that's a Google maps thing! Instead of searching in an entire city, you should search in multiple regions inside it for example since Google Maps will give the first 120 either ways.
thanks habibi this works, but how can i search multiple cities at once and leave it on over night? also can i have multiple chrome browsers running? thanks
This series will be more useful if you teach how to crawl the comments for each business from Google maps, as well as the ratings, or filter by rating stars
It just worked with the update you made in Github. Out of all the web scrapping videos I watched on RUclips, you have been the fastest and easiest. One other question, do you have a script to located emails too? @@AminBoutarfi
Thank you for the video. By the way, on my computer it has different xpath because I use the exact same code with your video, the listings result 0 lists, is it possible?
Feature still not implemented. You need to extend the project! You would need to be able to read all input (for different cities..etc) from a file, and use something like multiprocessing or just remove the sleep time (use proxy to not be blocked)
Hi admin, thanks for you video. It help me so much. But I have a issue and I can't fix it. Can u help me? When I search for a specific address (Currently Scraped: 1), it won't continue and has an error in page.hover. Is there any way to fix it?
@@AminBoutarfi Thanks for the reply. Still getting this error - playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded. =========================== logs =========================== waiting for locator("xpath=//button[contains(@data-item-id, \"phone:tel:\")]//div[contains(@class, \"fontBodyMedium\")]") Could me please tell me how to use the if condition so that if any of the xpaths are not available, the script won't stop and scrape whatever is available.
@@AminBoutarfi Just an update: I used the IF & ELSE statements before each xpath. However, the generated xlsx file and csv files have blank values. Can you please help me out with this?
@@AminBoutarfi An example is - if page.locator('name_xpath').count('name_xpath') > 0: business.name = page.locator(name_xpath).inner_text() else: business.name = ""
@@kunalsarmagermin8 Hi bro, did you can used if else statement for the case have no phonenumber ? could you share with me, I have a trouble with this. Tks
By default playwright throws a timeout error after waiting 30000ms (30 seconds) in either: 1- Internet was slow and didn't load page. In this case increase time out time: for example: page.goto('www.google.com/maps', timeout=60000). In here I'm playwright waits for 60 seconds. 2- Or you asked for an elements that wasn't found. In this case for check if element is present on the page before scraping as follows: if page.locator('your xpath'). count>0: Element = page.locator('your xpath'). Hope it helps!
Hello @AminBoutarfi after 2 problem (if page.locator('your xpath'). count() > 0: ) i have problem below . Could you please help me with it? Many Thanks. result = next(iter(done)).result() playwright._impl._api_types.Error: Error: strict mode violation: locator("xpath=//a[contains(@href, \"www.google.com/maps/place\")]").nth(9).locator("xpath=..").locator("xpath=//span[@role=\"img\"]") resolved to 2 elements:
Unfortunately no! the reason is simple, There are no emails on Google Maps. You would need to get the websites, the go through them and search for emails
i don't know how to get the X-path, can you guide me, i tried but the output is like this //*[@id="QA0Szd"]/div/div/div[1]/div [3]/div/div[1]/div/div/div[2]/div[2]/div/div[1]/div[1]/h1/text() doesn't seem right. I tried pointing to the element then choosing copy path
I am getting this error: Looks like you launched a headed browser without having a XServer running. Set either 'headless: true' or use 'xvfb-run ' before running Playwright. Do you know what's wrong? Your help would be much appreciated
I believe that you are running the script on a server without a UI. 1- install xvfb 2- run script as follows: xvfb-run python3 main.py -s= -t= Hope it helps!
Hello Bro this is a good tuto but still there are some erorrs , the name coms blank, scrolling not working always shows the first result even the script shows that scrolling ... please check it more more time and make a new video or update your repo. Thank u
Thanks for the feedback. Currently working on an Updated video/code. This is always the case with Web Scraping. If a website changes anything, we need to update the code so
Hi Amin Thanks a lot for the video i am getting error while installing Playwright which mention couldn't build greenlet and need to install "pyproject.toml-based" Could you help me how to fix this? Thanks never mind, i have downgraded the python version and it works Thanks
Thank you so much brother. Your video saved lots of my time and helped. Infect I have zero coding knowledge. Just followed your instructions its worked perfectly. Keep it up these good works. All the very best.
Thank you so much for this video!
Thank you for your support!
This is too amazing! Thank you so much for sharing. it's too helpful in my startup business :)
Thank you! Glad it helped!
@@AminBoutarfi Worked awesome the other day but today I'm getting this error:
result = next(iter(done)).result()
playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator("xpath=(//div[@role=\"article\"])[1]")
============================================================
any solution?
@@SoundTech-i6g Yes same here. The role atricle has been removed :(
bon courage 5oya kmal
Can you please show how to include the reviews as well?
Good Idea! Will add that in the next Google Maps Scraping tutorial!
@@AminBoutarfi I'm also waiting for this video.
thnx for the video, but do u have a tutorial about scrolling down to the end of page in order to get all what we can ?
Everything works fine. But "name" column is blank. Something wrong in XPATH I think
Could you show the code for using a Proxy service?
Will do that in future video!
how to implement all results, not just 5?
Hi, data is not being save in csv and excel
Do I simply need to download Python (and follow the code as per video) to scrape data from Google Maps? I am not a developer.
Hi Amin, firstly, thank you for this lesson, I feel more confident with pursuing the language. I do have an issue though , So I noticed the code is different on the github, around the listings area, I managed to get python to start chrome, open maps, input search box, it scrolls once then closes. I don't have any errors that i can see, I may just be missing a value.
Thanks for the support! This video is part 1, check the lastest part
how to add scrolling feature and I want to scrap data from multiple pages
I added the scrolling feature to the Github repo. hope it helps!
Arrived at all available
Total Scraped: 120
always stops at 120, but there are lots of businesses. But not continuing. Why? Do you have any solution?
Yes that's a Google maps thing! Instead of searching in an entire city, you should search in multiple regions inside it for example since Google Maps will give the first 120 either ways.
but still, 120 is too low for my search of interest. is there any way to do this thing properly? without limitation?
Did you find a way to scrape more data
thanks habibi this works, but how can i search multiple cities at once and leave it on over night? also can i have multiple chrome browsers running? thanks
I'm including that feature soon! Stay tuned
How to update the xpaths?? please
Sir, it does not open the chrome but it prints zero as I change -l and -s values. what should I do?
thank you
This series will be more useful if you teach how to crawl the comments for each business from Google maps, as well as the ratings, or filter by rating stars
In coordinates all the longitudes and latitudes are same in every lead why?
there's a bug, fixing soon!
I see when I make a search I can’t be specific. For example Latino restaurants in Dallas Texas. Only if I do restaurants in Texas USA.
I believe "Latino restaurants in Dallas Texas" works too, I just tried it
It just worked with the update you made in Github. Out of all the web scrapping videos I watched on RUclips, you have been the fastest and easiest. One other question, do you have a script to located emails too? @@AminBoutarfi
Thank you for the video. By the way, on my computer it has different xpath because I use the exact same code with your video, the listings result 0 lists, is it possible?
How can we do For Multi Cities or Multiple Business at one time 😢😢
Feature still not implemented. You need to extend the project! You would need to be able to read all input (for different cities..etc) from a file, and use something like multiprocessing or just remove the sleep time (use proxy to not be blocked)
Data not saving in excel and csv
how do i package it using Nuitka i m getting errors
I'm not familiar with Nuitka!
Hi admin, thanks for you video. It help me so much. But I have a issue and I can't fix it. Can u help me?
When I search for a specific address (Currently Scraped: 1), it won't continue and has an error in page.hover. Is there any way to fix it?
When I run program it only scrape 120 and I need 1k data is it possible to scrape huge Data
You need setup user-agent and proxy aggregator for scrape huge data
@@Faybmi how to set up do you have a tutorial
This video may have been released one year ago, but it’s still no longer relevant
but his github code is still relevant
if page.locator('website_xpath').count>0:
TypeError: '>' not supported between instances of 'method' and 'int'. Getting this error. Can you help me?
Count is a method that you are calling! you should use count() "with parenthesis". Hope it helps!
@@AminBoutarfi Thanks for the reply. Still getting this error - playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded.
=========================== logs ===========================
waiting for locator("xpath=//button[contains(@data-item-id, \"phone:tel:\")]//div[contains(@class, \"fontBodyMedium\")]")
Could me please tell me how to use the if condition so that if any of the xpaths are not available, the script won't stop and scrape whatever is available.
@@AminBoutarfi Just an update: I used the IF & ELSE statements before each xpath. However, the generated xlsx file and csv files have blank values. Can you please help me out with this?
@@AminBoutarfi An example is -
if page.locator('name_xpath').count('name_xpath') > 0:
business.name = page.locator(name_xpath).inner_text()
else:
business.name = ""
@@kunalsarmagermin8 Hi bro, did you can used if else statement for the case have no phonenumber ? could you share with me, I have a trouble with this. Tks
will this work on windows
Yes! On any computer OS I believe
"playwright._impl._api_types.TimeoutError: Timeout 30000ms exceeded" facing this error. Can you please help me with it. Thanks in Advance :)
By default playwright throws a timeout error after waiting 30000ms (30 seconds) in either:
1- Internet was slow and didn't load page. In this case increase time out time: for example:
page.goto('www.google.com/maps', timeout=60000). In here I'm playwright waits for 60 seconds.
2- Or you asked for an elements that wasn't found. In this case for check if element is present on the page before scraping as follows:
if page.locator('your xpath'). count>0:
Element = page.locator('your xpath').
Hope it helps!
@@AminBoutarfi It worked. Thanks a ton.
Hello @AminBoutarfi after 2 problem (if page.locator('your xpath'). count() > 0: ) i have problem below . Could you please help me with it? Many Thanks.
result = next(iter(done)).result()
playwright._impl._api_types.Error: Error: strict mode violation: locator("xpath=//a[contains(@href, \"www.google.com/maps/place\")]").nth(9).locator("xpath=..").locator("xpath=//span[@role=\"img\"]") resolved to 2 elements:
=========================== logs ===========================
waiting for locator("xpath=//a[contains(@href, \"www.google.com/maps/place\")]").nth(2).locator("xpath=..").locator("xpath=//span[@role=\"img\"]")
Please could I search email address also ?
Unfortunately no! the reason is simple, There are no emails on Google Maps. You would need to get the websites, the go through them and search for emails
i don't know how to get the X-path, can you guide me, i tried but the output is like this //*[@id="QA0Szd"]/div/div/div[1]/div [3]/div/div[1]/div/div/div[2]/div[2]/div/div[1]/div[1]/h1/text()
doesn't seem right. I tried pointing to the element then choosing copy path
I am getting this error: Looks like you launched a headed browser without having a XServer running. Set either 'headless: true' or use 'xvfb-run ' before running Playwright.
Do you know what's wrong? Your help would be much appreciated
I believe that you are running the script on a server without a UI.
1- install xvfb
2- run script as follows: xvfb-run python3 main.py -s= -t=
Hope it helps!
Hello Bro
this is a good tuto but still there are some erorrs , the name coms blank, scrolling not working always shows the first result even the script shows that scrolling ...
please check it more more time and make a new video or update your repo.
Thank u
Thanks for the feedback. Currently working on an Updated video/code. This is always the case with Web Scraping. If a website changes anything, we need to update the code so
How we can earn money from this data?
Hi Amin Thanks a lot for the video
i am getting error while installing Playwright which mention couldn't build greenlet and need to install "pyproject.toml-based"
Could you help me how to fix this?
Thanks
never mind, i have downgraded the python version and it works
Thanks
Glad you fixed it. Whenever you come across that error, note that it's a version thing
nadi canadi
please reply soon
Hi, how can i contact you privately ? I would like to request a quote for a project
Hey, here's my email: mr.amineboutarfi@gmail.com
Hi, data is not being save in csv and excel
Hi, data is not being save in csv and excel
Should be fixed!