This code does not work getting error after this section if __name__ == '__main__': main() pprint(output) NameError Traceback (most recent call last) in 1 if __name__ == '__main__': ----> 2 main() 3 pprint(output) NameError: name 'main' is not defined
syntax error brother pay attention cuz in coding even a point is an error .u r saying if _name_ == '__main__': but correct syntax is __name__ double underscore(_) . while u used on 1 underscore. _name_ < __name__
Hello is there a way to scrape certain followers with example minimum 500 followers I don’t want to scrape accounts won’t 0 followers, fake account. Any way?
Hi there! Your tutorial is awesome, both methods look really great. I was trying to copy step by step the Selenium code, and I'm getting the following exception when debugging: Message: unknown error: cannot find dict 'desiredCapabilities' I tried adding capabilities = chrome_options.to_capabilities(), and also adding desired_capabilities= capabilities below run_on_insecure_origins; but this doesn't seem to work. How can I solve this issue? Thanks in advance!
Hey, we're glad you enjoyed it! One of our guesses would be that you're not using Webdriver-manager. If not, here are some resources on how to set it up: www.selenium.dev/documentation/webdriver/getting_started/install_drivers/#1-driver-management-software github.com/SergeyPirogov/webdriver_manager Another guess - your software might need to be updated. The versions we've used are: Webdriver-manager = 3.2.2; Selenium = 4.1.0; You can always check if your browser is updated too. Let me know how it goes!
Thanks for your question! VPNs change your IP address, but it won't necessarily work for web scraping and social media platforms. The main issue is that VPNs aren't trying to hide you're using them. Thus, it's way easier to get detected using a VPN than proxies. Here's a video that explains it in more detail: ruclips.net/video/AwKXkkEQ4ms/видео.htmlsi=N4yG9pbURHCVCyG0
Hi:) First, get yourself a Python package: www.python.org/downloads/ & JSON (library): pypi.org/project/jsonlib/ For scraping with Selenium, get the following tools: Selenium: www.selenium.dev/ WebDriver: chromedriver.chromium.org/downloads Selenium-Stealth: pypi.org/project/selenium-stealth/ For scraping with Requests: Requests library: pypi.org/project/requests/ Random (library): pypi.org/project/random2/
Scraping public data is permitted. So as long as you're not getting private details or copyrighted material, and not scraping behind a login (that's how you make sure data is public), you should be ok.
Hi, we're sorry to hear that. Could you tell me more about what you're trying to scrape? And about your code, please describe what it looks like. And whether you copied it step-by-step?
@@Proxyway Hi! Thank you for video! Your code gives this error: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) How can it be fixed?
Would it be possible to scrape all unique usernames who have used a specific hashtag? Ex: I want to know: 1. all unique usernames 2. who have used the hashtag #proxyway 3. Since Jan 1st 2022 Where would I start to try and do something like that?
Lets say i want to scrap all the data from the posts someone has made. What is the limit of data i can extract?
This impressive thanks, Is there a way to return the body text based on certain criterion, lets say by passing certain key word?
Thanks for the video! Very clear explanations …
Happy to help!
very helpfull man thanks a lot
Hey , i am using proxy to scrape instagram posts but i facing CSRF token problem with login , how i can fix it ?
Thanks for this content it is great. Is there a touch point in Australia that I can access to develop a scraping tool?
Thank you for this video. Is there a chance to scrape following/followers contact information?
This code does not work getting error after this section if __name__ == '__main__':
main()
pprint(output)
NameError Traceback (most recent call last)
in
1 if __name__ == '__main__':
----> 2 main()
3 pprint(output)
NameError: name 'main' is not defined
syntax error brother pay attention cuz in coding even a point is an error .u r saying if _name_ == '__main__': but correct syntax is __name__ double underscore(_) . while u used on 1 underscore. _name_ < __name__
Do i need to print(resp_body) from scrape function
To find all elements inside body
Hello is there a way to scrape certain followers with example minimum 500 followers I don’t want to scrape accounts won’t 0 followers, fake account. Any way?
Does your Code Work Now?
Hi there! Your tutorial is awesome, both methods look really great. I was trying to copy step by step the Selenium code, and I'm getting the following exception when debugging: Message: unknown error: cannot find dict 'desiredCapabilities'
I tried adding capabilities = chrome_options.to_capabilities(), and also adding desired_capabilities= capabilities below run_on_insecure_origins; but this doesn't seem to work.
How can I solve this issue? Thanks in advance!
Hey, we're glad you enjoyed it! One of our guesses would be that you're not using Webdriver-manager. If not, here are some resources on how to set it up:
www.selenium.dev/documentation/webdriver/getting_started/install_drivers/#1-driver-management-software
github.com/SergeyPirogov/webdriver_manager
Another guess - your software might need to be updated. The versions we've used are: Webdriver-manager = 3.2.2; Selenium = 4.1.0; You can always check if your browser is updated too. Let me know how it goes!
Can VPN help for IP Address?
Thanks for your question! VPNs change your IP address, but it won't necessarily work for web scraping and social media platforms. The main issue is that VPNs aren't trying to hide you're using them. Thus, it's way easier to get detected using a VPN than proxies. Here's a video that explains it in more detail: ruclips.net/video/AwKXkkEQ4ms/видео.htmlsi=N4yG9pbURHCVCyG0
from where i get the software?
Hi:) First, get yourself a Python package: www.python.org/downloads/
& JSON (library): pypi.org/project/jsonlib/
For scraping with Selenium, get the following tools:
Selenium: www.selenium.dev/
WebDriver: chromedriver.chromium.org/downloads
Selenium-Stealth: pypi.org/project/selenium-stealth/
For scraping with Requests:
Requests library: pypi.org/project/requests/
Random (library): pypi.org/project/random2/
Is this for Instagram comment?
This code is for scraping public Instagram data, so mostly for posts or anything that's visible without logging in.
@@Proxyway So this can be used for Instagram comments. Got it. Can this code be used for replies on comments too?
Isn''t this against instagrams Terms of Use?
Scraping public data is permitted. So as long as you're not getting private details or copyrighted material, and not scraping behind a login (that's how you make sure data is public), you should be ok.
@@Proxyway Got it. Thanks. The login thing is what makes a huge difference
My selenium code is giving me the error
Hi, we're sorry to hear that. Could you tell me more about what you're trying to scrape? And about your code, please describe what it looks like. And whether you copied it step-by-step?
@@Proxyway Hi! Thank you for video! Your code gives this error: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
How can it be fixed?
@@igoweiqibaduk8283 Same here
Would it be possible to scrape all unique usernames who have used a specific hashtag?
Ex: I want to know:
1. all unique usernames
2. who have used the hashtag #proxyway
3. Since Jan 1st 2022
Where would I start to try and do something like that?