Perfectly explained! Just note that if you are receiving errors with Webdriver.Chrome not having an execuatable_path argument, simply omit this argument and run with the options only ( driver = webdriver.Chrome(options=options) ). New versions of selenium do not support this argument and it is not necessary to be included.
Thank you so much for this video. I wasted 24 hours in futility trying to find a way to get Selenium to work on Colab; TILL i found your video, and bam! It worked. I had errors with the executable path, so i had to remove it completely and passed only the options argument. ❤
Try to follow step by step, double check the xpath, make sure you are printing the output, also update the xpath to make sure it’s upto date with website.
@@ScrapingNinja Please help, I got the same error, I wrote exactly like your tutorial. Thank you very much. here is the code: options = webdriver.ChromeOptions() options.add_argument('--no-sandbox') options.add_argument('--headless') options.add_argument('--disable-gpu') options.add_argument('--disable-dev-shm-usage') d = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver', options = options) error: TypeError Traceback (most recent call last) in () 6 options.add_argument('--disable-dev-shm-usage') 7 ----> 8 d = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver', options = options) TypeError: WebDriver.__init__() got multiple values for argument 'options'
I have followed the steps but it says webdriver__init__() got unexpected keyword argument ‘executable_path’. I even tried using another notebook but same
@@ScrapingNinja I tried it but the hard part is to getting the right div id or class. I am not getting any result when I do something like this. job_descriptions = driver.find_elements(By.XPATH, '//div[@class="jobsearch-jobDescriptionText"]')
Make sure the content is loaded before you search for this. Also in inspect element press ctrl+f and past your xpath “without inverted commas” there and it will show you if your xpath is correct and locats the element
nameerror is mostly because you forgot to define something. It's cause by undefined variable or you miss to import library or incorrect recall variable/library
Thanks you so much, your tutorial is straightforward. I have a question, now -I installed chromium-browser using !apt-get install -y chromium-browser. -Then I run !ls/usr/lib/, however I could not find chromium-browser in the list of libraries, is it normall, could i find it in other directory ?
Perfectly explained! Just note that if you are receiving errors with Webdriver.Chrome not having an execuatable_path argument, simply omit this argument and run with the options only ( driver = webdriver.Chrome(options=options) ). New versions of selenium do not support this argument and it is not necessary to be included.
thanks for the input Soroush Taheri, i completely agree with you...
@@ScrapingNinja All thanks to you, sir! 🙌
Thank You!
Thank you so much, sir. You saved me! :)
@@omatematicoinvestidor you're welcome, sir! I'm glad it was helpful
Thank you so very much. After 9 hours of wasting time on google, I found this video. Keep making more videos bro. Great help
You are welcome Lochana i am glad you found this video useful, don’t forget to subscribe
Thank you so much for this video. I wasted 24 hours in futility trying to find a way to get Selenium to work on Colab; TILL i found your video, and bam! It worked. I had errors with the executable path, so i had to remove it completely and passed only the options argument. ❤
You are welcome, 🤗 I am glad you find this video useful, don’t forget to subscribe.
This is still relevant in 2024. Definitely a helpful vid.; Btw what to do if the same is to be done locally? I am on MacOS
Thanks man, and I think you can, i will check
im getting this error: ls: cannot access '/usr/lib/chromium-browser/chromedriver/': Not a directory. There no files like you as in 5:42
Where you are getting this error? Which line of code?
@@ScrapingNinja im executing on google colab
You can completely remove the executable path argument
excuse me, i wanna ask.. why the data is not come out. i mean all code is run executed green.. but the output is wont go out.. ?? any answer please
Try to follow step by step, double check the xpath, make sure you are printing the output, also update the xpath to make sure it’s upto date with website.
Hi! How should i tackle the problem that says' execuatable_path is not defined'
If you are using the colab you dont need to use this and if you aren’t you have typo its supposed to be executable_path
Great video.I has been 2 days I have been looking for this solution. JajakAllah Khairan.
I am glad you find this helpful don’t forget to subscribe..
@@ScrapingNinja already did
@@ScrapingNinja It would be of great help, if you give tutorials about social media scraping (twitter, facebook, Instagram etc)
good idea...
superr bro its working i have searched lot of videos it not working in google colab your code is working bro thanks
You are welcome 🙏 i am glad you liked the video..
Please help, I keep getting this issue: TypeError: WebDriver.__init__() got multiple values for argument 'options'
I have selenium 4.12.0
you are probably passing more than one arguments the options are supposed to be like this.. webdriver.Chrome(options=options)
@@ScrapingNinja Please help, I got the same error, I wrote exactly like your tutorial. Thank you very much.
here is the code:
options = webdriver.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--disable-dev-shm-usage')
d = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver', options = options)
error:
TypeError Traceback (most recent call last)
in ()
6 options.add_argument('--disable-dev-shm-usage')
7
----> 8 d = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver', options = options)
TypeError: WebDriver.__init__() got multiple values for argument 'options'
Hi Duyen your d=webdriver part is not correct please check the video or blog again
I have followed the steps but it says webdriver__init__() got unexpected keyword argument ‘executable_path’. I even tried using another notebook but same
watch ruclips.net/video/VSO14hgo6Gs/видео.html its latest.. new version doesn't need executable_path
This is awesome! Thank you :) Could you make a tutorial how to get job description info from indeed?
Thanks.. 🙏 i will try to make one but you can do it yourself by using either selenium or playwright and get the details from any website.
@@ScrapingNinja I tried it but the hard part is to getting the right div id or class. I am not getting any result when I do something like this.
job_descriptions = driver.find_elements(By.XPATH, '//div[@class="jobsearch-jobDescriptionText"]')
Make sure the content is loaded before you search for this. Also in inspect element press ctrl+f and past your xpath “without inverted commas” there and it will show you if your xpath is correct and locats the element
i use this to scrape google maps reviews, but it return in chinese 😵💫, also does Google Colab really not display pop ups?
Open whatsmyip website in colab and check where your colab ip is located, colab runs on servers and yours probably located in china.
Sir, thanks a lot for making this video. I've one query that can I use google colab in my android phone and then use selenium. Will it work ?
Yes it should work fine because even if you are using it on mobile the backend os is same..
Thank you so so much. So clear, so concise. Google documentation sucketh. Hard.
You are welcome.. :) don't forget to subscribe
i am getting this error "'str' object has no attribute 'capabilities'"
please help
Check to make sure you have correctly imported options and its not a string
How about headfull!?
Colab is linux based environment so headful is not going to work.
Thank you very much!!
You're welcome! don't forget to subscribe
excelent. iam brasil. Is possible to have interactions by elements of page, click, write, etc... for selenium colab?
Yes all the tasks are possible as we do in selenium..
NameError: name 'webdriver' is not defined
You need to import webdriver first
from selenium import webdriver
nameerror is mostly because you forgot to define something. It's cause by undefined variable or you miss to import library or incorrect recall variable/library
Is ka solution kia hai?
dont work no more it says version is outdated
you need to install again.. follow steps in new notebook
nice. thank you so much
You are most welcome, don't forget to subscribe
Driver.maximize window means, it has to pop a new window but it's not showing
its to maximize the current window
@@ScrapingNinja but it's not opening the window, a new window has to appear, Like a new chrome page
to open you need to
driver = webdriver.Chrome()
Where is the link?
Hi Nicolas which link?
@@ScrapingNinja the notebook
this the link colab.research.google.com/, got to files and click new notebook
Thanks you so much, your tutorial is straightforward. I have a question, now
-I installed chromium-browser using !apt-get install -y chromium-browser.
-Then I run !ls/usr/lib/,
however I could not find chromium-browser in the list of libraries, is it normall, could i find it in other directory ?
It could be at another directory try
!which chromium-browser
and it will tell you if its installed and where
Excellent.
Thank you! Cheers! don't forget to subscribe
Executable_path error is showing
Follow the code step by step if you still get the error create new colab notebook and follow the steps again.
Thanks a lot. +1 like +1 subs
Thanks 🫡