Selenium Course for Beginners - Web Scraping Bots, Browser Automation, Testing (Tutorial)
HTML-код
- Опубликовано: 24 июл 2024
- Learn Selenium by building a web scraping bot in Python. Selenium is a powerful web automation tool that can be used for browser automation, to test front-end code, and create web scraping bots.
💻 Code: github.com/jimdevops19/Seleni...
✏️ Course developed by JimShapedCoding. Check out his channel: / @jimshapedcoding
⭐️ Additional resources ⭐️
🔗 Python Download: www.python.org/downloads
🔗 Pycharm Download: www.jetbrains.com/pycharm/dow...
🔗 Selenium Documentation: selenium-python.readthedocs.io/
🔗 Copied and Pasted during the video:
‣ www.seleniumeasy.com/test/jqu... (Section 1&2)
‣ www.seleniumeasy.com/test/bas... (Section 3)
🔗 Chromedriver download website: chromedriver.storage.googleap...
🔗 All the Explicit waits: selenium-python.readthedocs.i...
🔗 My Personal website for code snippets: jimshapedcoding.com
⭐️ Course Contents ⭐️
⌨️ (0:00:00) Getting Started with the basics
⌨️ (0:16:44) Explicit vs Implicit
⌨️ (0:28:11) Sending Keys & CSS Selector
⌨️ (0:43:42) Structure a Bot Project
⌨️ (1:03:13) Deal Searching
⌨️ (1:44:38) Booking Filtrations
⌨️ (2:07:24) Execution from a CLI
⌨️ (2:21:31) Deal Reporting
🎉 Thanks to our Champion and Sponsor supporters:
👾 Wong Voon jinq
👾 hexploitation
👾 Katia Moran
👾 BlckPhantom
👾 Nick Raker
👾 Otis Morgan
👾 DeezMaster
👾 AppWrite
--
Learn to code for free and get a developer job: www.freecodecamp.org
Read hundreds of articles on programming: freecodecamp.org/news
Third time is a charm!
Thanks FCC, I am glad to share this with the best coding community. Hope everyone will have a great time learning Selenium! :)
Your work is *COMMENDABLE* I appreciate your *EFFORTS*
Congrats Jim🎉
@JimShapedCoding I already sub your channel from last 8months
sadly wont listen to 3 hour video! :(
You guys are doing history here by democratizing high quality education for free!! 🙏✅
for those who don't know,
the newer version of selenium has removed the driver.find_element_by_id and similar functions, the new way of writing the same is-
driver.find_element("id" , "")
hope that helps
Thank you for the help, I just want to ask you, where did you find this info? I can't find a new documentation for selenium.
Thank you , really appreciate for this information! :3
@@yassinedghoughi9500 python gives warning when you use old version of code.
How can we extract a word with a tag of yt- formatted -string
And im scraping data by looping through 50 yt channels. Sometimes it stops looping with an error of unable to locate
But when im executing them individually all 50 channels are able to scrape well. What might be wrong
its more like beautifulsoup now :D
9:15 - r as the prefix to a string indicates a "raw string"; that is, character sequences like
will be treated as literals and not as escaped control characters (as they normally are in a string.) It's useful when you expect your string to include backslashes, for instance Windows paths, or certain regex expressions.
Thanks you
Many thanks for this amazing tutorial.
Not only was it great intro to OOP, it also provided enough info to build my own app from scratch:
Automated my timesheet entries at work. It now takes no more than 2 seconds to open browser, login, navigate to desired page, enter all work and time related values, and to pass the submit process with approval from management :)
Feeling good while conscious of how much more there is to do and to learn :D
I am glad I chose this video to learn selenium scraping. I learned a lot from you!. You are really a great teacher. Hats off to you!
Just started using Selenium and this will save me a bunch of time trying to figure out all the methods, thanks!
greaaaattttttttt, just finished valentin's postman crash course and willing to learn automatization for testing. gj
Great video, to the point and explaining the details for those who are unfalmiliar with Python. !
One of the best and most complete tutorials, thank you for this.
Glad to read this :)
use this for browser to not automatically closing:
`
options = webdriver.ChromeOptions()
options.add_experimental_option("detach", True)
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
`
Hey, how can I put it in code from "structure a bot project" (as a part of the booking module)?
Thanks a lot!
thanks you're amazing!
@@workoutinspirations9990 Have you found out the answer??
😅
I am also stuck there
Thank you so much guys, you guys are helping us to continue learning during these covid times. THANK YOU
God finallyyyy FCC uploaded selenium tutorial, not 1000th tutorial on react or HTML 😁 I'm very happy! Thank you lovely FreeCodeCamp ❤️ hope here will be more videos with test automation library's, such as playwright and selenium!
This is gold. Please more info about testing automation 🙏🙏🙏
great tutorial! explains everything in a newbie-friendly way...
Great content. One of the best explanations in a coding video that I've ever seen.
They are doing really a great job.😘.We all should appreciate them sothat they always get inspired by this. 😘👍.Carry on guys .😊😊
After watching the tutorial: This is one of the best tutorial for beginners, after this you will be able to scrap easily and make your own bots and can understand the documentation easily.
Thank You Jim and FCC for this amazing course 🙏
It's "scrape/scraping". "Scrapping" means something else. :)
Thanks to your coaching, now I feel more confident in my coding skill a bit, haha, thank you, Jim!
Another amazing tutorial. Thanks Jim
Great tutorial. I find a lot of interesting features in this video, which I am going to use in my automation project. Thanks.
Thank you so much for this video. I really find it very helpful and easy to follow along. I was able to build bot that is able to scrape huge data from multiple paged website and dumps the record to json while keeping track of records that has already been scraped incase the bot fails at any point it will be able to continue from where it stoped when you run it again and finally merge all dumped json files together and generate an excel sheet. But I got my basis from this video all thanks to you man. I deeply appreciate.
Awesome Content thanks Jim shapedCoding channel we wish to introduce more series in this field of python in web scraping 🥺🌸+ thanks FCC for sharing this awesome content on web scraping with us 👏👏🌸
Started listening salenium today, and this video tutorial is gem 💎. This guy is really awesome at teaching. #ThanksMuch
You guys have been dropping bangers
This guy learn me Flask and Web Scraping with Bs4 for free. THANKS JIM
PD. Your RUclips videos of your canal help me to my carrer
Almost all the tests links are dead AND the code is outdated. Selenium now uses find_elements() NOT find_by_id() or find_by_class_name() etc.
How does find_elements() work?
@@Vyzinn class = .classname, id = #id etc.
@@reklamsz3m3t a belated but sincere thank you to you :)
Can you please tell me How to find this attribute name losser-id = "name-of-loser" in selinium
@@shivamgupta-hn2nf self.find_element(By.CSS_SELECTOR,'tag[losser_id="name_of_loser]')
Here for tag,you have to find which tag this stuff is located in . Like div or a or li. (if you aren't using classes , use driver instead of self )
Jim is always a great teacher ✌🏻✌🏻
I loved his beautiful soup tutorial too 👌
brilliant... been waiting for selenium tutorial for ages
Your tutorial is amazing. One of the best coding video I ever saw on yt. There are some elements which changed in the meantime, but I could manage it by my own. It help me to build ezze a bot on another project. Thank you for that!
What?
I am not able to send_keys, can u help?
Really helpful stuff, many thanks!
Help! the selenium easy links does not work! It said: "Page Not found"
Amazing course! Thank you for share
Finally...I have been waiting for this course all my life😂😂👍🏾
Thanks for the great content. Just one thing : Regarding selenium links: the pages aren't found. Anyone could help? thanks!
Hi, the seleniumeasy demo site is no longer working
Great tutorial! Thank you!💐🎂
Hi! Can you use Expected Condition to wait after I go through login page, and then after the next page to perform web scraping on the 3rd page?
If yes, then I would have to refer to the 3rd page and methods in the Expected Condition?
the first time I watch a excellent tutorial video. thank you very much from Vietnam. wish you all the best
Great tutorial and great and clear teaching
Nicely explained man, thanks
This is the best tutorial I have ever seen. Thank you very much!
how? it's literally outdated. there's no way you learned anything useful with this.
@@Superdupafool wow
Thanks for teaching. I have learnt a lots
This tutorial is great!
I love this course ❣️
This is a great Selenium Tutorial.
Anyone else have an issue with the os.environ['PATH']? I had to add a semicolon for it to work: r";C:/SeleniumDrivers". Anyone know why it doesn't add it automatically? Thanks.
Are the seleniumeasy links working? They seem to be removed from the website.
Wow, posted 1 day ago, and fixed my exact problem. No idea why this works.
Thanks for sharing this, i didnt know i should add semicolons there.
THANK YOU FOR THIS!!!!
@@et8175 thanks man..I am searching for this..
This is an amazing tutorial thank you very much
Thanks for the video!
Amazing tutorial!
i needed this😭
the SeleniumEasy links no longer work
where was this 8 months ago
Greatest channel ever
Please keep continue this diamond tutorials thanks a lot
This is great!
Nice tutorial man
Finally a tutorial I will surely watch.
I have an issue called 'procrastiantion'. I just don't end up watching most of the vids I wanna watch...
You watched?
@@sinbad2597 hell no 😂
Please upload more of this tutorial and also teach automated data collection. Your style of teaching is nice
It was a great one for sure
And as always thank you FCC
great video learn lot from you!
hey its this guy. Love this guy.
Beautiful 🙏🏻
Thank you so much guys
It's helpful for me. thank you very much.
Great Project! Loved the accent. However, there is one thing i would request for the next selenium tutorial, I am would like to learn more about multiple browser window handling and switching. Thank you for this amazing #CrashCourse
Hi all! The seleniumeasy links are not working anymore so can anyone suggest a URL like those for us just starting? Thanks!
how can i click each element with same class name in row, like click a tab copy some content text and then return to click the second
otherwise how can i add an argument to go back to previous page using this exact code?? because when i use built in driver.back() it return an error that driver has not been defined
none of the links actually work , it says the page can’t be found .
can you please try to fix that ?
Hi
Could you provide me with the version details, what version of python you used, what version of pycharm you used, etc etc
Hi, which link is the one you mentioned in your video 44:50, which explains python project structures.
Hey, help me please !
I am trying to use default profile of Chrome (or Firefox) but i cant. I need to keep my login informations because i want keep my accounts on until i close them. But every single time when i run project it creates new profile (i think?) and my login informations disappears, my accounts closes.
I couldn't find any useful documentation or advise, please help !
Thank you, the example url of the download button is lost. And I found a similar website with download button without id, how can I do?
is there any way to see the result in terminal without opening the chrome browser?
is find elementS not better? If you find element and it doesn't find that element then it will throw an error where as if you use find elements it returns a list and it will be empty if it doesn't find anything but it won't throw an error and then you can check the list.
Does anyone know if there is a way to use selenium but on an already existing tab (so i don't have to sign in on the thing it opens)
Wow! Thanks
the links of selenium easy is not working
Why the driver still quits after using the if statement to control the .quit()?
it works until after search click.. filtration didnt work on me :(
copied word by word but it says unable to locate element
how do i fix this
this is awesome
The given link seems like not working anymore, any other links can be used to follow this tutorial?
Was getting 'chromedriver' executable needs to be in PATH despite following along with the recommendations and I figured out the issue for both Mac and Win.
First, I added a print statement right after the os.environ path setting piece:
print(os.environ['PATH'])
I noticed it was missing a ; in Windows (: in Mac). So, I modified driver_path append step to this and it worked:
os.environ['PATH'] += os.pathsep + self.driver_path
os.pathsep adds your OS specific path separator character.
There are other ways to do fix this problem, but this is what I did in a pinch to see what I could do to make it work like the teacher was doing it.
Thank you, it worked for me.
Great solution! Thanks! I figured out that simply adding a ; in front of the driver path works as well, so my line looks like
os.environ['PATH'] += r";C:/selenium_drivers"
@@MiturGrunge what is meaning of semi-colon?
Can you share you your code at this part for an example
It seems the link for the test\jquery... does not work. Would you please check it?
Thank you 😍🔥🔥
How do i setup the chrome driver for MAC OS? I am using a MAC M1 system. Need help
Thanks, Jim
if I want to implement this to a web page, that is, get the results, save them in variables and display them on my web page. it's possible?
Thank you for your video🎉
hello, the video is very good, very intuitive, however in 2023, many of the examples used are invalidated, mainly due to the attributes of the elements of the sites, such as the booking one, they were changed which made it very difficult to scrape the website and follow the progress video was very difficult because of this.
i am also stuck with that. I am unable to select currency in booking website. If you have solution for this let me know please.
Thank you very much!
Thank you!
what I dont understand is what happens if we upload this project to a server. will it still work if that server is on linux for example and they dont have chrome installed on that server. is it using the user's chrome?
9:30 I'm getting 'FileNotFoundError: [Errno 2] No such file or directory: 'chromedriver; if I use os.environ['PATH']
However if I use PATH="/Users/surajk/Pictures/Selenium/chromedriver", I'm not getting any error.
Any suggestions.
Hey, I had a similar issue. After inspecting the path I found that each path string is seperated by a ; so changing it to += ';C:\\SeleniumDrivers' may work for you. I was running in a venv though, so I had to add it to my venv site-packages. Hope that helps.
@@sootybuu2963 Had the same issue, now it works! Thank you
My chrome version is 118. something what should I do?
😫guys please.. I beg u, really u are making much efforts. Giving all these for free is not a small task.
I have a question. How to stop web browser load but not close browser or time out with robot framework?
I really love the tutorial, I have been folloeing your tutorials always and thumbs up. I have an enquiry; suppose I want selenium to be refreshing a specific website at an interval of 2 seconds without stoppind, and if a certain link pop ups on that website, click on it. What is the procedure for that?
Videos on this channel are just free gold mines, can’t believe you can get so much knowledge for free.
"Thank you sincerely for creating and sharing the video. I'm incredibly grateful for the wealth of knowledge I gained about coding through your content. The concepts I learned, particularly finding new locators and changing the codes, have provided me with valuable opportunities to put my skills to the test. I can't express enough how much I appreciate your efforts in helping me expand my coding abilities. Thank you wholeheartedly once again!"