Web Scraping Ninja
Web Scraping Ninja
  • Видео 148
  • Просмотров 185 704
Selenium in Google Colab; Simple Selenium Project in Python | Create Graph in google sheets
Want to take your Google Sheets visualizations to the next level?
This video tutorial dives deep into creating dynamic charts using Python's Selenium library and Google Colab.
We'll guide you through the process step-by-step, from:
Setting up your Google Colab environment
Importing essential Python libraries like Selenium
Interacting with Google Sheets using Selenium commands
Building interactive charts that update with your data
By the end of this video, you'll be able to:
Automate chart creation in Google Sheets using Python
Generate dynamic charts that react to changes in your data
Impress your audience with visually compelling presentations
This video is perfect for:
Data analysts who want to str...
Просмотров: 648

Видео

Selenium in Google Colab Tutorial For Beginners: Web scraping To Google Sheets
Просмотров 5 тыс.8 месяцев назад
In this tutorial you will learn about selenium automation and web scraping in google colab, the data collected will be save in google sheets by authenticating the google credentials in a very simple and easy steps. 00:00 Introduction 01:00 Install Selenium on Google Colab 06:08 Scrapie Website Using Selenium in Colab 14:40 Write Google sheets from Google Colab 22:44 Summary
Write Dictionary and List to CSV and Json Files | Python Jupyter Notebook
Просмотров 1399 месяцев назад
If you are struggling to save your scraped data to csv and json files, in this video i will walk you through each steps, In First part of the video you will learn how you can use csv and pandas libraries to save a csv file First you will learn how to save simple string with comma separated values to a csv file using context manager, after this you will learn how to import csv library and pandas...
These Web Scraping Proxy APIs will BLOW YOUR MIND 🤯
Просмотров 30710 месяцев назад
These Web Scraping Proxy APIs will BLOW YOUR MIND 🤯
How to Run Scrapy on Jupyter Notebook and Crawl a Website to CSV
Просмотров 2,1 тыс.Год назад
How to Run Scrapy on Jupyter Notebook and Crawl a Website to CSV
Python scrapy on google colab real quick.. part-1
Просмотров 130Год назад
Python scrapy on google colab real quick.. part-1
Web Scraping 2.0: Scrapy & Google Colab - The Future of Web Scraping
Просмотров 4,4 тыс.Год назад
Web Scraping 2.0: Scrapy & Google Colab - The Future of Web Scraping
Scrapy Tutorial: Python Web Scraping
Просмотров 1,8 тыс.Год назад
Scrapy Tutorial: Python Web Scraping
Roadmap for Python Web Scraping: Beginner to Pro
Просмотров 402Год назад
Roadmap for Python Web Scraping: Beginner to Pro
Web Scraping 101: 6 Steps
Просмотров 210Год назад
Web Scraping 101: 6 Steps
Top 19 Highest Paying Skills in 2023
Просмотров 99Год назад
Top 19 Highest Paying Skills in 2023
Selenium Automation in Jupyter Notebook: Extracting Google Search Results
Просмотров 2,7 тыс.Год назад
Selenium Automation in Jupyter Notebook: Extracting Google Search Results
Get started with Playwright in Google Colab
Просмотров 1,5 тыс.Год назад
Get started with Playwright in Google Colab
Complete Python Web Scraping Roadmap | 2023
Просмотров 938Год назад
Complete Python Web Scraping Roadmap | 2023
Beginners Tutorial: Selenium and Python on Your Mobile Phone
Просмотров 6 тыс.Год назад
Beginners Tutorial: Selenium and Python on Your Mobile Phone
Introduction to Proxies for Web Scraping and Automation
Просмотров 350Год назад
Introduction to Proxies for Web Scraping and Automation
Create Python API with Scraped Data in 30 minutes
Просмотров 1,2 тыс.Год назад
Create Python API with Scraped Data in 30 minutes
Upload Your Prize Bond Numbers & Get Notified of Wins with Our Website
Просмотров 268Год назад
Upload Your Prize Bond Numbers & Get Notified of Wins with Our Website
How to Run Selenium on Google Colab Notebook: Fixing the "Unexpectedly Exited" Error (status code 1)
Просмотров 12 тыс.Год назад
How to Run Selenium on Google Colab Notebook: Fixing the "Unexpectedly Exited" Error (status code 1)
Python Selenium Tutorial in Hindi: Dynamic Websites Scraped
Просмотров 6 тыс.Год назад
Python Selenium Tutorial in Hindi: Dynamic Websites Scraped
Web Scraping Ebay Products
Просмотров 2,8 тыс.Год назад
Web Scraping Ebay Products
Tutorial: Selenium Automation on Google Colab Notebook
Просмотров 28 тыс.Год назад
Tutorial: Selenium Automation on Google Colab Notebook
Freelancer to Software Company Office
Просмотров 78Год назад
Freelancer to Software Company Office
Automate Web Scraping with Selenium & Jupyter Notebook | Tutorial & Examples
Просмотров 4,8 тыс.Год назад
Automate Web Scraping with Selenium & Jupyter Notebook | Tutorial & Examples
How to Earn Money Online in Pakistan
Просмотров 1762 года назад
How to Earn Money Online in Pakistan
google.com
Просмотров 382 года назад
google.com
Complete Python Web Scraping Tutorial: Scrapy, Selenium, Requests, BeautifulSoup and lxml
Просмотров 6 тыс.2 года назад
Complete Python Web Scraping Tutorial: Scrapy, Selenium, Requests, BeautifulSoup and lxml
Automate Your Excel Tasks With Python: Simple steps
Просмотров 2432 года назад
Automate Your Excel Tasks With Python: Simple steps
Scrapy Tutorial: Learn How to Build a Web Crawler with Examples
Просмотров 1,3 тыс.2 года назад
Scrapy Tutorial: Learn How to Build a Web Crawler with Examples
Selenium Automation In Jupyter Notebook: Step By Step Guide
Просмотров 12 тыс.2 года назад
Selenium Automation In Jupyter Notebook: Step By Step Guide

Комментарии

  • @Kamleshparidwal2011
    @Kamleshparidwal2011 7 дней назад

    Plz reply

  • @Kamleshparidwal2011
    @Kamleshparidwal2011 7 дней назад

    Sir aap python expert ho to plz kuch help kr dijiye muje Main web automation ki script bna rha hu to ek jagah pr atka hua hu 2 din se plz help 🙏 Ek Web field me date nhi dal pa rha hu us field se interact to ho rha hai python code but calendar ka pop open hi rh raha hai date select nhi horhi

    • @ScrapingNinja
      @ScrapingNinja 6 дней назад

      Popup is probably in so try switching to , look for examples online and your problem will be resolved

    • @Kamleshparidwal2011
      @Kamleshparidwal2011 6 дней назад

      @@ScrapingNinja thankyou sir main try krta hu

    • @Kamleshparidwal2011
      @Kamleshparidwal2011 День назад

      @@ScrapingNinja sir help no there there are open a calendar 🗓️ widget

    • @Kamleshparidwal2011
      @Kamleshparidwal2011 День назад

      @@ScrapingNinja kya main code beju sir

  • @syedsaifulhuda6584
    @syedsaifulhuda6584 15 дней назад

    Assalamualaikum sir Maine apki bhi videos dekhein or bht sary or RUclipsrs ki bhi Kuch samjh nhi arha or Jo main videos dekh ky code perform kar rha hu woh work nhi kr rhy to kindly mujhy Thora guide krein mujhy ek month hogea mujhy samjh nhi arha mujhsy contact karlein please

    • @ScrapingNinja
      @ScrapingNinja 15 дней назад

      Ws, some websites change over time so the exact code doest always work, you need to use the techniques and terms and apply them in your context instead of just copy pasting the code…

    • @syedsaifulhuda6584
      @syedsaifulhuda6584 15 дней назад

      @ScrapingNinja nhi Matlab bilkul copy paste nhi kar rha hu lekin jesa main bs4 ko use kr rha hu code run nhi kr rha nah response code nah hi request.get url main Kuch run horaha hai terminal main error arha hai

    • @syedsaifulhuda6584
      @syedsaifulhuda6584 15 дней назад

      Agr possible hoto kia hum connect hoskhtay hain via phone or meeting mujhy concepts clear krnay hain thoray to please?

  • @sunnyy6295
    @sunnyy6295 24 дня назад

    Requests pn first really?? What about dynamic content loading then. U need some headless browser scrapping

    • @ScrapingNinja
      @ScrapingNinja 21 день назад

      Most of the dynamic content is retrieved by requests using their ajex request

  • @PrasunKhan1983
    @PrasunKhan1983 Месяц назад

    I get an error while running !apt-get install chromium chromium-driver The error message is Reading package lists... Done Building dependency tree... Done Reading state information... Done Note, selecting 'chromium-chromedriver' instead of 'chromium-driver' Package chromium is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source However the following packages replace it: chromium-bsu E: Package 'chromium' has no installation I am running Colab in Chrome browser on an Android phone.

    • @ScrapingNinja
      @ScrapingNinja Месяц назад

      chromium-driver is installed and it installs chromium as well so you can start using it, ignore the error

  • @aarizmansuri9482
    @aarizmansuri9482 Месяц назад

    Web development or scraping wich best ?? Best sallary packege

    • @ScrapingNinja
      @ScrapingNinja Месяц назад

      Both are interrelated learn web development and you can use that knowledge for web scraping..

  • @starlight5531
    @starlight5531 Месяц назад

    i did everything same as you but my csv file is empty, why ?

    • @ScrapingNinja
      @ScrapingNinja Месяц назад

      Can you see scraped data in your out logs?

  • @alpha-pubgmobile7304
    @alpha-pubgmobile7304 Месяц назад

    WebDriver.init got an unexpected keyword argument 'executable_path' help pls

  • @gregoriyepez3492
    @gregoriyepez3492 Месяц назад

    What do You think about brightdata?

    • @ScrapingNinja
      @ScrapingNinja Месяц назад

      Brightdata is proxy service, they are very reliable and one of the best in business.

  • @100busan
    @100busan Месяц назад

    I am from Korea. I love you, Guy

  • @AnuragSharma-c2z
    @AnuragSharma-c2z Месяц назад

    I want to do free lancing

    • @ScrapingNinja
      @ScrapingNinja Месяц назад

      Get a skill, no matter major or minor and get started

  • @rajanair79
    @rajanair79 2 месяца назад

    Appreciate you video, but Unfortunately i couldnt learn anything as you were just writing the code and didnt explain what all those code mean and when to write what

  • @allah-swt
    @allah-swt 2 месяца назад

    Thanks a lot Mate, literally spent hours with ChatGPT trying to resolve the error in Selenium. It is always better to start off with a working piece of code rather trying to generate entire code from scratch.

  • @thupham4139
    @thupham4139 2 месяца назад

    I got an error.ReactorNotRestartable(). The Crawler Process doesn't seem to allow rerunning a spider in the same notebook multiple times. Have you tried any workaround?

    • @ScrapingNinja
      @ScrapingNinja 2 месяца назад

      In async you need to add task to existing task group i haven’t tested it yet but you can try adding task in the loop.

    • @thupm2601
      @thupm2601 2 месяца назад

      @@ScrapingNinja I tried !scrapy runspider spidername. It works and allows rerunning it.

  • @MeryQuaidKiBaatSunain
    @MeryQuaidKiBaatSunain 2 месяца назад

    What is filter function ?

    • @ScrapingNinja
      @ScrapingNinja 2 месяца назад

      Filter function is to filter your existing prizebonds if you want to search for specific prozebond in the list to delete or see etc.

  • @MuhammadSalman-uz3ug
    @MuhammadSalman-uz3ug 2 месяца назад

    GitHub account nahi hai

  • @oddocid9734
    @oddocid9734 2 месяца назад

    I dont see many channels with the dame quality of content. Good work!

  • @brutekerb4582
    @brutekerb4582 2 месяца назад

    debir haber visto este tutorial de primeras muchas gracias thank you so much

    • @ScrapingNinja
      @ScrapingNinja 2 месяца назад

      I am glad you find this helpful.

  • @annmariajoseph4298
    @annmariajoseph4298 2 месяца назад

    cant find the second code

    • @ScrapingNinja
      @ScrapingNinja 2 месяца назад

      Follow this video.. Tutorial: Selenium Automation on Google Colab Notebook ruclips.net/video/mOJiWrjFVKY/видео.html

  • @ThEKing-anon
    @ThEKing-anon 3 месяца назад

    Commands were missing I found it and it worked. Thanks to God.

    • @ScrapingNinja
      @ScrapingNinja 2 месяца назад

      Follow this video.. Tutorial: Selenium Automation on Google Colab Notebook ruclips.net/video/mOJiWrjFVKY/видео.html

  • @claudiatalaveraabajas4481
    @claudiatalaveraabajas4481 3 месяца назад

    I can't find the second command on the link you provided, thank you!

    • @ScrapingNinja
      @ScrapingNinja 3 месяца назад

      Follow this video.. Tutorial: Selenium Automation on Google Colab Notebook ruclips.net/video/mOJiWrjFVKY/видео.html

    • @annmariajoseph4298
      @annmariajoseph4298 2 месяца назад

      me too

  • @puteriananditya1935
    @puteriananditya1935 3 месяца назад

    i use this to scrape google maps reviews, but it return in chinese 😵‍💫, also does Google Colab really not display pop ups?

    • @ScrapingNinja
      @ScrapingNinja 3 месяца назад

      Open whatsmyip website in colab and check where your colab ip is located, colab runs on servers and yours probably located in china.

  • @austinbright-j3o
    @austinbright-j3o 3 месяца назад

    Can I scrape sites using this that use captchas

    • @ScrapingNinja
      @ScrapingNinja 3 месяца назад

      You will need to use proxies to avoid captcha.

    • @austinbright-j3o
      @austinbright-j3o 3 месяца назад

      @@ScrapingNinja I’m new to code. Can I do this in Google colab? Does it cost money to implement?

  • @josemariomonteiro3628
    @josemariomonteiro3628 4 месяца назад

    SessionNotCreatedException: Message: session not created exception: Missing or invalid capabilities (Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 6.1.85+ x86_64)

    • @ScrapingNinja
      @ScrapingNinja 4 месяца назад

      Check your code to make sure you have set compatibilities correctly

    • @ScrapingNinja
      @ScrapingNinja 4 месяца назад

      Options

  • @Programmingworld-g9m
    @Programmingworld-g9m 4 месяца назад

    Bro make a video on chrome webdrivers

    • @ScrapingNinja
      @ScrapingNinja 4 месяца назад

      Chromium and chrome are same..

  • @Nidal_C
    @Nidal_C 4 месяца назад

    Playwright

  • @vibing5562
    @vibing5562 4 месяца назад

    sir I wanted to scrap shopping website like amazon product can you pls suggest how to take there lxml code because they have given it in the form of div id,script data and mostly reactjs code finding it very difficult to scrape products

    • @ScrapingNinja
      @ScrapingNinja 4 месяца назад

      You need to adapt, first try to parse with lxml if its not present try looking for something in script tag, sometimes data is loaded as json in script if its not their either try something else like selenium/playwright.

  • @edmilsonsc1371
    @edmilsonsc1371 5 месяцев назад

    I wait for your answer friend

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      Can you please repeat the question?

  • @FaberLSH
    @FaberLSH 5 месяцев назад

    Thank you so much. You tutorial helped me a lot!!

  • @JP-vo8ik
    @JP-vo8ik 5 месяцев назад

    SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 114 Current browser version is 126.0.6478.126 with binary path /usr/bin/google-chrome error

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      You need to update chromedriver using apt-get

  • @mariana_anst
    @mariana_anst 5 месяцев назад

    Unfortunately couldn’t pass from step on 5:18, the command driver = web_driver() didn’t work. Just giving up on trying to use selenium on colab. Visual Code has been better to me…

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      Can you share the error?

    • @allah-swt
      @allah-swt 2 месяца назад

      It works perfectly fine on Colab. I'm using the selenium on colab.

  • @progammier_official
    @progammier_official 5 месяцев назад

    It doesn't work without shell command maybe😢

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      It does work without shell commands now, follow this Tutorial: Selenium Automation on Google Colab Notebook ruclips.net/video/mOJiWrjFVKY/видео.html

  • @yudaperwira6307
    @yudaperwira6307 5 месяцев назад

    how to stop scraping until certain page? like until 25 page to stop? please reply me

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      you can try different logics here for example: if you are looping over pages you can break, If you are clicking on next page, check if page=25 in next_page_link

  • @bharatsoni9758
    @bharatsoni9758 5 месяцев назад

    bhai kha se ho aap?

  • @alifurRahman-xn9ns
    @alifurRahman-xn9ns 5 месяцев назад

    music background me mat lagao disturb hota hai

  • @Asuna-bn1zv
    @Asuna-bn1zv 5 месяцев назад

    no me funcilno

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      Make sure to include options, it wont work without options.

  • @dakshbhatnagar
    @dakshbhatnagar 5 месяцев назад

    This is still relevant in 2024. Definitely a helpful vid.; Btw what to do if the same is to be done locally? I am on MacOS

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      Thanks man, and I think you can, i will check

  • @SelfBuiltWealth
    @SelfBuiltWealth 5 месяцев назад

    Thank you so much it legit worked you are soo underrated god bless you❤❤

  • @BrighterMorning
    @BrighterMorning 5 месяцев назад

    Nice video Nailed the point

  • @TrendBinder112
    @TrendBinder112 5 месяцев назад

    very low voice

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      Thanks for feedback 👍👍

  • @ilvhimekazain
    @ilvhimekazain 5 месяцев назад

    bro, try using this method again to scrape all the product review components on the Lazada website, when I tried it like in the video and just adjusted what I wanted to scrape by copying the inspection results and then running the code, the results didn't appear like at minute 08:21

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      make sure you are using correct xpath and also try to print page source to see if the correct page is being loaded

  • @animeshroy7301
    @animeshroy7301 6 месяцев назад

    i am getting this error "'str' object has no attribute 'capabilities'" please help

    • @ScrapingNinja
      @ScrapingNinja 6 месяцев назад

      Check to make sure you have correctly imported options and its not a string

  • @realitywords-17398
    @realitywords-17398 6 месяцев назад

    Hey Bro! I am a beginner and I am getting different errors. Nobody taught me to handle those errors (particularly IP blocking) and I am getting different responses. Can you help me?

    • @ScrapingNinja
      @ScrapingNinja 6 месяцев назад

      what error you are getting? share error message here

    • @realitywords-17398
      @realitywords-17398 6 месяцев назад

      @@ScrapingNinja Bro I am talking about general problems which every person encounters such as captchas, IP blocking etc. Kindly make a video regarding how to prevent IP blocking for beginners using tools such as selenium and bs4 and show bypassing in a easy way! Also please make a video on how to bypass bot detection and captchas. I have searched for it everywhere, I am getting the related tutorials but of no use because I am a beginner and they aren't teaching for beginners. So waiting for your kind response.

    • @2ru2pacFan
      @2ru2pacFan 4 месяца назад

      ​@@realitywords-17398 Use a proxy server to avoid it.

  • @aidenstyle8604
    @aidenstyle8604 6 месяцев назад

    I am from Thailand. I love you, Guy

    • @ScrapingNinja
      @ScrapingNinja 6 месяцев назад

      Thank you so much dear.. ❤️

  • @LeitordeMapa
    @LeitordeMapa 6 месяцев назад

    Perfect, you saved my day!

    • @ScrapingNinja
      @ScrapingNinja 6 месяцев назад

      Glad I could help! don't forget to subscribe.

    • @LeitordeMapa
      @LeitordeMapa 6 месяцев назад

      @@ScrapingNinja You earned it, for sure

  • @chomchomeizy4098
    @chomchomeizy4098 6 месяцев назад

    Thank you so much for this video. I wasted 24 hours in futility trying to find a way to get Selenium to work on Colab; TILL i found your video, and bam! It worked. I had errors with the executable path, so i had to remove it completely and passed only the options argument. ❤

    • @ScrapingNinja
      @ScrapingNinja 6 месяцев назад

      You are welcome, 🤗 I am glad you find this video useful, don’t forget to subscribe.

  • @sweela1
    @sweela1 6 месяцев назад

    AssalamuAlaikum buhat acha sikha rahe aap, Sir saath he notebook ko github main push karna bhi btadia krain n main VS code pr Selenium se chrome k latest version jo 124 ya 126 hai tu usk saath web scrapping karna chahri tu kindly uska bhi btadain

    • @ScrapingNinja
      @ScrapingNinja 6 месяцев назад

      ws, thanks you cand ownload github desktop to clone with github and push you files and as far as web scraping using selenium in concerned method is same regardless of version. you can watch video on my channel related to web scraping using selenium

    • @sweela1
      @sweela1 6 месяцев назад

      ​@@ScrapingNinja Sir chrome k latest version k drivers available nahin hain wahan jo aapne website btayi hai n secondly main iss video main 48:00 pr aakr stuck hogai hun q k main VS Code use krti hun n aap pycharm use karre hain, I am a beginner tu VS Code ka btadain k kese hoga

    • @ScrapingNinja
      @ScrapingNinja 5 месяцев назад

      You are struggling with scrapy shell part?

  • @Arhambadboy238
    @Arhambadboy238 7 месяцев назад

    Good 👍 information

  • @MuhammadIsmail-jj1kk
    @MuhammadIsmail-jj1kk 7 месяцев назад

    Hi friend, I can't find more videos of this tutorial selenium bot. It will be more easy to learn if you make set of tutorial by numbers like class1 class 2 etc. Thanks and reply to where i can find more videos about this tutorial

    • @ScrapingNinja
      @ScrapingNinja 7 месяцев назад

      I have some videos i will make playlist for you.

    • @MuhammadIsmail-jj1kk
      @MuhammadIsmail-jj1kk 7 месяцев назад

      Thanks, brother. I will be thankful if you make it