The Ultimate Guide to Selenium Bots - Automating Twitter
HTML-код
- Опубликовано: 2 июн 2024
- In this video we will explore the art of bot making for the purpose of understanding how it's being used against us 🤖🤖🤖
We will create a simple retweeting bot that searches for posts that contain a certain keyword like: "cat" or "dog" and then retweets selected posts with a quoted message.
The code and the lesson-flow I will demonstrate does not contain any unethical practices: we use a single account and we automate it for harmless purposes. However the techniques you will learn can be applied on different use cases ☢️☢️ so PLEASE use this information responsibly!!! ☢️☢️
-----------------------------------------------------------------
⭐ THANK YOU TO THE SPONSORS ⭐
-----------------------------------------------------------------
This video is brought to you by Bright Data, an innovative web platform that collects and structured web data so that you don't have to!
⭐⭐⭐ Check them out right now: brightdata.grsm.io/pythonsimp...
-----------------------------------------------------------------
🎬 RELATED TUTORIALS OF MINE 🎬
-----------------------------------------------------------------
⭐ Anaconda Beginners Guide (Install Python):
• Anaconda Beginners Gui...
⭐ Simple Web App with Flask (HTML introduction):
• Simple Web App with Fl...
⭐ Advanced Web App with Flask (HTML introduction):
• Flask SQLite Web Appli...
⭐ Python For Loops for Beginners:
• Python For Loops - Pro...
-----------------------------------------------------------------
⏰ TIME STAMPS ⏰
-----------------------------------------------------------------
00:00 - intro
00:52 - what is a bot?
02:25 - dummy account
03:25 - web driver set up
04:37 - Selenium set up
05:25 - Jupyter Notebook set up
06:17 - initialize web driver object
07:38 - fix Session Not Created Exception
09:24 - quick introduction to HTML (1 sentence)
09:47 - find element by tag name
11:04 - Developer Tools and how to use it
12:46 - find elements by XPATH
14:12 - why Jupyter Notebook is the best editor for automation
14:30 - continue log in automation
16:26 - search keyword shortcut
17:46 - retweet automation
19:23 - quote tweet automation
20:44 - find elements that contain certain data
24:24 - pause code execution with time.sleep()
25:22 - combine all notebook cells and review code
28:13 - BrightData
29:30 - retweet multiple tweets at once
30:50 - scroll to the bottom of the page (to fetch a new batch of tweets)
32:03 - randomly select a quote message from a pre-defined list
33:52 - fix Element Click Intercepted Exception
35:36 - run bot with full automation
36:14 - your thoughts on the ethics of bot making
37:07 - thanks for watching! :)
-----------------------------------------------------------------
🤖 COMPLETE CODE ON GITHUB 🤖
-----------------------------------------------------------------
github.com/MariyaSha/TwitterBot
-----------------------------------------------------------------
💻 IMPORT COMMANDS 💻
-----------------------------------------------------------------
from selenium import webdriver
from selenium.webdriver.common.by import By
---------------------------------------------------------------------------------
🕸️ WEB DRIVERS FOR DIFFERENT BROWSERS 🕸️
---------------------------------------------------------------------------------
⭐ Chrome:
chromedriver.chromium.org/dow...
⭐ Firefox:
github.com/mozilla/geckodrive...
⭐ Edge:
developer.microsoft.com/en-us...
--------------------------------------
🤝 Connect with me 🤝
--------------------------------------
🔗 Github:
github.com/mariyasha
🔗 Discord:
/ discord
🔗 LinkedIn:
/ mariyasha888
🔗 Twitter:
/ mariyasha888
🔗 Blog:
www.pythonsimplified.org
--------------------------------------------
💳 Credits 💳
--------------------------------------------
⭐ Beautiful animated graphics by:
mixkit.co
⭐ Beautiful icons by:
flaticon.com
⭐ Beautiful images by:
www.freepik.com/ 
Наука
Unfortunately, useless feature. Elon musk bans all the more or less popular bots. Cleaning is live for a few days. I would recommend to switch the platform.
Useless? 🤔 I don't know if you watched it, but this video is not here to encourage folks to make bots... on the contrary!
Nothing would make me happier than to see LillyPotter38 erased forever and ever, I was hoping they'd do it by now, but I don't think they are able to detect it 😀
The biggest problem is - the type of bot I show you how to make here is hidden from Twitter's servers.
It doesn't interact with Twitter directly, but it uses a browser window called Web Driver. Us, human users, also use a browser window to interact with Twitter.
To ban my bot - it needs to be detected first. To detect it - it needs to display abnormal behavior (such as tweeting the same message time and again for example or directly interacting with Twitter's servers using code rather than a browser).
So yes, Twitter banned some of the bots! particularly the very obvious ones that interact with Elon's polls directly.
I just don't think they are aware about the quirks and features of the more sophisticated bots, hopefully this video can start a public discussion and we can get rid of them once and for all 😉
@@PythonSimplified Randomizing the time of time.sleep() is always a good idea. Exactly the same click times are conspicuous in the long run. I am sure that such user behavior can be evaluated. And thus there is a chance to identify the user as a bot.
@@NormannCfC I also thought that random wait is a must for quite a long time! 😃
I was worried that if we have fixed intervals between our interactions - the platform will detect the bot in no time.
Now I'm not entirely convinced that the intervals are actually fixed... I think that in addition to time.sleep() there are time resources associated with the request to the server, as well as the CPU and memory consumption of your local machine.
This may be a variation of microseconds, but it still results in a different time interval.
However, while I'm not sure if it's actually necessary - it's definitely a much better practice and much more human-like behaviour! 😉
I haven't done any random wait for this specific bot, but my Liking Twitter Bot from earlier this year is utilizing it to the max💪💪💪:
github.com/MariyaSha/TwitterBot/blob/main/Twitter_LikingBot.ipynb
@@PythonSimplified That was just a general thought and not directly related to Twitter. One could probably also simulate the movements of the mouse or, depending on the content of the website, determine the time that a person needs to grasp the content and only click after this time. The longer I think about it, the more ideas I come up with on how to "humanize" such a bot. An exciting topic 🙂
@@NormannCfC cat and mouse kind of game.. who out-smarts who 🤓 To continue to evade detection you need to make it the most human of the bots, as detection also becomes more sophisticated. Fun times for developers on both sides 🤓🤣
This was the first video of yours I've watched, and it is excellent. Tutorial videos are not easy, but you seem to do it with confidence and are very knowledgeable about the content. What I really love, is you take an idea of creating a simple bot, and as a viewer I'm learning so much more than that (using a web driver, selecting elements, etc). I'm still a beginner with Python, but your videos give me some great ideas for some of my own coding, thank you.
This is a genre I didn't realize I needed. Just so relaxing to watch and learn from your style of explaining things 🥰
Hello! I ran into one dead end in the code and already went into despair, but I watched several of your videos, got inspired, felt a surge of strength and found a solution to the problem) Thanks for the cheerful charge! )
Impresive video. After one day hitting my head against the new Twitter API policies I found your fantastic video whhich opens a lot of room not only to interact with tweeter but for making some juicy webscraping. Thanks for the easy to follow pace even for newbies like me. Great stuff!
And BOOM! That was amazing ! Thx 4 the video
You're amazing! I love see your videos!
I've been only watching for 5 minutes but I gotta write a comment to let you know your explanation is very easy to follow for noobs like me. Thank you for sharing this knowledge!
Did you set it up ?
This was awesome, thank you for sharing!
Great tutorial. You are very talented and easy to follow. I have little to none knowledge in Python or coding, but with the help of your excellent tutorial, I will create my very own bot. It will be glorious and it will rule the internets. Thanks! 😉🌹
Great video! Thank you for you're great tutorials.
Excellent video, extremely well explained. Thank you very much.
Most people do forget about coding when looking at You) Very helpful, Thank You)
Love your videos, question:
Visual Studio or VS Code, which one to use and why?
Excelente aporte, me encanta tus tutoriales y la manera simpática de explicarlos, GRACIAS.
i love the way u explain python u r so professional 💙
your pronunciation is amazing!!!
Better teacher than my professors . Pls teach DEsign n algorithms n Neural networks n give notes also
Great tutorial !!! 👍👍👍🏆🏆🏆🏅🏅🏅 Thank you so much !!! Merry Christmas !!! ⛪☀ 🤗🎁🎀🎄
Thank you so much! 😀
Merry Christmas!!! 🎄🎄🎄
Another great video, thank you, your the best
Definitely not a bot here so... FANTASTIC! 😀
Worth to mention that chromedriver executable (no matter which platform you're using: macOS / Linux it will be chromedriver, for Windows chromedriver.exe of course) should be either in PATH env variable (which is also presented on any OS) or should be passed to the "executable_path" as a string parameter to the constructor
Came for beauty of eyes but stayed for the content. I'm just joking, you are a natural teacher.
Again a very good tutorial ! 😀😉
Thank you so much! 😀
Thank you for this amazing tutorial :)
You're absolutely welcome! enjoy! 😁😁😁
Great, simply great. Thanks so much
Ça m’apprend et m’aide tellement merci 1000 fois 🙏
This is cool you posted this. I used soup and selenium to recreate that flight tracker but got really gun shy after the ban. Oddly enough it harder to bot Instagram than it is to bot twitter.
You're having issues with Instagram???
Checkout my updated notebook for Instagram Web Scraping😀: github.com/MariyaSha/WebscrapingInstagram/blob/main/ImageExtracting_Updated-DEC2022.ipynb
I just revised it to the current syntax of Selenium (it changed recently), fixed the scrolling limitations and added some code that automatically installs a web driver (rather than manually downloading it).
I'm actually very disappointed with Instagram way more than with any other social media platform. My Web Scraping Instagram repo on GitHub has hundreds of clones, my super old tutorial has 150K+ views - but my bot is still there, it wasn't detected yet. They didn't even bother to make any changes to their DOM, so my code from 2+ years ago is still operational (only Selenium changed a bit, but if you install an older version of Selenium it works like a charm 😉).
In terms of the flight tracker, I find it a bit problematic. When someone is facing death threats from every direction - tracking their jet allows anyone who want to cause them harm the opportunity to do so.
To me, it's in the same realm as stocking someone, which may not be a criminal offence - but it doesn't shade any positive light either 😅😅😅 hahahaha.
I was quite shocked that some folks were upset that the flight trackers were banned... I may be wrong, but did they have any other purpose besides doxing?? 🙃
@@PythonSimplified Hey thanks for the help. Agree slightly with you on the flight tracker, I don't think it was the bot that led the stalker to Elon. The kid could have pulled the data from an api like flightaware. For some reason Elon's plane tail number is publicly posted. At the same time, I have no idea the purpose of tracking someone, I can't honestly think of a good reason. He ended up offering the kid 50K to turn it off.
i love the RUclipsr's English accent, it's a pretty good English
Really worth to watch❤❤
Wonderful teacher
Amazing mariyaaa!!
Thank you so much Ali!! Super happy you liked it! 😀😀😀
very helpful ! thnx
Thank you for the explanation, Mariya! It's way more interesting to read the news now that I know what is going on inside the brackets AND before the semicolon. 😁
You are right. If bots are this effortless to create it seems impossible to stop them... If I ever made a bot, I'd only code it to battle the Duolingo bird that I've invited to live in my phone.
Happy holidays everyone! Is anyone's New Year's resolution to keep learning Python? Mine is, and I'm so glad that Mariya is here to make it simple! ☺️❤️
Welcome. What do you think is the best book to learn Python?
AAAMammamAAAZING WOW Thank you
thanks a lot for the tutorial
wanted to automate tweeting using apis but the API ratelimit for free tier made me look for alternatives
this is an awesome way i will test on a test account before using the same logic to my actual account.
it is possible to run this on a server 24*7 that way wont have to login again and again
thanks for subtitles
Is there a specific reason for choosing Bulgaria as the account location ? :D
your channels in underrated
Thank you Smath! 😊
The best part of being underrated is - I don't really invest time in marketing or social media. When you keep that in mind - the channel seems to grow faster than I could ever imagine! hahaha
It's all thanks to lovely folks recommending it and spreading the word, I just focus on the content (and maintaining my sanity 😅😅😅 hahahaha)
Love your videos and explanations. I understand how to get info from a TAG or CLASS, but what if i want to get several items in a div class. For an example if i have this :
and i want the info from all elements. The data-prodid, data-statusid and so on?
Thanks in advance 🙂
You are always so inspiring 😄 thank you for another great tutorial
Thank you so much for the lovely comment Emanuele! Glad you liked it! 😀
hi, love your videos, i have a question, i have a python app which uses selenium, but when run from pycharm it works fine, but when converted to exe a terminal window appears when chrome is accessed, i use headless command for chrome driver, i use selenium 4 so no need to download and give path to the driver, i have tried the noconsole windowed command when converting i have tried a lot of other things also to get rid of this command prompt but it dosent go away, can you suggest a possible fix for it? thanksn
YOu are amazing!!!
Hi Mariya! You always give us an excellent explanation.
...
I have been watching your Kivy and KivyMD videos, and I would really like it if you could make a video explaining how to replicate the Termux (Play Store app) android service with Python and KivyMD.
Thanks for teaching
at my click Next button after type usename, this error coming, can you teach me ?
TypeError: 'WebElement' object is not subscriptable
Hi. I've a problem with kivy. I have finished developing and on the of converting to apk, it isn't get resources from the internet. I was using pyrebase4. It's crashing. Without pyrebase it's okay. Kind guide me on how to solve this.
is there a way to filter out the promoted/ad tweets? I am getting those sometimes when I run the program. Not sure if there is a way to filter out those specific tweets. Thanks!
In the Quote RT section
"Sure, I can try to provide an answer from the perspective of a fictional character in the real world.
As a bot maker, it's important to consider the ethical implications of the technology we create. One key ethical consideration is the issue of bias. We need to ensure that our bots are not biased in their decision-making or in the information they present, especially if they are used in contexts such as hiring or loan approvals where they could have significant impacts on people's lives.
Transparency is also crucial. We need to be open about how our bots work and what data they are using, so that users can understand how they are making decisions and have confidence in their outputs.
Privacy is another important ethical concern. If our bots collect personal data from users, we need to be clear about what data is being collected and how it will be used, and we must implement appropriate security measures to protect user data.
Finally, if our bots are autonomous, we need to carefully consider the potential consequences of their actions. For example, if our bot is driving a self-driving car, it should be programmed to prioritize the safety of passengers and pedestrians over other goals.
Overall, it's crucial that we take a responsible and ethical approach to bot making, in order to ensure that the technology we create is fair, transparent, and respectful of people's rights and privacy."
ChatGPT opinion about ethics of bot making.
Looks like ChatGPT is talking about a very specific aspect of bots... I see no mention of phishing, fraud, spamming, copyrights violations and many other implications on ethics 😉
It's either not as smart as they say it is, or it has the same bias it was warning us about 😅 hahahaha
Thank you so much for sharing! 😀
Would be good to see a vid on your thoughts about ChatGPT's python skills.
I was about to post the same question 🤔
@@ryuyasho4468 just when we thought we were safe as programmers in a world of AI... we made our replacement 😅
we had great salary, amazing conditions and an incredible profession - but now it seems that we will also own nothing and be happy... thank you ChatGPT 🤣🤣🤣
@@PythonSimplified 😂
@@PythonSimplified Nah, I happily refuse the Great Reset Idea. Work all of our lives for nothing and be happy for it??? No thanks. As for the ChatGPT, I think it'll help us how to solve problems at the same speed, enrich our brain 🧠. Or at least that's what I hope for 💜
@@PythonSimplified sounds like just what the whole world is coming to. No conspiracy theory there.
got stuck at login button with Error - ElementNotInteractableException , is there a way aroud this?
Would love a counter tutorial to detect bots or ethics of this stuff. Fast AI makes you watch a video about Ethics in AI before they teach you any generative AI.
I would love to see something like that too!! I'm unfortunately starting to think that web drivers will allow bots to run undetected forever and ever 😅
It's hard to talk about ethics in the context of bots, because no matter how nice, polite and harmless your bot will be - the purpose of it is to pretend it's a human being and by doing so, to trick everyone else who engages with it. Bots in their very core are very unethical, and yet - we see them everywhere! and despite the big "bot cleansing" that started last week - many of them still run undetected (including 600+ Korean dating bots that follow LillyPotter38 🤪)
815. Big like ❤😊 love it
You should teach english as well. I can understand everything :) Regards from Brazil!!!
Could u wear a decolte so we can concentrate on your pythons
There was a time that i was trying to make an Instagram bot, and I had to put a random delay beetwen each letter typed to get more human likely typed. But I don't remember how i did that delay haha
Great idea! There's a bunch of ways to add random delay 😉
In the case of typing, I don't know if it will make much of a difference. But I do use it often between one post to another.
⭐ My favourite random wait is:
import random
import time
time.sleep(random.choice((2, 10))
⭐ Another way is using the WebDriverWait class which waits for the element to fully load, or become clickable before Selenium searches for it.
I've used it in my Web Scraping Instagram tutorial 😁:
ruclips.net/video/iJGvYBH9mcY/видео.html
I must admit though that I didn't encounter any issues with a non-random wait, it's just that waiting 2 seconds precisely in between each interaction is not a very human thing to do (unless someone has a serious OCD hahaha 🤣🤣🤣)
just what the world needs: more bots
I noticed your video is mirrored when presenting your screen. If that is intended sorry I mentioned it! :)
I am try ,but don't know, by the way are you from ,please because I am from Brazil
А есть что то подобное, что бы искать людей по кодслову и читать их?
well, you should be the best
what a set up lol extreme
BOOM!!!
Learning python and simpin = python simplified 😍
cool
One thing I would like to ask: Many people (including you) say that bots are being banned on X(Twitter). I heard that too. That Elon is cleaning up the bots to make the platform better. But when I open Twitter I see bots EVERYWHERE! They are constantly at work. And the general opinions are that Elon is actually letting the bots inflate the numbers, so X looks good on advertiser's paper.
So which is it?
My bot is still on Twitter many months later, and no one deleted it. It is also followed by many other bots that were not deleted either.
There are many reasons why:
1. They are not active enough to meet the threshold of "abnormal behavior" to label them as bots. (My personal hypothesis)
2. Sometimes, humans use these bot accounts properly, without automation. (Like in my case), and then it's unclear if a human or a bot is operating the account.
3. They make the numbers of users look bigger on purpose like you suggested.
However, an important point: the number of users dorsn't mean much to advertisers. What's important is engagement and reliability. If 100 users are active and trust the platform, it is much better than 10,000 users who are not active or do not trust the platform. Just my personal observation 😉
@@PythonSimplified thank you
Wow wow
You're beautiful and your code is simply pretty
Te amo
why windows.. today, i dont feel like doing anything tororot tororot..
Do you do anything in AI?
Yup, I have an entire playlist of AI tutorials! 😀
If you're a big fan of Neural Networks - you gonna love it! 😉
Check it out here: ruclips.net/p/PLqXS1b2lRpYTpUIEu3oxfhhTuBXmMPppA
maybe ypu should have udemy course 😁. you do explain well
3:53 - voice change to more smooth. The mic change colour, so is the mic changing filter with colour? 🤔🤓 ..or is she doing some ai filter to the sound? 🙂🙃
Changes in the tone of my voice = afternoon filming versus morning filming (my voice is very deep in the morning, but after 3 cups of coffee it becomes much more girly 😉)
Changes in mic colour = my microphone is an RGB mic so I can change the colour combinations/patterns/animation as much as I want. Since the background of the slides is blue - I've adjusted the colour of the mic to yellow so there's a nice contrast 😀
I always filter the sound: I reduce noise coming from my PC and I ensure it sounds as pleasant as possible... and I think this should cover all the questions you had 😊😊😊
@@PythonSimplified That's crazy, would not have guessed such change by time of day, but does make sense now that you say it 👍 I was thinking it was an RGB mic, but perhaps it could have more to it. 🙃
While you are here, don't know if you notice, but sometimes your hands get cut off, I guess they go out of frame. Don't know if you have space to avoid that. 🤓
Great content regardless - keep it up! 💪👍👌
With ai progressing the way it is, you’ll soon be able to upload selfies of fake people, making your account appear even more real.
You can already do that! 😉 There are plenty of companies offering you hundreds of thousands of photos of non-existing humans. This is the first site the popped up in the search: generated.photos/faces
But there are soooooo many of them, and we might actually do something similar in future Ai/ML Simplified videos 😀
I posted a comment with the bot OMG!! SO FUNNY!!.... a content where someone's dog died
Video Title - "How to make Elon Musk angry"
Hopefully more "impressed" than angry 😉 hahaha
But angry is good too, it would mean that someone in Twitter is aware of WebDriver bots which was the main purpose behind this tutorial 🤪
I would like to see the tutorials she records on the plane......
around other pasengers yelling, "BOOOOM!!"
@@Python_Simplified selected for what, im sorry i must've missed that, but im down.
i love you.
(for what you do. of course. 😅😳)
Eeeeeee.... Elon don't like bots. But if you would have some time and make a video about telegram bots i would appreciate it.
I'll keep my same profile pic and make a giant list of gaffes by creepy Joe Sniffer and spam them in the comments section of all the corporate media outlets!
Hi sister i love you ❤❤❤
Thank you Bat Man, I love you too! you're my favourite super hero! 😁😁😁
Wait, so I wasn't supposed to give those pretty ladies on the dating apps my credit card information? Well, shite.
How can you be so beautiful?
🙏🙏🤗🤗
I would like to spend my life with girl like this 💖
U sooo 🤭 cutie ☺️❤️💜💙
hey angel .... why u r so cute
Very suspicious subject and bots also resemble a huge menace in todays online community. Not something that would ever need explaining or taught to be created.
mem Are you marriage
sorry mam '
i was just kidding but i like your answer
but are you angry with me
Let you Instagram here.
Странно, но ваш английский становится хуже, пора на родину 😅
i have this problem
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"tag name","selector":"input"}
I'm basically your Twitter Bot.
I love this trap =*
so, are you telling me that all the hot asian girls that follow me on twitter and offer me amazing crypto deals are not real?? xD amazing tutorial, thanks!