![Thomas Janssen | Tom's Tech Academy](/img/default-banner.jpg)
- Видео 69
- Просмотров 440 565
Thomas Janssen | Tom's Tech Academy
Нидерланды
Добавлен 17 сен 2020
My name is Thomas (or Tom), and on this channel you learn everything about coding in Python 💥
Whether you just want to learn the basics, scrape data from the internet or build real applications - you're in the right place!
Whether you just want to learn the basics, scrape data from the internet or build real applications - you're in the right place!
Scraping Websites with Infinite Scrolling: The EASY way!
Scraping websites which use "infinite scrolling" (meaning the website keeps loading items when you scroll down) can be extremely complicated. In this video I show you the easiest method which works on 99% of all websites!
💭 Join my Discord community and get help from many other Python developers:
discord.gg/WHJWFNDXXX
⭐️ Copy the source code here ⭐
tomstechacademy.com/code-example-scraping-websites-with-infinite-scroll/
⭐️ The website we scrape ⭐
www.fintastico.com/fintech-uk/
⭐️ Timestamps⭐
00:00 - Intro
00:51 - The website we want to scrape
01:55 - Install and import libraries
03:14 - Scrolling through the webpage and detecting the end
06:20 - Extracting the data and saving it in a txt file
08:37 ...
💭 Join my Discord community and get help from many other Python developers:
discord.gg/WHJWFNDXXX
⭐️ Copy the source code here ⭐
tomstechacademy.com/code-example-scraping-websites-with-infinite-scroll/
⭐️ The website we scrape ⭐
www.fintastico.com/fintech-uk/
⭐️ Timestamps⭐
00:00 - Intro
00:51 - The website we want to scrape
01:55 - Install and import libraries
03:14 - Scrolling through the webpage and detecting the end
06:20 - Extracting the data and saving it in a txt file
08:37 ...
Просмотров: 241
Видео
ScrapeGraphAI - REVOLUTION in WEB SCRAPING!!!
Просмотров 5 тыс.Месяц назад
Do you often scrape data from the internet with for example BeautifulSoup? Now that's history! ScrapeGraphAI is a NEW Python library which will do the heavy lifting for you. You don't have to go through the HTML of a website anymore, just provide the website and a prompt to ScrapeGraphAI and the library will fetch your data for you! 💭 Join my Discord community and get help from many other Pytho...
Running LLM's on your PC with LM Studio | Mistral 7B on your PC
Просмотров 839Месяц назад
Large Language Models have taken the world by storm! But do we really want to send all our data to OpenAI? Why not run LLM's locally on our PC? In this video you'll learn how to use LM Studio to run models on your PC! I first show you how to chat with the model, the same way we do with ChatGPT, and in my second example you learn how to create a local server and connect with it from a Python scr...
How to ACTUALLY use Chat-GPT with Python | Complete Tutorial
Просмотров 1,8 тыс.2 месяца назад
Do you want to use the Chat-GPT API from your Python scripts? In this video you learn how to get a Chat-GPT API key, how to connect with the API from Python, how to get structured JSON response and how to use Function Calling! 💭 Join my Discord community and get help from many other Python developers: discord.gg/WHJWFNDXXX ⭐️ Copy the source code here ⭐ tomstechacademy.com/code-example-python-c...
Schedule Python Scripts in the Cloud for FREE
Просмотров 1,1 тыс.3 месяца назад
Scheduling your Python scripts in the cloud for free? It's possible with pythonanwhere.com! 💭 Join my Discord community and get help from many other Python developers: discord.gg/WHJWFNDXXX ⭐️Link to Pythonanywhere.com⭐ www.pythonanywhere.com/ ⭐️Download my script here⭐ tomstechacademy.com/code-example-download-the-price-of-bitcoin-every-day/ ⭐️Link to the API⭐ api.coindesk.com/v1/bpi/currentpr...
FINALLY a No-Code Cloud Scraping tool that WORKS!
Просмотров 4523 месяца назад
Do you want to scrape data from the internet without learning how to code? In this video you learn how to use a No-Code Web Scraping tool. Best of all? Because it runs in the cloud you can schedule it and scrape data whenever you want! ⭐️ Use this link if you want to support my channel⭐ browse.ai?via=tomstechacademy ⭐️Scroll through the video⭐ 00:00 - Intro 01:28 - Monitor changes on a specific...
Function Calling: How to get structured data from Chat-GPT?
Просмотров 2 тыс.3 месяца назад
Do you want to call the Chat-GPT API and get back structure data? In that case you have to use Function Calling. This will allow you to get back Chat-GPT's response in JSON format, so you can use it in your application. In this video I'll show you the easiest way to use function calling in order to get structured data from the API response. 💭 Join my Discord community and get help from many oth...
SQLite3 Tutorial - Learn SQL for Python in 17 Minutes
Просмотров 3664 месяца назад
Do you want to learn how to use SQL in Python? In this video you learn CRUD: Create, Read, Update and Delete rows with Python! We build a console application which lets you manage a database. SQLite3 is the database we'll be using - and it's already part of Python 3, so you don't need to install it! 💭 Join my Discord community and get help from many other Python developers: discord.gg/WHJWFNDXX...
Build a Test Automation Framework with Python and Selenium
Просмотров 6024 месяца назад
Are you considering a career in Test Automation? Test Automation is one of the fastest growing disciplines! Did you know that the test market is growing with 19% a year? Just imagine Automated Testing is growing even harder! In this video we'll build a Test Automation Framework with Python and Selenium, which will automatically generate a Test Report in HTML! Automated Testing was never this ea...
Create a Python Virtual Environment in VS Code on Mac OS
Просмотров 4,1 тыс.4 месяца назад
Creating a Virtual Environment (Venv) is easy on Mac OS! In this video we create and activate a virtual environment, ⭐️ Follow me ⭐ 💼 LinkedIn: www.linkedin.com/in/thomas janssen/ ✅ Instagram: tomstechacademy ❗ Some of my videos contain affiliate links. When you click on these kind of links, I get a small percentage in case you make a purchase via the website. Be aware that this ...
Setup Python and Visual Studio Code (VS Code) on Mac
Просмотров 6 тыс.4 месяца назад
Want to install Python3 and VS Code on a Mac pc? In this installation guide I show you exactly which steps to follow! ⭐️ Follow me ⭐ 💼 LinkedIn: www.linkedin.com/in/thomas janssen/ ✅ Instagram: tomstechacademy ❗ Some of my videos contain affiliate links. When you click on these kind of links, I get a small percentage in case you make a purchase via the website. Be aware that this...
This is HOW to create a Python Virtual Environment in VS Code with Venv
Просмотров 2,1 тыс.4 месяца назад
Virtual environments are the way to go when you want to build scalable projects which are easy to transfer. A virtual environment allows you to use a different Python version for different projects, and different dependencies. You can run Python version 3.11 global, and for example Python 3.8 in your venv - with different versions of selenium. ⭐️ All the terminal commands I use⭐ python -m venv ...
Web Scraping with Python and BeautifulSoup is THIS easy!
Просмотров 19 тыс.5 месяцев назад
Do you want to scrape structured data from a website with Python and BeautifulSoup? Then you're in the right place! In this video I show you the easiest way to scrape data from multiple pages. BONUS: At the end of this video you'll learn how to avoid your IP address from being exposed by using a proxyserver. 💭 Join my Discord community and get help from many other Python developers: discord.gg/...
Get consistent JSON response from OpenAI / Chat-GPT
Просмотров 10 тыс.6 месяцев назад
As from GPT-3.5- Turbo, Chat-GPT allows to set response_format as part of the parameters of an API call. But even using this parameter doesn't guarantee the response is in a consistent format. In this video I show how you can ensure a consistent response and how to avoid invalid JSON response because of insufficient tokens. 💭 Join my Discord community and get help from many other Python develop...
EASIEST way to use Chat-GPT in Python
Просмотров 13 тыс.6 месяцев назад
EASIEST way to use Chat-GPT in Python
Power Automate Desktop Tutorial for Beginners | Zero to Hero
Просмотров 8 тыс.9 месяцев назад
Power Automate Desktop Tutorial for Beginners | Zero to Hero
Web Scraping in Power Automate Desktop | Multiple Pages | Tutorial
Просмотров 14 тыс.10 месяцев назад
Web Scraping in Power Automate Desktop | Multiple Pages | Tutorial
Invoice Extraction: Extract PDF Invoice to Excel with UiPath
Просмотров 9 тыс.Год назад
Invoice Extraction: Extract PDF Invoice to Excel with UiPath
How to get an Open AI / Chat-GPT API Key
Просмотров 83 тыс.Год назад
How to get an Open AI / Chat-GPT API Key
Send email with Dynamic Values UiPath | Send mail Tutorial
Просмотров 8 тыс.Год назад
Send email with Dynamic Values UiPath | Send mail Tutorial
How to use Chat-GPT in Power Automate Desktop? Full Tutorial
Просмотров 4,9 тыс.Год назад
How to use Chat-GPT in Power Automate Desktop? Full Tutorial
UiPath Installation Guide | Download UiPath Community Free
Просмотров 15 тыс.Год назад
UiPath Installation Guide | Download UiPath Community Free
Automate any Web Application with Power Automate Desktop!
Просмотров 6 тыс.Год назад
Automate any Web Application with Power Automate Desktop!
Power Automate Desktop Excel Tutorial | Excel Tips & Tricks
Просмотров 2 тыс.Год назад
Power Automate Desktop Excel Tutorial | Excel Tips & Tricks
String Functions in Power Automate Desktop | Get Subtext | Crop | Trim | Replace | Reverse | Split
Просмотров 5 тыс.Год назад
String Functions in Power Automate Desktop | Get Subtext | Crop | Trim | Replace | Reverse | Split
Power Automate Desktop DateTime Tutorial | Add Days to Date | Calculate Difference | Change Format
Просмотров 7 тыс.Год назад
Power Automate Desktop DateTime Tutorial | Add Days to Date | Calculate Difference | Change Format
Creating Excel Pivot Tables with UiPath Studio X | Tutorial
Просмотров 2,1 тыс.Год назад
Creating Excel Pivot Tables with UiPath Studio X | Tutorial
Studio X Tutorial - Remove Large Files with UiPath Studio X
Просмотров 239Год назад
Studio X Tutorial - Remove Large Files with UiPath Studio X
UiPath Studio X Web Automation Tutorial | Create a Purchase Order Bot
Просмотров 2,6 тыс.Год назад
UiPath Studio X Web Automation Tutorial | Create a Purchase Order Bot
Tutorial: Add Clients to MyCRM with UiPath Studio X
Просмотров 632Год назад
Tutorial: Add Clients to MyCRM with UiPath Studio X
So what I understand from this is that you aren't specifically interested in the output of the function. Rather you are defining the function in such a way that it expects the parameters to be key value pairs of country and population. You then extract the arguments passed by the LLM to this function and this is essentially structured as a dict with the two parameters as keys.
Didn't work for me at all. I seem to be outputting python commands to a separate terminal to you. Nvm, will just use the cmd prompt to set up my virtual environments...imo VS code is truely awful. Thanks for the video though.
Can i scrape the reviews from amazon through this same method? Cuz I'm unable to access the page through my requests
it got login error even after entering valid credentails and also how to receive mail from outlook
Many people are not offended by the monkey logo of Robocorp, and some suggest that the company's board might be represented as monkeys. ChatGPT ❤🎉
Great intro to Studio X, many thanks.
i get api error
If you receive any error, feel free to join my discord (link in description) and post your question there 😀
thanks so much for your tutorial. I got the understanding good. Best so far
I dont think this is a "framework", this is just a simple demo test.
Thank you so much for showing the demo! I still have a question:does scrapegraphai support full stack crawling? or it only works for one page
i dont understand and you didn't explain whats the first screen on your video we see? what programme is that?
Hi, do you mean VS Code? You can watch this installation manual: Getting Started with Python - Install Python and VS Code + Extension ruclips.net/video/KkVLNvFh0-M/видео.html
When i am trying this i get the error asyncio error cannot be called
Hi, thanks for your comment. I have seen this error before, is it possible you're running Python 3.12? If so, try running it in a venv with 3.11. Good luck! If you need more help, you can reach me on my Discord (link in description).
Thanks- this video was of value to me, and I look forward to power automating my work tasks!
Glad it was helpful!
Nece tutorial, thanks
You are welcome, glad my video was helpful for you!
Great Video , Thanks
Thank you, glad to hear my video is useful for you 😀
needs paid version of chatgpt. useless.
This was so helpful!! Thank you !!! Could you please make a video on how to go to each product subpage and extract details like description from there (for ecommerce)?
Hi Anna, there's another video on my channel which shows how to loop through urls, visit every one of them and extract information. You can find it here: ruclips.net/video/nBzrMw8hkmY/видео.html If you have any questions, don't hesitate to join my Discord and I will be happy to help you: discord.gg/WHJWFNDXXX
Thank you this video was super quick and just what I needed!
Thank you for your comment Kelley, glad I could be of help 😀
Interesting. I was trying to figure out how to get some structured responses to my Chat GPT requests. This json format, and Json example approach looks very useful way to coerce the responses into a useful syntax for my overall NPC adventure application. Many Thanks
Thank you for your comment Jules, glad I could be of help. Good luck with your application 🚀
Thank you so much for your insightful video; it was incredibly beneficial. Could I ask you to consider making videos about web scraping periodically? Your expertise in this area is greatly appreciated!👏👏
Thank you for your comment! You can expect more videos about web scraping in the next months 😊. Any specific topics you're interested in? You can also join my Discord (link in description) to exchange knowledge about web scraping.
I just give chat gpt the html data as example. Give it what I want scraped and it’ll generate the code for me
Thanks for Sharing your expertise, that’s another way of doing it 😀
I found ScrapeGraphAI pretty useless, it doesn't except for the simplistic examples.
It’s a great tool for people without Python / BeautifulSoup skills. For most cases which are a bit more complex, the conventional approach is probably the best ;)
Man, there are f-strings in python and they are awesome. Please use them instead of string concatenation - this way there is no need to manually convert integers to strings
Thanks for your feedback, greatly appreciated 😊! Will definitely use them in the next videos.
This brilliant. Thank you! Is there a way to increase the Explorer font size? It's killing me. 🙏
Good tutorial.
Thank you for your comment, glad my video is useful for you 😀
Thanks
Thanks for your comment, glad my video is helpful for you 😀
Hy, Thomas you've done a great job, I learned a lot of imp things from you by watching only this video. Plz make a playlist on web scraping from basic to advance so that a new person can understand better. Once again amazing video
Thank you for your comment, expect more videos about web scraping soon! Cheers, Thomas
@@TomsTechAcademy Thomas, that's good for all new web scrapers bcz your style of conveying useful content is very simple thanks ☺☺☺
excactly what i needed, thanks
Thank you for your comment and glad my video was helpful for you :)
I wish we could prints these flows out or send the to a text file so I could make sure I have the right code. In your video it's hard to see your Flow as I have 3 windows open just to learn this Flow.
Thanks for your feedback, I'm going to see how I can make it easier to follow along
Power Automate doesn't support zoom in or zoom out at this point.
For ch 3, the Flow only gets the first page of items over and over again and puts them in the spreadsheet. In your video you click the "Anchor 2" button which doesn't seem correct. When I get my Flow to try to find and click the Next button that doesn't work either. It still gets page 1 of the "Touch" products over and over again, with the Nokia 123 item, and appends the first page of items to the spreadsheet over and over. Ok I got it working but I don't know how to share the code with others.
This helped to install Python3 on Macbook Pro, Thank you for short quick video with detailed instructions! I can run the code on Collab or Jupitor notebook however I want to run Python3 on my laptop
Glad it helped! I also prefer to run my code local :)
In the if statement "If CurrentItem IsEmpty then exit" the Flow is getting employee numbers for rows 10-12 but there are no employee names there. I tried using the condition IsBlank but that still adds the numbers for blank employees. Is there a way to make sure the name length is > 0? How would we fix this? I also tried %CurrentItem.Length%, which did not work because it's not a string variable. And I tried a variable of "%CurrentItem.['Name'].Length%" which gives me a syntax error. I also tried "%CurrentItem.['Name']%.Length" which gave a syntax error. Ok this condition seems to be working: "%CurrentItem['Name']% IsBlank" then exit and will not get Employee Numbers for names that are blank.
Thanks for the great chapters in your video desc! This tells me what I might learn from this video.
Thank you for your comment, glad you like my videos :)
Thank you, bro.
Welcome!
Friend... I can't thank you enough for this video. I've spent hours trying to mix and match the code from various videos that are out of date, or don't explain anything properly. The OpenAI site's API reference was basically non-sense to me. Even GTP 4o uses the "old" syntax. And when I told it to check the latest info on the openai git site, it still gave me the wrong code! Anyways, Thank you thank you thank you! I will be coming back regularly and hopefully you'll have more videos on using the latest from OpenAi APIs. Be well!
Thank you my friend, glad my video was useful for you 😀 you can expect more videos in the next weeks! 🚀
Thanks a lot!!
Thank you for your comment Manish, happy coding!
No yadda, yadda - just straight to the point, including code. Amazing! Subscribed.
Thank you for your comment, and glad my video was helpful for you 😀
Great Work Thomas!
Thank you Farhan!
whenever I install scrapegraphai I am hit by AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'? [end of output]
Hi, I think I had this exact same error - are you running Python in a virtual environment? If you run Python 3.12 you can try switching to 3.11
Hey please make video to download files from web table hyper link where you have multiple pages.
Hi, I want to use 3 invoices but the thing is that the values of each invoice will overwrite each other. I added a read range workbook as well as an assign and now it says that it cannot process as its being used by another process. Can you help me ? Thanks a lot
I have also added increment row but it still isn't working
Hi, please post this question on my Discord channel - this way I can help you better: discord.gg/WHJWFNDXXX
I keep getting this error "PermissionError: [Errno 13] Permission denied: 'wired.xlsx'" and I've even typed the code shown in your video exactly and still I get that error, any help would be greatly appreciated
Hi Meta, it looks like you're trying to write a file to a directory where you don't have enough permissions. Assuming you're using Windows, can you try the following? Navigate to your Desktop, right click on any of your files there. Click properties. Go to the tab "general" and read location. This should be something like: "C:\Users\thoma\Desktop" (where thoma is your username). Copy this location to VS Code. Now add \wired.xlsx and change all slashes for double slashes (you should now have something like this: df.to_excel("C:\\Users\\thoma\\Desktop\\wired.xlsx",index=False) Run the code again. Let me know whether this works for you. If not, feel free to join my Discord if you need more help (link in video description).
@@TomsTechAcademy Hi, I managed to get it to produce two results and now I'm having issues again, It is now giving me this message "Traceback (most recent call last): File "c:\scrapperai\main.py", line 26, in <module> df = pd.DataFrame(result['pages']) ~~~~~~^^^^^^^^^" any help would be greatly appreciated!
Brilliant. Thank you.
You're very welcome!
this video tutorial was suprising beginner friendly
Thank you for your comment Amanda, and glad my video was useful for you :) I always try to make my videos as beginner friendly as possible
Thanks for demo-ing the tool
Thank you for your comment, much apreciated!
It extracted the titles, not the articles themselves.
Thanks for writing this
So interesting, I don't understand why you don't have more subscribers, I guess quality is better than quantity :)
Thank you for the kind words Tristan! Happy weekend 😊
will this work if i send the folder to someone else to run my projects?
Short answer; yes.
@@TomsTechAcademy thanks
Thank you!
Thank you for your comment Arda. Happy coding 💻
I am trying to create a virtual environment from my Mac Terminal ( not using VS Code ) and I pasted in the command python3 -m venv .venv but no folder by the name venv is getting created in my project folder. Can you please help me.
Hi Ashok, did you use cd (change directory) in your console to navigate to a specific directory? Is it possible your virtual environment has been created in some kind of root folder?
@@TomsTechAcademy I was able to get it done. How do I check if there is a virtual environment in a particular folder?