![tuplespectra](/img/default-banner.jpg)
- Видео 77
- Просмотров 244 182
tuplespectra
Добавлен 31 дек 2022
Process Engineer to Machine Learning Engineer | Interview with Raphael | Exponential Salary Increase
#datascientist #motivationalstory #inspiry story
Listen to Raphael sharing his journey on how he made the career switch from Process Engineering into Machine Learning Engineering.
*Books I recommend*
1. Fundamentals of Data Engineering: Plan and Build Robust Data Systems: amzn.to/4bfzmvX
2. Grit: The Power of Passion and Perseverance amzn.to/3EZKSgb
3. Think and Grow Rich!: The Original Version, Restored and Revised: amzn.to/3Q2K68s
4. The Book on Rental Property Investing: How to Create Wealth With Intelligent Buy and Hold Real Estate Investing: amzn.to/3LLpXRy
5. How to Invest in Real Estate: The Ultimate Beginner's Guide to Getting Started: amzn.to/48RbuOb
6. Introducing Python: Modern Comput...
Listen to Raphael sharing his journey on how he made the career switch from Process Engineering into Machine Learning Engineering.
*Books I recommend*
1. Fundamentals of Data Engineering: Plan and Build Robust Data Systems: amzn.to/4bfzmvX
2. Grit: The Power of Passion and Perseverance amzn.to/3EZKSgb
3. Think and Grow Rich!: The Original Version, Restored and Revised: amzn.to/3Q2K68s
4. The Book on Rental Property Investing: How to Create Wealth With Intelligent Buy and Hold Real Estate Investing: amzn.to/3LLpXRy
5. How to Invest in Real Estate: The Ultimate Beginner's Guide to Getting Started: amzn.to/48RbuOb
6. Introducing Python: Modern Comput...
Просмотров: 421
Видео
Petroleum Engineer to Data scientist | Join Africans in Data Community to expand your network
Просмотров 3784 месяца назад
#datascientist #motivationalstory #communityservice #buildyournetwork In this video, Toyosi Bamidele discussed how she transitioned from a petroleum engineer to a data scientist and the motivation behind founding “Africans in Data Community”. She further explained how you can become a member of this community and the benefits you stand to gain when you become a member. In addition, she gave som...
Complete Roadmap/Guide on How to Become a Data Engineer in 2024
Просмотров 9115 месяцев назад
#dataengineering #dataengineeringroadmap In this video, I share with you a complete guide/roadmap on how to become a data engineer in 2024. You do not have to know everything highlighted in this guide before you start applying for DE roles. Please click on the google drive link below to request a view access to the google doc where I listed the roadmap. docs.google.com/document/d/1hWkKlPvZOBigm...
Tips to become a data engineer in 2024 - What no one ever told you | Keep Learning | Be Consistent
Просмотров 5906 месяцев назад
In this video, I will like to share with you what I have learnt while learning programming and tips to become a data engineer in 2024. *Books I recommend* 1. Grit: The Power of Passion and Perseverance amzn.to/3EZKSgb 2. Think and Grow Rich!: The Original Version, Restored and Revised: amzn.to/3Q2K68s 3. The Book on Rental Property Investing: How to Create Wealth With Intelligent Buy and Hold R...
Tutorial 3.1-How to load/restore DVD rental database to PostgreSQL server16 using pgAdmin| Windows11
Просмотров 3,4 тыс.7 месяцев назад
In this video I showed how to load or restore the DVD rental database to our PostgreSQL sever 16 on Windows 11 computer. We will use this DVD rental database for the next series of videos to understand the various SQL concepts from the beginner level to intermediate/advanced level. Please go to this link to download the dvdrental.tar sample database file www.postgresqltutorial.com/postgresql-ge...
Tutorial 2.1 - How to install PostgreSQL 16 and pgAdmin on windows 11 computer | Step-by-Step Guide
Просмотров 1,3 тыс.7 месяцев назад
I will be showing you a step by step guide on how to download and install PostgreSQL server 16 and PgAdmin on your windows 11 computer. Follow this link to download postgreSQL www.postgresql.org/download/ *Books I recommend* 1. Grit: The Power of Passion and Perseverance amzn.to/3EZKSgb 2. Think and Grow Rich!: The Original Version, Restored and Revised: amzn.to/3Q2K68s 3. The Book on Rental Pr...
Master SQL | Go from Zero to Hero | SQL Live Class BootCamp
Просмотров 9957 месяцев назад
I am excited to launch this 8-weekends intensive SQL bootcamp which will cover Basic, Intermediate and Advance SQL concepts. 👉 Training start date: Jan 27th 2024. 👉 Days: Saturday and Sunday 👉 Class Time: 5am PST | 6:30pm IST | 2pm WAT 👉 Fee: 40USD 👉 Certificate of Completion will be provided 👉 Register for BootCamp here: awesumedge.com/register-jan2024postgresql/ PostgreSQL Playlist: ruclips.n...
Airflow Tutorial | Automate EMR ETL Jobs with Airflow | Airflow Project | Data Engineering Project
Просмотров 2,1 тыс.7 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 7 - Terminate EMR cluster task | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 4217 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 6 - Create EMR cluster and Add steps tasks | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 1,3 тыс.7 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 1 - Project Overview | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 1,6 тыс.8 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 3 - Initialize airflow and explore UI | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 3978 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 5 - Connect VS code and create task 1 | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 5268 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 4 - Create S3 buckets and required folders | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 3118 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 2 - Provision EC2 and Install dependencies | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 7208 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 6 - Data Transformation|pyspark|What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 6468 месяцев назад
Part 6 - Data Transformation|pyspark|What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 5 - EMR Studio|Jupyter Notebook|What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 4408 месяцев назад
Part 5 - EMR Studio|Jupyter Notebook|What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 4 - How to Create EMR Cluster | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 4728 месяцев назад
Part 4 - How to Create EMR Cluster | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 3 - How to create AWS VPC | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 2998 месяцев назад
Part 3 - How to create AWS VPC | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 2 - Create User and S3 bucket | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 3948 месяцев назад
Part 2 - Create User and S3 bucket | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 1 - Project Overview | What is AWS EMR | Extract and Transform Redfin data with AWS EMR
Просмотров 8878 месяцев назад
Part 1 - Project Overview | What is AWS EMR | Extract and Transform Redfin data with AWS EMR
What is AWS EMR | Extract and Transform Redfin data with AWS EMR | EMR Studio | Pyspark Notebook
Просмотров 8 тыс.9 месяцев назад
What is AWS EMR | Extract and Transform Redfin data with AWS EMR | EMR Studio | Pyspark Notebook
LeetCode 197 - Rising Temperature | Common SQL Interview Question | Self-Join | LAG window function
Просмотров 2739 месяцев назад
LeetCode 197 - Rising Temperature | Common SQL Interview Question | Self-Join | LAG window function
Redfin Analytics|python ETL pipeline with airflow|Data Engineering Project|Snowpipe|Snowflake|Part 2
Просмотров 2,1 тыс.9 месяцев назад
Redfin Analytics|python ETL pipeline with airflow|Data Engineering Project|Snowpipe|Snowflake|Part 2
Redfin Analytics|python ETL pipeline with airflow|Data Engineering Project|Snowpipe|Snowflake|Part 1
Просмотров 6 тыс.9 месяцев назад
Redfin Analytics|python ETL pipeline with airflow|Data Engineering Project|Snowpipe|Snowflake|Part 1
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 3
Просмотров 1,7 тыс.10 месяцев назад
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 3
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 2
Просмотров 4 тыс.10 месяцев назад
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 2
YouTube Channel Join Button | YouTube Memberships
Просмотров 12010 месяцев назад
RUclips Channel Join Button | RUclips Memberships
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 1
Просмотров 7 тыс.10 месяцев назад
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 1
Zillow Data Analytics (RapidAPI) | End-To-End Python ETL Pipeline | Data Engineering Project |Part 3
Просмотров 3,3 тыс.10 месяцев назад
Zillow Data Analytics (RapidAPI) | End-To-End Python ETL Pipeline | Data Engineering Project |Part 3
@tuplespectra My S3ToRedshiftOperator task is not changing status in airflow. I have written the correct code, added and updated all the roles, policies and permissions and added the redshift connection in airflow, restarted the server as well. What am I missing? TIA
thank you very much
Pip install --upgrade awscli command not running in virtual environment
This is a masterpiece. Thank you again and again !!!
tq, good work brother.
Amazing content! Thank you brother. Please do upload more such videos!!
Bro, your explanation is really amazing. Nobody explain at that level. if possible can you start some videos on GCP cloud data engineering projects also. Thank you for great learning
Hey bro .... Did you complete this project on AWS , how much was the total cost or it was within the free tier limit
where I can get the airflow scripts for this project?
Where I can find the airflow scripts??
It's like T'Challa is teaching AirFlow! 🤣
Thank you! This video shows clear instructions!
Good tutorial for my learning, if possible create production ready project ( combination of local and cloud) Thank you😊
Thanks! Will surely do that. My website development is in progress.
Tutorial is really great. However, I am having some issues with loading the airflow UI. When I use ''airflow standalone'', and go to the 8080 port on the url from s3 instance it takes a long time to load the UI. Most times it doesn't load at all. I can't seem to understand what the reason is. Could you please help me?
Hugely underrated video that explains how to connect to the dvdrental.tar database. Precise and detailed. Outstanding job!
Hi bro. Congrats for the video!! I have experiencing a major issue, the airflow service and the SSH connection collapses after a few minutes of the instance initiation... What can I do?
appreciate your work
I saw the failed message again, but then I checked and the database had been restored. Thanks!
The scheduler does not appear to be running. Last heartbeat was received 32 minutes ago. The DAGs list may not update, and new tasks will not be scheduled.I have followed all commands you mentioned but i was getting like this in my airflow user interface and the command prompt which i connected to ec2 was running airflow standalone where to run new commands so that i can run airflow scheduler ? pls reply
I´m getting this error when execute airflow standalone ModuleNotFoundError: No module named 'tenacity.asyncio'
I solved this way pip uninstall tenacity Found existing installation: tenacity 8.4.0 pip install tenacity==8.2.1
amazing bro thanhs
After 7 hours trying to connect my instance, you really help me!
Glad it helped.
which app did you use to create the data pipeline visualization?
i'm still a bit confused, so is the AWS can be used freely, sir?🙏
Really helpful video!
Explanatory!
Very detailed!
Really helpful video!
Really helpful video!
Was a really helpful Bootcamp! 👏💯
Really helpful video!
Really helpful information!
Hi, Thanks for the video. It definitely solved my confusion points but still I have one doubt. Why airflow ? I mean we could've used the lambda for that work too right?
Excellent Tutorial
Thanks for your comment.
Thank you very much for making the concepts so easy to understand👌
You are welcome!
I am getting following error while creating a virtual environment. E: Unable to locate package python3.11.9-venv E: Couldn't find any package by glob 'python3.11.9-venv' If anyone can help then it would be great!!
Hey jay try this sudo apt install3-venv
You are superb brother..
Thank you so much 😀
Please make a video on self Join
Thank u sooo much. It helped
Glad it helped
Really great content! I just have one question, at 26:40, the part where you checked the Inbound Rules of VPC Security Group for the Redshift Cluster, is it standard practice to allow all inbound traffic and all IPv4 (Type = All traffic, Source=0.0.0.0/0)? AWS kept showing the warning to limit inbound traffic to only known IP, so I tried to set Inbound Rules to MyIP or the Public IP address of the EC2 instance I'm using to run Airflow, but for some reason step 4 failed to work when I do so (it worked fine if I follow the setting in your video for the inbound rule). Hope to hear your thoughts on this
The best ETL video I have ever come across. Thank you sir ❤🔥❤🔥❤🔥💯💯
Glad you liked it!
Absolutely wonderful tutorial, thank you for the great content man! I learned a ton from your videos and now subscribed from your channel
Thanks for the sub!
i get my load_data task failed i configured everything right but still get failed for the last task i couldnt figure it out anyone with the same scenario got any soln?
This is just what I have been searching for, thank you good sir, please kindly post more videos, you are awesome
Thank you!
te quiero mucho :D estos videos siempre son la respuesta
Thank you!
Solution to ODBC Error: Go to the security group of the Redshift cluster, click the box for the security group and edit inbound rules, Port range => 5439, source => Ipv4 Everywhere, save
Thanks alot for this video, I learnt alot from it
Glad to hear that!
Amazing content!! My friend 🙌🙌
Thank you 🙌
Excellent information, thank you so much for posting this video here
Glad it was helpful!
If you have problems with installing dependencies it is because instead of sudo apt install3.10-venv, replate it to sudo apt install3-venv to get the latest version. Currently, it's at 3.12
Invalid operation upon using that.
This worked thanks