tuplespectra
tuplespectra
  • Видео 77
  • Просмотров 244 182
Process Engineer to Machine Learning Engineer | Interview with Raphael | Exponential Salary Increase
#datascientist #motivationalstory #inspiry story
Listen to Raphael sharing his journey on how he made the career switch from Process Engineering into Machine Learning Engineering.
*Books I recommend*
1. Fundamentals of Data Engineering: Plan and Build Robust Data Systems: amzn.to/4bfzmvX
2. Grit: The Power of Passion and Perseverance amzn.to/3EZKSgb
3. Think and Grow Rich!: The Original Version, Restored and Revised: amzn.to/3Q2K68s
4. The Book on Rental Property Investing: How to Create Wealth With Intelligent Buy and Hold Real Estate Investing: amzn.to/3LLpXRy
5. How to Invest in Real Estate: The Ultimate Beginner's Guide to Getting Started: amzn.to/48RbuOb
6. Introducing Python: Modern Comput...
Просмотров: 421

Видео

Petroleum Engineer to Data scientist | Join Africans in Data Community to expand your network
Просмотров 3784 месяца назад
#datascientist #motivationalstory #communityservice #buildyournetwork In this video, Toyosi Bamidele discussed how she transitioned from a petroleum engineer to a data scientist and the motivation behind founding “Africans in Data Community”. She further explained how you can become a member of this community and the benefits you stand to gain when you become a member. In addition, she gave som...
Complete Roadmap/Guide on How to Become a Data Engineer in 2024
Просмотров 9115 месяцев назад
#dataengineering #dataengineeringroadmap In this video, I share with you a complete guide/roadmap on how to become a data engineer in 2024. You do not have to know everything highlighted in this guide before you start applying for DE roles. Please click on the google drive link below to request a view access to the google doc where I listed the roadmap. docs.google.com/document/d/1hWkKlPvZOBigm...
Tips to become a data engineer in 2024 - What no one ever told you | Keep Learning | Be Consistent
Просмотров 5906 месяцев назад
In this video, I will like to share with you what I have learnt while learning programming and tips to become a data engineer in 2024. *Books I recommend* 1. Grit: The Power of Passion and Perseverance amzn.to/3EZKSgb 2. Think and Grow Rich!: The Original Version, Restored and Revised: amzn.to/3Q2K68s 3. The Book on Rental Property Investing: How to Create Wealth With Intelligent Buy and Hold R...
Tutorial 3.1-How to load/restore DVD rental database to PostgreSQL server16 using pgAdmin| Windows11
Просмотров 3,4 тыс.7 месяцев назад
In this video I showed how to load or restore the DVD rental database to our PostgreSQL sever 16 on Windows 11 computer. We will use this DVD rental database for the next series of videos to understand the various SQL concepts from the beginner level to intermediate/advanced level. Please go to this link to download the dvdrental.tar sample database file www.postgresqltutorial.com/postgresql-ge...
Tutorial 2.1 - How to install PostgreSQL 16 and pgAdmin on windows 11 computer | Step-by-Step Guide
Просмотров 1,3 тыс.7 месяцев назад
I will be showing you a step by step guide on how to download and install PostgreSQL server 16 and PgAdmin on your windows 11 computer. Follow this link to download postgreSQL www.postgresql.org/download/ *Books I recommend* 1. Grit: The Power of Passion and Perseverance amzn.to/3EZKSgb 2. Think and Grow Rich!: The Original Version, Restored and Revised: amzn.to/3Q2K68s 3. The Book on Rental Pr...
Master SQL | Go from Zero to Hero | SQL Live Class BootCamp
Просмотров 9957 месяцев назад
I am excited to launch this 8-weekends intensive SQL bootcamp which will cover Basic, Intermediate and Advance SQL concepts. 👉 Training start date: Jan 27th 2024. 👉 Days: Saturday and Sunday 👉 Class Time: 5am PST | 6:30pm IST | 2pm WAT 👉 Fee: 40USD 👉 Certificate of Completion will be provided 👉 Register for BootCamp here: awesumedge.com/register-jan2024postgresql/ PostgreSQL Playlist: ruclips.n...
Airflow Tutorial | Automate EMR ETL Jobs with Airflow | Airflow Project | Data Engineering Project
Просмотров 2,1 тыс.7 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 7 - Terminate EMR cluster task | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 4217 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 6 - Create EMR cluster and Add steps tasks | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 1,3 тыс.7 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 1 - Project Overview | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 1,6 тыс.8 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 3 - Initialize airflow and explore UI | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 3978 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 5 - Connect VS code and create task 1 | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 5268 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 4 - Create S3 buckets and required folders | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 3118 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 2 - Provision EC2 and Install dependencies | Airflow Tutorial | Automate EMR Jobs with Airflow
Просмотров 7208 месяцев назад
#dataengineering #emr #airflow #spark #pyspark #aws #etlpipeline #redfin In this video, I explained how to use airflow to automate EMR jobs. I showed you how to create EMR cluster, poll the states of the EMR, add EMR steps and terminate the EMR cluster. Airflow was used to orchestrate the entire ETL pipeline. The EMR steps involved extracting redfin data from the redfin datacenter web address a...
Part 6 - Data Transformation|pyspark|What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 6468 месяцев назад
Part 6 - Data Transformation|pyspark|What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 5 - EMR Studio|Jupyter Notebook|What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 4408 месяцев назад
Part 5 - EMR Studio|Jupyter Notebook|What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 4 - How to Create EMR Cluster | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 4728 месяцев назад
Part 4 - How to Create EMR Cluster | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 3 - How to create AWS VPC | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 2998 месяцев назад
Part 3 - How to create AWS VPC | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 2 - Create User and S3 bucket | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Просмотров 3948 месяцев назад
Part 2 - Create User and S3 bucket | What is AWS EMR| Extract and Transform Redfin data with AWS EMR
Part 1 - Project Overview | What is AWS EMR | Extract and Transform Redfin data with AWS EMR
Просмотров 8878 месяцев назад
Part 1 - Project Overview | What is AWS EMR | Extract and Transform Redfin data with AWS EMR
What is AWS EMR | Extract and Transform Redfin data with AWS EMR | EMR Studio | Pyspark Notebook
Просмотров 8 тыс.9 месяцев назад
What is AWS EMR | Extract and Transform Redfin data with AWS EMR | EMR Studio | Pyspark Notebook
LeetCode 197 - Rising Temperature | Common SQL Interview Question | Self-Join | LAG window function
Просмотров 2739 месяцев назад
LeetCode 197 - Rising Temperature | Common SQL Interview Question | Self-Join | LAG window function
Redfin Analytics|python ETL pipeline with airflow|Data Engineering Project|Snowpipe|Snowflake|Part 2
Просмотров 2,1 тыс.9 месяцев назад
Redfin Analytics|python ETL pipeline with airflow|Data Engineering Project|Snowpipe|Snowflake|Part 2
Redfin Analytics|python ETL pipeline with airflow|Data Engineering Project|Snowpipe|Snowflake|Part 1
Просмотров 6 тыс.9 месяцев назад
Redfin Analytics|python ETL pipeline with airflow|Data Engineering Project|Snowpipe|Snowflake|Part 1
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 3
Просмотров 1,7 тыс.10 месяцев назад
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 3
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 2
Просмотров 4 тыс.10 месяцев назад
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 2
YouTube Channel Join Button | YouTube Memberships
Просмотров 12010 месяцев назад
RUclips Channel Join Button | RUclips Memberships
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 1
Просмотров 7 тыс.10 месяцев назад
Customer Churn Data Analytics|Data Pipeline using Apache Airflow, Glue, S3, Redshift, PowerBI|Part 1
Zillow Data Analytics (RapidAPI) | End-To-End Python ETL Pipeline | Data Engineering Project |Part 3
Просмотров 3,3 тыс.10 месяцев назад
Zillow Data Analytics (RapidAPI) | End-To-End Python ETL Pipeline | Data Engineering Project |Part 3

Комментарии

  • @sarthakrana5658
    @sarthakrana5658 12 часов назад

    @tuplespectra My S3ToRedshiftOperator task is not changing status in airflow. I have written the correct code, added and updated all the roles, policies and permissions and added the redshift connection in airflow, restarted the server as well. What am I missing? TIA

  • @FallenJakarta
    @FallenJakarta День назад

    thank you very much

  • @salmanshikalgar4482
    @salmanshikalgar4482 3 дня назад

    Pip install --upgrade awscli command not running in virtual environment

  • @collinspo
    @collinspo 4 дня назад

    This is a masterpiece. Thank you again and again !!!

  • @dheerajmadhu9419
    @dheerajmadhu9419 15 дней назад

    tq, good work brother.

  • @pareekshitgaddam9912
    @pareekshitgaddam9912 18 дней назад

    Amazing content! Thank you brother. Please do upload more such videos!!

  • @dudee420
    @dudee420 21 день назад

    Bro, your explanation is really amazing. Nobody explain at that level. if possible can you start some videos on GCP cloud data engineering projects also. Thank you for great learning

    • @avinash390
      @avinash390 14 дней назад

      Hey bro .... Did you complete this project on AWS , how much was the total cost or it was within the free tier limit

  • @basavarajpn4801
    @basavarajpn4801 22 дня назад

    where I can get the airflow scripts for this project?

  • @basavarajpn4801
    @basavarajpn4801 22 дня назад

    Where I can find the airflow scripts??

  • @rohan_mehra
    @rohan_mehra 22 дня назад

    It's like T'Challa is teaching AirFlow! 🤣

  • @binbinxia5750
    @binbinxia5750 24 дня назад

    Thank you! This video shows clear instructions!

  • @hafizadeelarif3415
    @hafizadeelarif3415 24 дня назад

    Good tutorial for my learning, if possible create production ready project ( combination of local and cloud) Thank you😊

    • @tuplespectra
      @tuplespectra 3 дня назад

      Thanks! Will surely do that. My website development is in progress.

  • @Nawar-t56
    @Nawar-t56 29 дней назад

    Tutorial is really great. However, I am having some issues with loading the airflow UI. When I use ''airflow standalone'', and go to the 8080 port on the url from s3 instance it takes a long time to load the UI. Most times it doesn't load at all. I can't seem to understand what the reason is. Could you please help me?

  • @soccershots6297
    @soccershots6297 29 дней назад

    Hugely underrated video that explains how to connect to the dvdrental.tar database. Precise and detailed. Outstanding job!

  • @xavierromerocarrion1369
    @xavierromerocarrion1369 29 дней назад

    Hi bro. Congrats for the video!! I have experiencing a major issue, the airflow service and the SSH connection collapses after a few minutes of the instance initiation... What can I do?

  • @AndrewLapteff
    @AndrewLapteff Месяц назад

    appreciate your work

  • @jamveher
    @jamveher Месяц назад

    I saw the failed message again, but then I checked and the database had been restored. Thanks!

  • @jrohit9664
    @jrohit9664 Месяц назад

    The scheduler does not appear to be running. Last heartbeat was received 32 minutes ago. The DAGs list may not update, and new tasks will not be scheduled.I have followed all commands you mentioned but i was getting like this in my airflow user interface and the command prompt which i connected to ec2 was running airflow standalone where to run new commands so that i can run airflow scheduler ? pls reply

  • @josecardons6221
    @josecardons6221 Месяц назад

    I´m getting this error when execute airflow standalone ModuleNotFoundError: No module named 'tenacity.asyncio'

    • @josecardons6221
      @josecardons6221 Месяц назад

      I solved this way pip uninstall tenacity Found existing installation: tenacity 8.4.0 pip install tenacity==8.2.1

  • @josecardons6221
    @josecardons6221 Месяц назад

    amazing bro thanhs

  • @Programmer22-pv4xs
    @Programmer22-pv4xs Месяц назад

    After 7 hours trying to connect my instance, you really help me!

  • @sophialawal7306
    @sophialawal7306 Месяц назад

    which app did you use to create the data pipeline visualization?

  • @themagniranthine
    @themagniranthine Месяц назад

    i'm still a bit confused, so is the AWS can be used freely, sir?🙏

  • @kehindefagbohungbe
    @kehindefagbohungbe Месяц назад

    Really helpful video!

  • @kehindefagbohungbe
    @kehindefagbohungbe Месяц назад

    Explanatory!

  • @kehindefagbohungbe
    @kehindefagbohungbe Месяц назад

    Very detailed!

  • @kehindefagbohungbe
    @kehindefagbohungbe Месяц назад

    Really helpful video!

  • @kehindefagbohungbe
    @kehindefagbohungbe Месяц назад

    Really helpful video!

  • @kehindefagbohungbe
    @kehindefagbohungbe Месяц назад

    Was a really helpful Bootcamp! 👏💯

  • @kehindefagbohungbe
    @kehindefagbohungbe Месяц назад

    Really helpful video!

  • @kehindefagbohungbe
    @kehindefagbohungbe Месяц назад

    Really helpful information!

  • @coolkid24760
    @coolkid24760 Месяц назад

    Hi, Thanks for the video. It definitely solved my confusion points but still I have one doubt. Why airflow ? I mean we could've used the lambda for that work too right?

  • @viishhnudatta5124
    @viishhnudatta5124 Месяц назад

    Excellent Tutorial

  • @mdobaidullahal-faruk3457
    @mdobaidullahal-faruk3457 Месяц назад

    Thank you very much for making the concepts so easy to understand👌

  • @JayPatel-yf7ot
    @JayPatel-yf7ot Месяц назад

    I am getting following error while creating a virtual environment. E: Unable to locate package python3.11.9-venv E: Couldn't find any package by glob 'python3.11.9-venv' If anyone can help then it would be great!!

    • @adamsapple3070
      @adamsapple3070 Месяц назад

      Hey jay try this sudo apt install3-venv

  • @JoysonFernandes-kd1zx
    @JoysonFernandes-kd1zx Месяц назад

    You are superb brother..

  • @exalteduzondu6643
    @exalteduzondu6643 Месяц назад

    Please make a video on self Join

  • @user-ob5ms3tn7e
    @user-ob5ms3tn7e Месяц назад

    Thank u sooo much. It helped

  • @tuananhdo6006
    @tuananhdo6006 Месяц назад

    Really great content! I just have one question, at 26:40, the part where you checked the Inbound Rules of VPC Security Group for the Redshift Cluster, is it standard practice to allow all inbound traffic and all IPv4 (Type = All traffic, Source=0.0.0.0/0)? AWS kept showing the warning to limit inbound traffic to only known IP, so I tried to set Inbound Rules to MyIP or the Public IP address of the EC2 instance I'm using to run Airflow, but for some reason step 4 failed to work when I do so (it worked fine if I follow the setting in your video for the inbound rule). Hope to hear your thoughts on this

  • @nicholasmageto6110
    @nicholasmageto6110 Месяц назад

    The best ETL video I have ever come across. Thank you sir ❤‍🔥❤‍🔥❤‍🔥💯💯

  • @980Jair
    @980Jair 2 месяца назад

    Absolutely wonderful tutorial, thank you for the great content man! I learned a ton from your videos and now subscribed from your channel

  • @RaghulS-nl6wx
    @RaghulS-nl6wx 2 месяца назад

    i get my load_data task failed i configured everything right but still get failed for the last task i couldnt figure it out anyone with the same scenario got any soln?

  • @tuananhdo6006
    @tuananhdo6006 2 месяца назад

    This is just what I have been searching for, thank you good sir, please kindly post more videos, you are awesome

  • @thepig13yt81
    @thepig13yt81 2 месяца назад

    te quiero mucho :D estos videos siempre son la respuesta

  • @ataimebenson
    @ataimebenson 2 месяца назад

    Solution to ODBC Error: Go to the security group of the Redshift cluster, click the box for the security group and edit inbound rules, Port range => 5439, source => Ipv4 Everywhere, save

  • @ataimebenson
    @ataimebenson 2 месяца назад

    Thanks alot for this video, I learnt alot from it

  • @xavierromerocarrion1369
    @xavierromerocarrion1369 2 месяца назад

    Amazing content!! My friend 🙌🙌

  • @narasa12
    @narasa12 2 месяца назад

    Excellent information, thank you so much for posting this video here

  • @kristiandaclan9236
    @kristiandaclan9236 2 месяца назад

    If you have problems with installing dependencies it is because instead of sudo apt install3.10-venv, replate it to sudo apt install3-venv to get the latest version. Currently, it's at 3.12