🚀 Best Tech Career for 2023 - Data Engineering
HTML-код
- Опубликовано: 26 июл 2024
- Join Course Python for Data Engineering - datavidhya.com/courses
In this video, we'll take a look at the key highlights and milestones of the past 5 years in the field of data engineering.
From the rise of big data and cloud computing to the emergence of new tools and technologies, we'll cover it all in just 10 minutes.
Whether you're a seasoned data engineer or just starting out, this video is a great way to get a quick overview of the key developments in the field. So join us as we take a quick tour through 5 years of data engineering!
Join Data With Darshil Discord Server: / discord
Timestamps
0:00 Introduction
0:04 Understand Business
0:39 Storing Data
2:20 Coding
2:51 What is Data Engineering?
3:42 Why Data Engineering?
4:15 Data Pipeline and Processing
6:35 Data Engineering tools
7:38 Cloud Platforms (AWS/GCP/Azure)
8:20 Modern Data Stack (Fivetran, Snowflake, dbt)
Follow me here for more content:
🎯Twitter - / parmardarshil07
👦🏻 My Linkedin - / darshil-parmar
📷 Instagram - / darshilparmarr
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
3 Books You Should Read
📈Principles: Life and Work: amzn.to/3HQJDyP
👀Deep Work: amzn.to/3IParkk
💼Rework: amzn.to/3HW981O
Tech I use every day
💻MacBook Pro M1: amzn.to/3CiFVwC
📺LG 22 Inch Monitor: amzn.to/3zk0Dts
🎥Sony ZV1: amzn.to/3hRpSMJ
🎙Maono AU-A04: amzn.to/3Bnu53n
⽴Tripod Stand: amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: amzn.to/3IRpiL2
👉Data Engineering Complete Roadmap: • Data Engineer Complete...
👉Data Engineering Project Series: • Data Engineering Proje...
👉Become Full-Time Freelancer: • Best Freelancer Series...
👉Data With Darshil Podcast: • Podcast Series - Data ...
✨ Tags ✨
become data engineer, fast way to become data engineer, data engineer vs data science, how to learn data science, how to learn big data
✨ Hashtags ✨
#dataengineering #aws #bigdata
Don't forget to hit LIKE button and share your thoughts and experience in comment section 👇🏻
Bhai data analytic for finance kai liye bata do koi sasta course with benefit certificates
Doing GREAT 🙏💕💲
You answered a lot of my questions. Thanks a lot
Very informative!
Thanks for putting this video together! 👍
Great, I was searching this for a long time. This helped to understand what tools/programs fits on which area as clear as possible. There is only one like button, wish there should had more👍🏻
Thank you
This video cleared many of my doubts Darshil. Thank you.
This is amazing, 10 mins of 100 min information. Thank you do much very informative and straight to the point I am a software engineer and curios what Data Engineering is and this answered my questions.
Excellent information. Thank you.
This is gold content. Thank you!
In depth information
Great work 💥
Hi Darshil you are doing great job and Tq for for uploading the projects that you did and I wanted you to explain an end to end projects without using any cloud services so that it will give more clarity.
Great bro 😊
This video clears some of my doubts. Great Video Thanks
Glad it was helpful!
Smart and simple way to know data engineering....👍
Businesses collect data, not for service or security, or product improvement but to find ways to maximize profit by escaping legal regulations and aggravate charge when high demand
prefect video thanks bro
Great Explanation!! As a Java back end developer who wants to move to DE this will be really helpful. Thanks once again
Hi darshil, you have covered all things in this video.
I have worked on Both ETL and Analytics. I will say that we use programming very rarely. Instead we use lot of SQL and make use of lot of tools. No data engineering or analytics happens without tools.
Hi, can you let me know as per your experience what all skills are required to become a Data Engineer? I'm a Front-End developer, I'm planning to switch to Data Engineer as I don't want to code anymore. I really feel exhausted and difficult coding.
@@nike0822 wow dude....i started watching this video because i am going through the same situation that you just described.
@@nike0822 Data Engineering won't make you escape coding.
@@ayanwoyegideonayandele3729 data engineering depends 80% on tools
That's why data engineering is boring because of tools
insightful, thank you.
You're very welcome
Best explanation 🔥hoping to see more videos like this❤
Brother, want to hug you and thank you. What clarity toward these vast subjects. If you somewhat start your classes on these topic I sure want to join.
Thank you and I am soon coming up with courses on Data Engineering
Good overview for beginners to understand data engineering
Great video and very Informative please bring more videos with real time animation.
Darshil Darshil just brilliant and simple. Also can you please make a playlist on spark? It'll be really helpful.
Thank You!
I am currently working on Python for Data Engineering course, I will launch Spark course in the future for sure
Darshil you are my hero in data related field.love you so much .AdvanceMerry Christmas and advance happy new year
Same to you
Easy to understand video man. Keep doing it for us❤️✌️
Thanks, will do!
Great 👍
So well explained thx 🙏
You're welcome!
Thanks 🙏💕
Thanks brother ❤️
Always welcome
Every thing is there in salesforce
Thank you so much bro ❤❤❤❤
You have explained total flow of data engineering in few minutes
hats off to you bro
Great work🙏🙏
Thanks
In 2023, we should take look to Apache Beam. Lot of advantages over Spark !
Best source to learn AWS? Where did you learn? Please do tell
Is MacBook m1 8gb 256 enough for data science?
Hi Darshil,
I want to thank you for the information you share with all the people out there who need it the most.
I am one of them, after listening to your podcast with a technical suneja, i was really inspired. Currently, I help my family to run a small business. I saved a lot of money the entire 2022 and bought a laptop so that I can get started with data science studies.
Unfortunately, my laptop was stolen a month ago and I am doing some odd jobs so that I can buy a new one.
My question to you is, until I get a new laptop, is there something I can do/learn theoretically about data science?
I may need another 3-4 months to save and purchase a laptop but 3 months is a long time and I don't want to waste time.
Is there something I can read about data science or some theory? Maybe a book?
I know, most of data science is practical and projects are the best way to learn but until I get a new laptop, I want to make the most of out the time I have.
I hope you would be kind enough to reply and help me.
Thank you! 🙏🏻
(Following your channels, because the way you teach concepts is really appealing)
Read articles online
Can you please do videos on how can we contribute to open source projects?
Can u make video how to become powerbi data analyst or power bi devloper
Please launch your python and sql course for data engineering.
Hello! Thank you for your invaluable video! I find it extremely useful for beginners! I would like to ask about one thing regarding this map. I learnt Pandas in terms of Data Wrangling and Transformation. Therefore, how about Pandas for Data Engineers? Is it useful tool for ETL/ELT transformations? Obviously, the next step will be PySpark, but I would like to start learninig Pandas. It seems it is a good path for the next one. What do you think about it ? I would appreciate it if you could share your views about it.
Hello,
Pandas is good for understanding fundamental transformation and how to handle data. Even when you learn PySpark you will be dealing with Pandas DF when you want to perform some operations on top of RDD so you will convert RDD into DataFrame to do some work!
Continue what you are doing, you are on the right path
@@DarshilParmar ok thank you for your advice!
how can i find job in DE
Hi Darshil,
Thanks for the video.
Can you suggest me some data engineer courses?
East or west darshil bhaiya is best🫶🏻
Bro best
Thank you!
finished watching
🤩
super bro ❤️
Thank you so much
Currently I am doing 1st year AIDS . Please tell me how to become a data engineer within 4 years . Please guide me what I do in 1to 4 years
Hi Darshil,
I self taught data analytics in the last 1 year + 6 months, and made a few projects but I am finding hard to get a foot in the door. I am changing career.
In 2023, I am planning to upskill myself more and try data engineering as I see many jobs showing up in DE but I come from from hotel hospitality domain(Non STEM domain) & have an unemployment gap so I am little scared if lacking a STEM degree would impact my selection for an entry level job in India?
Since you are in the field, hope you can give some good advice.
It's not about degree anymore. Upskill ans you will definitely find a door
Brother do u think Salesforce is also best carrier ?
It's a good career but again it's a tool based so you are conditioned on one tool and you will get job in that field only
Start with salesforce and then explore more as you go forward
@@DarshilParmar okay thank you
Hi darshil , can you please let us know why did you left web development for data engineering ? Was it due to DSA , or due to number of opportunities in Data Engineering ?
Watch this video - ruclips.net/video/AS2EyYK4x_Q/видео.html
@@DarshilParmar thanks
Hii Darshil,
Currently, i am working as an ERP Functional Engineer Should I go for Data Engineering.
Badiyaa yaar acha samjhaya tune 👌
Thank you
fresher with NON IT domain exp...can i choose data Eng .. or should i choose full stack first after gaining some expe.. then move to DATA Field ahh
kindly guide me ??
update?
Most underrated Data Engineering content creator @DarshilParmar 💯.
Great video my friend, just one request:
Pls add some subtitles 🙏🏼 auto cc is useless
Sure, I will do it from the next videos
Hello Darshil first of all thank u very much for uploading this video which i really needed. But I have a doubt that how much MATHS is required in Data science or Engineering.
Data Engineering core work does not really required that much math knowledge, analytics side of data engineering do require basic math that too only college level
Probability, Basics Stats and Algebra
Don't worry much about math for data engineering
@@DarshilParmar ok brother thank you very much for clearing this big doubt .😇😇😇😇
Darshil when your python course will be launched
Means how much time it will take to launch
Two weeks mostly, many things to cover so I’ll half launch initially then will keep uploading more
@@DarshilParmar i am very excited
List some resources to get be updated in Data engineering .
Check this out - ruclips.net/video/sGPWVYriqW4/видео.html
Hello bro.
Hope you are doing well.
I am hadoop admin. 2 years of experience in Hadoop Ecosystem mainly administration roles. like launching deploying managing big data hadoop clusters experience in AWS , Cloudera. Never learned any programming language. neither Pyton nor SQL .
Currently lost the job and now there are no hadoop admin vacancies . So what will be the best option for me ?
1. switching my career into Data Engineering by learning python, pyspark , and sql ?
Or
2 learn python django first .. go into development part.. and after gaining work experience on python .. move to pyspark and sql .
Really confused .. please guide
Hi, i might not be an expert you're looking for, but currently working as an ETL Developer.
In my country, usually it's starting from ETL role then it progresses to Data Engineer.
I think if you wanted to switch to data engineer, you can look for an ETL role, it's not really steep learning curve, mainly you can look to learn abour data warehouse, snowflake schema, star schema, fact and dimension table.
Start your own projects using sql and etl tools, you can start using like postgresql and pentaho (etl tools). And create the fact and dimension table, usually a company uses the SCD type 2.
I wish you luck :)) keep up the good works
@@andrewmaxwell9399 Thank you so much Andrew !
@@andrewmaxwell9399 Currently skilling up myself in MySQL and Basic Python .. I will go through schema and ETL stuff.. Thanks for the guidance buddy !
Hi Darshil, this was good content....
Just want to suggest an video which might be helpful for other data engineers/aspiring data engineers?
Daily routine of Data Engineers? What to learn how much time to spend on learning new skills for professionals? How to target for product based companies and where is DevOps coming in picture?
RDBMS + SQL + Hadoop + Cloud + Kafka/realtime processing - All these technologies are huge and has various flavours so how to stitch all this technologies and learn in one go?
Please make an video on this or an live podcast where we can talk about?
Hi Ravi,
Thanks for the suggestions, I will keep this in my list and see what I can make out of it
Is Spark and MapReduce in high demand?
Yes
Bro we are waiting for your videos since from 2 weeks
I am sick bro, I’ll upload as soon as I recover
@@DarshilParmar ohh k bro sorry
Get well soon bro ♥️
Where's your GitHub pal? 😎
He doesn’t have a GitHub?
Which data engineer is good scope - Azure or GCP
Azure
Sir i am completed big data Hadoop course as fresher In BSc.....how I can make my entry in big data job?
You can
@@DarshilParmar can you tell me how can I get a job in the u.s. what blotcamp or what career I can do to get a job?
After 12th what we can do a degree or online learning for data engineering 🤔 can we get job without a degree 😀
You can get job without degree if you have right skills and projects in the field
Azure is the top of the letter? Hahahhaha
It’s just a thumbnail ;)
Speak english is also important!
Bro.. be more dramatic to gain subscribers quickly.
Thank you, I'll see what I can do
Dramatic is ok, but be truthful as you always✌️