1. Programming Language (Python, Java, Scala) 2. SQL 3. Basic Linux Commands 4. Data Modelling 5. Big Data 6. Spark and/or Kafka 7. Data Orchestration (Airflow) 8. Cloud
Thank you Darshil only a self-taught Data Engineer can understand the struggle of finding suitable sources for learning. There is so much information out there so many roadmaps to follow not saying that those roadmaps are bad, but I have experienced that it is very easy for a fresher to get lost in this loop of roadmaps. I will definitely follow this roadmap because first of all, it is free (LOL) and second it is from you a self-taught successful freelance Data Engineer. Cheers Mate :)
Wonderful Darshil.....This is great stuff actually. I think you are making someone's life so easy with your readysetgo document. God bless you and your family.
Roadmap: 1. Research on what is DE. - Videos recommended by darshil. 2. Basic fundamentals, playlist by Harvard, 07 videos. 3. Programming Language: Python. 4. SQL, most important skill. - Hacker Rank for practice. - Ankit Bansal for Interview on SQL 5. Linux. - Basic commands. 6. Projects. - Data model, very important. 1-2 months. 7. Big data. - Pipelines - File formats. - OLAP Vs OLDP. Etc. 8. Simplilearn, course. Theory as of now. 9. Data warehouses. 10. Practical. - Batch processing and stream processing. (SPARK and KAFKA) 11. Data orchestration - Airflow. Tweepy and Pandas, libraries. 12. Cloud Computing. - AWS or Azure (Same fundamentals) - ETL Pipelines in AWS - Other projects.
Bro I am bsc cs graduated student but an average student will data engineering will be easy for me or should I complete first msc cs and then go for it what will be better pls reply 🤞
Darshil please make videos of python interview questions for data engineer. Like sql questions that you make earlier. I believe that will really help full for all.
I really want to learn data engineer so badly and i have no background in program so this video really help me I will start to learn for one hour everday and after 5 days i will make 2 hour and so on Thank you very much 💜
Hello brother, I am from Infosys Limited. I am working on a BI/DMDW project as an ETL developer. We are using information cloud (IICS) in our project. In my free time , we have a learning platform where i upskill myself in the domain of data engineering. Like doing Microsoft Certified Data Engineer Certification. Learning real time data processing etc. And i will complete two personal projects after my learning will complete. Is it fine to crack another company interview?. Having 2.5 years of experience in Infosys in the domain of ETL and switch into Data Engineer
Hey, Currently am responsible for maintaining and designing the Data pipeline for end to end data flow. Am pretty much interested to learn and grow in the same field ie., Data engineering. So doing Masters in Data engineering is a good option..??
Hi, I have 3.5 years of manual testing experience and now I want to switch for good career path which will be in demand and useful for me. Please give your suggestions, I am skeptical at the moment what should I learn. Whether Data science would be good career path or not.
Can u please make a detailed video like this for the roadmap of data scientist skills choosing the free courses on yt That would be of really grt help 🙏 and even answer that can I become directly a data scientist as a fresher
Thanks a lot for this informative roadmap for data engg. Would you pls come up with some information regarding how to get clients/projects in this segment as a freelancer.. (as I have no intention to land-up to any job, so it is important to know, even before I start learning, if it is possible to grab freelancer projects in data engineering). Thanks in advance for your help...
bhaiya data engineer m fresher k liye itni kaam job kyu hoti h or isse career start krna jyd shi h y phle frontend s suru krkw bad m isme shift hona ??
Basically your Data Engineering skillset looks like this: RDBMS Database + NOSQL + Datawarehousing/Data Lakes + Traditional ETL/Custom ETL (You can learn both) + Python/Scala/Java + Distributed Computing + Visualization Tools Tools to learn (One of your choice): RDBMS Database - Oracle, MYSQL, Postgre SQL, MS SQL etc NOSQL - MongoDB, Dynamo DB etc Datawareshousing/Data Lakes - Includes concepts about data architecture. Snowflake, AWS Redshift, Bigquery, Synapse Analytics are some examples. Traditional ETL - Informatica, AWS Glue, Matillion, Talend, Azure Data Factory etc., Custom ETL - This is combination of a data orchestration tool and your python data transformation libraries. Here you write custom code for transformations using python. Airflow for orchestration, Python and Pandas for Transformations. Distributed Computing - Spark and Hadoop (Not complete alternatives of each other) Visualization Tools - Tableau, Power BI, Looker etc., Keep adding these skills with the tools of your choice. If you ask me RDBMS + Datawarehousing + ETL are the must have and should be started with.
I'm in love with this content. I recently read a similar book, and I was completely in love with it. "Mastering AWS: A Software Engineers Guide" by Nathan Vale
They both have lot of overlap skills but have similar path, in data science you might have to learn extra tools and skills such as Machine Learning and modelling I have Data Science roadmap ruclips.net/video/kU58_NaEjCk/видео.html and ruclips.net/video/fzyv5wxAeG8/видео.html
if you are from india, i would suggest you to consider CSE, because in most colleges that provide these specialized courses mostly teach subjects of CSE.
Web 3 and Data Engineering are different! Data Engineering or in general data market won't go for atleast 10-15 years or more W3 will take time to adapt, people are still researching and working on it's applications
Great video sir... Sir I'm from a bsc physics background and i want to switch my career to data engineering... is it possible for me to become a data engineer without prior knowledge in all data engineering field?
Darshil I am a food technologist and having 4 years in food industry but now want to shift in data field should I learn on my own or else to take any course as I am having doubt after learning skills will I get shortlist with food tech background
@@dkboyz actually I am also want to start but There is so many things to become data engineer. I dont know whether i will get any interview call or not
These things can be learned when you need them, these are no mandatory at the start! If you get a job and there let's say you need to hit some APIs then you can learn then and there, you don't need to master API at this moment
1000 Likes and I’ll upload next project video soon❤️
DAMN! I wish I could give you millions of like ❤️❤️❤️
You have covered querys of every beginner who know nothing about DE. LOVE YOU DUDE ❤️✨
i wish i could like more than once!
1. Programming Language (Python, Java, Scala)
2. SQL
3. Basic Linux Commands
4. Data Modelling
5. Big Data
6. Spark and/or Kafka
7. Data Orchestration (Airflow)
8. Cloud
Thank you for posting this
Thank you Darshil only a self-taught Data Engineer can understand the struggle of finding suitable sources for learning. There is so much information out there so many roadmaps to follow not saying that those roadmaps are bad, but I have experienced that it is very easy for a fresher to get lost in this loop of roadmaps. I will definitely follow this roadmap because first of all, it is free (LOL) and second it is from you a self-taught successful freelance Data Engineer.
Cheers Mate :)
Wonderful Darshil.....This is great stuff actually. I think you are making someone's life so easy with your readysetgo document. God bless you and your family.
Dude thank you sooooooo much you have no idea how your channel helped me, i don't know what/how else I can do to thank you enough. 😭
Tons of valuable information in this video. Thank you Darshil ❤
Roadmap:
1. Research on what is DE.
- Videos recommended by darshil.
2. Basic fundamentals, playlist by Harvard, 07 videos.
3. Programming Language: Python.
4. SQL, most important skill.
- Hacker Rank for practice.
- Ankit Bansal for Interview on SQL
5. Linux.
- Basic commands.
6. Projects.
- Data model, very important.
1-2 months.
7. Big data.
- Pipelines
- File formats.
- OLAP Vs OLDP.
Etc.
8. Simplilearn, course.
Theory as of now.
9. Data warehouses.
10. Practical.
- Batch processing and stream processing.
(SPARK and KAFKA)
11. Data orchestration
- Airflow.
Tweepy and Pandas, libraries.
12. Cloud Computing.
- AWS or Azure
(Same fundamentals)
- ETL Pipelines in AWS
- Other projects.
Bro I am bsc cs graduated student but an average student will data engineering will be easy for me or should I complete first msc cs and then go for it what will be better pls reply 🤞
Completely ❤️ it. Thanks 🙏 for creating such a great Roadmap.
You are welcome
Darshil please make videos of python interview questions for data engineer. Like sql questions that you make earlier.
I believe that will really help full for all.
I’ll try my best Tejas
@@DarshilParmar thank you
Yeah darshil It will be more helpful if you make a video on this
bro , i need help. i know pyhton , sql ,dsa
now what to do next in data engineering roadmap
Thank you so much. Darshil
for sharing Road Map and free resources to learn Data Engineer
My goodness gracious. This video was great! Only proble was to find out how much more I still need to study if I want to become a data engineer 😂
Have you finished it?
Hi darshil , I am also from maharastra ❤️
Kay चाललं भावा मस्त मजेत ना .. 😀
Aurangabad ❤️.
👍👍
Majet bhava
@@DarshilParmar where do you live ?
Sending your way a warehouse full of gratitude. Comprehensive.
Thank you Saurabh
I really want to learn data engineer so badly and i have no background in program so this video really help me
I will start to learn for one hour everday and after 5 days i will make 2 hour and so on
Thank you very much 💜
Just found your channel!
Crazy content
Keep going!🎉
Thank you for this video can you make videos on GCP Data Engineering projects
Thanks thanks and a ton of thanks,this one is a much needed one.
Awsome Content,Amazing Road map==> for DE++ Free On You Tube.....Thanks a ton brother....Keep up the spirit....Respect from Bangalore....
Best Video that I found
That you youtube for recommending this amazing channel
thank you, very valuable infirmation
Hey darshil, you are a genuine guy. Just that.
Very Helpful Content, thanks!
Can you tell which certifications really helpful to give more weightage to the profile?
Heartily thanks Darshil✨
This ll helps us alot..
Where you provided link for all that spark and Kafka
Thank you for your instructions
Thanks a lot 🙂🙂 for the roadmap.
Hello brother,
I am from Infosys Limited. I am working on a BI/DMDW project as an ETL developer. We are using information cloud (IICS) in our project. In my free time , we have a learning platform where i upskill myself in the domain of data engineering. Like doing Microsoft Certified Data Engineer Certification. Learning real time data processing etc. And i will complete two personal projects after my learning will complete. Is it fine to crack another company interview?. Having 2.5 years of experience in Infosys in the domain of ETL and switch into Data Engineer
Yes it’s completely fine to crack other companies while you are learning new skills
@@DarshilParmar Thanks a lot bro. and thanks for giving reply.
Hi bro can contact u ...I need some guidance..it's about how to do project on my own
Thanks a lot, Darshil😇
Hey, Currently am responsible for maintaining and designing the Data pipeline for end to end data flow. Am pretty much interested to learn and grow in the same field ie., Data engineering. So doing Masters in Data engineering is a good option..??
Hi, I have 3.5 years of manual testing experience and now I want to switch for good career path which will be in demand and useful for me.
Please give your suggestions, I am skeptical at the moment what should I learn.
Whether Data science would be good career path or not.
what u chose?
Do it for data science please 🙏
One of the best video for Data engineers
fabulous information. i appreciate your efforts bro
Do we need this many learning’s , how about start data engineering career from learn Databricks and ADF
Can u please make a detailed video like this for the roadmap of data scientist skills choosing the free courses on yt
That would be of really grt help 🙏 and even answer that can I become directly a data scientist as a fresher
I already have two videos on this
ruclips.net/video/kU58_NaEjCk/видео.html
and
ruclips.net/video/fzyv5wxAeG8/видео.html
I like timestamp 0:03 😀
Darshil can you make for data science road map too..
Great! This video.
i see datavidhya is no more available, can you help to provide valid links, thanks
Gcp data engineering exam ko clear karne ke liye best resource bata de bahii
Ya azure
Kya pick Karu gcp vs azure
is it i5 8GB ram is enough for HADOOP,MAPREDUCE,HIVE,HBASE,OOZIE,SQOOP,JAVA?
Thanks a lot for this informative roadmap for data engg. Would you pls come up with some information regarding how to get clients/projects in this segment as a freelancer.. (as I have no intention to land-up to any job, so it is important to know, even before I start learning, if it is possible to grab freelancer projects in data engineering). Thanks in advance for your help...
hey, was wondering if you got successful after an year ..
thanks for making this video... really helps a lot for noob data engineers like me... 👍
You are welcome!
Thank you brother
Welcome
thanks a lot!
This is what I need. Done like your video
You are welcome
bhaiya data engineer m fresher k liye itni kaam job kyu hoti h or isse career start krna jyd shi h y phle frontend s suru krkw bad m isme shift hona ??
Hi Darshil please make the same in Hindi language so it easy to understand ...so please start making in Hindi language as well..
Basically your Data Engineering skillset looks like this:
RDBMS Database + NOSQL + Datawarehousing/Data Lakes + Traditional ETL/Custom ETL (You can learn both) + Python/Scala/Java + Distributed Computing + Visualization Tools
Tools to learn (One of your choice):
RDBMS Database - Oracle, MYSQL, Postgre SQL, MS SQL etc
NOSQL - MongoDB, Dynamo DB etc
Datawareshousing/Data Lakes - Includes concepts about data architecture. Snowflake, AWS Redshift, Bigquery, Synapse Analytics are some examples.
Traditional ETL - Informatica, AWS Glue, Matillion, Talend, Azure Data Factory etc.,
Custom ETL - This is combination of a data orchestration tool and your python data transformation libraries. Here you write custom code for transformations using python. Airflow for orchestration, Python and Pandas for Transformations.
Distributed Computing - Spark and Hadoop (Not complete alternatives of each other)
Visualization Tools - Tableau, Power BI, Looker etc.,
Keep adding these skills with the tools of your choice. If you ask me RDBMS + Datawarehousing + ETL are the must have and should be started with.
Thank you ❤️
Best content ever ❤
Thanks a lot sir ♥
This is GOLD.
Apki new video he resources ki paid
Ki un dono me se konsi better he
I'm in love with this content. I recently read a similar book, and I was completely in love with it. "Mastering AWS: A Software Engineers Guide" by Nathan Vale
Hello Darshil when you have youre live session..?
Soon…
Hi can you please make video on best courses from udemy for data engineer
I already have it - ruclips.net/video/sGPWVYriqW4/видео.html
Thanks
When we're learn data engineer and also need to learn pyspark, java... Etc
Hi I am sql support can i move to data engineering?
Will ChatGPT 4 replace data engineers, is it worthwhile to learn that? Thanks.
I have video on this please watch it
Motivated!!✌
Video is very helpful bhaiyya
Lol, The Seattle Guy recommends you and you recommend him. 😅🎉
Q) Does data analytics / Data scientist have same roadmap ?? There are alots of roadmap uploaded on various platform can anyone suggest ??
They both have lot of overlap skills but have similar path, in data science you might have to learn extra tools and skills such as Machine Learning and modelling
I have Data Science roadmap
ruclips.net/video/kU58_NaEjCk/видео.html
and
ruclips.net/video/fzyv5wxAeG8/видео.html
@@DarshilParmar thank you 🙏
Thank you so much bhai....
You are welcome bhai
Extract transform load?
@Darshil Parmar Thanks for this amazing content. It's really helpful and please keep providing this type of contents. It helps a lot. 🙏🙏
I’ll try my best
whet degree do you consider is better for a data Engineer position , data science Bachellor degree or a software Engineer degree ?
if you are from india, i would suggest you to consider CSE, because in most colleges that provide these specialized courses mostly teach subjects of CSE.
Hi Darshil could you please share the sheet link for reference Thank You In Adavance.
Is Dsa also necessary for data engineer role?
Yes
Hi darsheel......i m a data scientist .......n want to be a permanent freelancer ......plz help....
Just be good in logical . Rest is easy
Loved The Video
Thank you!!!
Thanks bro
You are welcome Pratik
Could you tell what will be the job of data engineering in web 3.0
Web 3 and Data Engineering are different!
Data Engineering or in general data market won't go for atleast 10-15 years or more
W3 will take time to adapt, people are still researching and working on it's applications
Great video sir...
Sir I'm from a bsc physics background and i want to switch my career to data engineering... is it possible for me to become a data engineer without prior knowledge in all data engineering field?
Yes, you can switch to Data Engineering just by following this roadmap
@@DarshilParmar thanku 😄
@@nakulmohan7263 have u switch into data?
Darshil I am a food technologist and having 4 years in food industry but now want to shift in data field should I learn on my own or else to take any course as I am having doubt after learning skills will I get shortlist with food tech background
have u got data job?
@@Somnath-je9nd Nai bhai I am still in learning phase
@@dkboyz bhai have u given any interview till date?
@@dkboyz actually I am also want to start but There is so many things to become data engineer. I dont know whether i will get any interview call or not
heyy is api not included in this path? or it is not important?
These things can be learned when you need them, these are no mandatory at the start!
If you get a job and there let's say you need to hit some APIs then you can learn then and there, you don't need to master API at this moment
where are the links that you described in the video
Description
@@DarshilParmar can you also do similar vedio of data science course step.by step
@@MrTechGuy9999 I already have it
ruclips.net/video/fzyv5wxAeG8/видео.html
you can enroll free CS50 course in edX
Yes it is but this series I mentioned is latest, not sure if it’s available on Edx
Is this sufficient bro 😊 can u say im trusting like footprints as fresher ❤ can u sah
Yes 100%
Tku
What about etl tool bro can u say it is important like informatica
hello Darshil ! i am from pune :D
is there any way to get in touch with you ?
I am soon going to open 1-1 session for this
True that even paid service also waste of money and time
Python ops imp?
drive pdf link??
♥️♥️♥️♥️
link does not work
It is working
Hey much needed video. I have fallen for you for selflessly sharing things , after watching all your video.Will you marry me 🙃!!!!
After so long.
Shivansh thanks for support from the start
@@DarshilParmar ❤️🙌
You are a youtuber and not a data engineer. Period.
I can be both! On weekday I work on my projects and on weekends I do my RUclips stuffs to share knowledge and experience to other people
No se le entiende un pingo!
bhai agar apka course khreed lun 5000 wala usme sb milega????????????😝😝😝😝
Thanks a lot...!
You are welcome