data engineer roadmap 2025
HTML-код
- Опубликовано: 8 фев 2025
- In this video, I have talked about a roadmap that you can follow to become data engineer in 2025.
My Resume:- github.com/man...
Please read md file first and then do the changes
Python:-
Thoery:-
My channel:- • Python Fundmental
Telusko:- • #0 Python for Beginner...
Code With Harry:- • Introduction to Progra...
Practice:-
Hackerrank:- www.hackerrank...
Geeksforgeeks:- www.geeksforge...
Leetcode:- leetcode.com/p...
SQL:-
Theory:-
w3 Schools:- www.w3schools....
Risabh Mishra:- • What Is SQL ? | Databa...
Ankit Bansal:- / @ankitbansal6
Practice:-
Hackerrank:- www.hackerrank...
Leetcode:- leetcode.com/p...
Datalemur:- datalemur.com/
Spark:-
My Playlist:- • spark fundamental (The... , • spark practical (DataF...
Prashant Pandey Sir:- www.udemy.com/... , www.udemy.com/...
Databricks community edition:- community.clou...
Azure Cloud:-
Wafa Studies:- • Azure Databricks , • Azure Data Factory , • Azure Synapse Analytic...
Ramesh Retnasamy:- www.udemy.com/...
Prshant Pandey Sir:- www.udemy.com/...
Warehouse:-
My Playlist:- • Data warehouse Modelling
Snowflake:- www.udemy.com/...
CI/CD:-
Git:- • Git Tutorial for Begin...
GithHub:-
Azure Devops:- • Day-15 | Ultimate Azur...
Airflow:-
Coder2j:- • Airflow Tutorial for B...
Marc course:- www.udemy.com/...
Rejected:-
1. Doordash
2. Addepar
Selected with offer letter in Hand:-
1. Walmart
2. Expedia
3. Thoughtworks
4. PayPay
5. Particle Black
6. Mindera
7. Status Neo
Selected but did not get offer letter
1. Nielsen
2. EPAM
3. British Petroleum
4. Gartner
5. Nemetschek
Did not hear from the HR
1. Cholas MS
Directly connect with me on:- topmate.io/man...
For more queries reach out to me on my below social media handle.
Directly connect with me on:- topmate.io/man...
My Second Channel -- / @competitivegyan1
Interview series Playlist:- • Interview Questions an...
Follow me on LinkedIn:- / manish-kumar-373b86176
Follow Me On Instagram:- / competitive_gyan1
Follow me on Facebook:- / manish12340
My Gear:-
Rode Mic:-- amzn.to/3RekC7a
Boya M1 Mic-- amzn.to/3uW0nnn
Wireless Mic:-- amzn.to/3TqLRhE
Tripod1 -- amzn.to/4avjyF4
Tripod2:-- amzn.to/46Y3QPu
camera1:-- amzn.to/3GIQlsE
camera2:-- amzn.to/46X190P
Pentab (Medium size):-- amzn.to/3RgMszQ (Recommended)
Pentab (Small size):-- amzn.to/3RpmIS0
Mobile:-- amzn.to/47Y8oa4 ( Aapko ye bilkul nahi lena hai)
Laptop -- amzn.to/3Ns5Okj
Mouse+keyboard combo -- amzn.to/3Ro6GYl
21 inch Monitor-- amzn.to/3TvCE7E
27 inch Monitor-- amzn.to/47QzXlA
iPad Pencil:-- amzn.to/4aiJxiG
iPad 9th Generation:-- amzn.to/470I11X
Boom Arm/Swing Arm:-- amzn.to/48eH2we
My PC Components:-
intel i7 Processor:-- amzn.to/47Svdfe
G.Skill RAM:-- amzn.to/47VFffI
Samsung SSD:-- amzn.to/3uVSE8W
WD blue HDD:-- amzn.to/47Y91QY
RTX 3060Ti Graphic card:- amzn.to/3tdLDjn
Gigabyte Motherboard:-- amzn.to/3RFUTGl
O11 Dynamic Cabinet:-- amzn.to/4avkgSK
Liquid cooler:-- amzn.to/472S8mS
Antec Prizm FAN:-- amzn.to/48ey4Pj
![Seungmin "그렇게, 천천히, 우리(As we are)" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/kAzmhLHePqU/mqdefault.jpg)
![The ONLY Data Science & AI Roadmap You NEED [Step-by-Step Guide]](/img/1.gif)
If you are using my resume template please change email to yours all the places. Now a days I am getting more interview invites than you guys on behalf of you due to the mistakes you have been making. Please please please change your email everywhere applicable.
😂😂😂😂😂😂😅😅😅😅😅😅
😂
I was exactly looking for this and planning to schedule a call with you for this today itself and you made this video and made it available to all for free. Great!!❤👏
How to schedule call with him?
@PiyushKumar-vc3qg topmate link is there in his LinkedIn profile
Congrats on your selection, very informative video !!!
Thank you for the shoutout 🙏
🙂
Is it good ??
Indeed it is! @@bhavyamt9987
Ankit Bansal Zindabad!!!
I absolutely love how you emphasis the power of hard work and that we must always believe in ourselves, your content is genuinely wholesome and I wish you all the best for this year Manish, you are really inspiring. I’ve sent a LinkedIn request to you.
Back to back such insightful videos. Thank you so much for the motivation in the new year!
Bhaiya congratulations
.. you are my inspiration ❤️
This is the best video i have seen in recent time, I was looking for a roadmap for Data engineering but most of the roadmap lacks clarity on the content to follow. Thanks for sharing this video and looking forward to more such amazing videos which will help budding Data Engineers.
Excellent work! The effort you put into this is truly outstanding.
Very detailed and honest approach, many people suggest many paid courses, but yours is truly efficient and your video is a proof that if you work hard you can achieve things, kudos to you!
Hats off to your success and your efforts in making such videos. Thanks for sharing
1. Missed Spark internals & optimisations (partitioning, shuffling, join strategies, 5S - skewness etc)
2. Pls make video on Docker desktop setup
3. Updated version of local spark setup (facing lot of issues with older version)
For your first point he already has full Playlist dedicated to spark. Everything this there in that. On top of that, once you complete it, you can go for spark problem solving.
@harshu0721 yeah but as it was mentioned specifically in SQL section, so in Spark also it was good to mention as I faced lot of questions on this topic in interviews
@@harshu0721 how did you used those 2 playlist, did you watched 1 video everyday from those playlist simuntanoeusly, since manish said we should watch them parallely
Thanks Manish Bhai... Maine Sari cheeze kar rakhi hai (including project on your channel) and all note and topic, tech stack was overwhelming ... Was looking for a revision plan... Thanks for the great video
You are very honest. You are straight forward and one thing i too believe in is having strong basics and real time experience is enough just brishup n sit for interview.
Thank you for your guidance manish . I have cracked many offers easily becasue of knowledge you shared .Kudos . All the best to you!
Congratulations brother.
Thanks for your guidance.
Each of your videos are ready helpful.
Thank you so much Manish for these amazing videos.
You are my truly inspiration, No marketing, straightforward roadmap 🔥
Bht badhiya samjhaya aapne .. experienced logon ke liye project ka bhi source bataiye.. bht help hogi ...bht bht dhanywaad..
Thanks for sharing such detail video for interview prep.
This is one of the Best Guidance videos for Data engineers. 🔥🔥
Thank you manish bhai for the detailed explanation.
Its time to replace the bookmarked data engineer roadmap 2023 video with 2025. Thanks for sharing at the right time!
Manish Bhai…. really great compilation 🎉
Congratulations Manish🎉
Aap such m gem ho!!
Plz make a video on Kafka & system design..
Thank you so much. I will definitely consider that for my future videos.
Eagerly waiting for you reply sir it will really motivate me !!
Hi, Thankyou Manish, because of your spark playlist I got my 1st offer for Remote Data engineer as a fresher in 7th semester. I'm loving it. I'm the youngest in my team who is still in college 🎉. Thankyou so much
Love you bro, 💓
Thanks Manish for this genuine roadmap
Please post a video 📷 for industry level data engineering project how do they look like , strategies used to build those .
bohot acha video ! Great insights !
Thanks for valuable information ❤
Hi @Manish, congratulations for the new job 🥳. Just a request, can you please create topics on kafka and CI/CD?
bhai mast video hai , really hand on 😆
Hi Manish, you are true inspiration. I have question for you.
1-Please let me know how did you negotiate your salary and what are the points you gave to HR in discussion?
2-How much expectations you had from next switch?
3-Will you also take consent from company for continuing your RUclips channel?
4- suppose if you are having 2 years of DE experience and currently you are working in different domain , how much should individual expect a hike from next switch if having 10lpa currently?
If you done DE project then you can except 2-3 year 14 lpa for more experience you can get more
Nice , you are motivating us
Thanks for the guidance
Data Engineering is one of the toughest job in the planet we have to put in lot of efforts.
there r peoples on utube adding endless tools that r required to be a data engineer feels overwhelming
As I said in starting, if you are already following someone please don't watch. This will confuse you. Choose any one roadmap and continue preparation
Hello Manish really appreciate your efforts for making this video. let me know if you and in your network knows about the gcp in demand services as i worked in hsbc and they r using GCP cloud to migrate their applications .bdw please keep making more videos. best one till now . thank you bhai.
Amazing bhaiya 🎉
Bhai ek 30 minutes ka video banao ki 2 months ka notice period kese handle kare. Jab aap naukri me profile update karte ho. Aur call start hota hai. To hr ka 1st question 'are u serving notice period '. Aap yahi se hi chhat gaye. Aur agar aap notice period 15 days bhi show karte ho then also aapko call Naa barabar ayega. Please ek video banaiye iss par.
Very insightful
Sach bola bro sabka luck acha nahi hota agar luckily job milgaya na vaha bhi hardwork karna hi padega....so better work hard and get job 😂
We need a Cloud Oriented DE Project From u. It's a genuine request Manish Bhaii
Hi Manish i am a big data admin having 5 year experience i want to switch in data engineer so should i follow data engineer role or senior data engineer role please suggest
Sir continue data engineering videos 🙏🙏🙏
Manish Bhaiya pdf books aap de sakte ho kya jo aapne dikhaya hai video me ?
hello manish.I'm a fresher.please make a roadmap video for freshers
Manish sir ... plz daily live class kijiye naa
Bhai very good content❤👏!! I have 6 years full stack web dev with C#, SQL, Aspnet, MVC, Core kya DE mei move karna sahi option hoga as per the market and demand? aur 6 month tak approach kaise kare learn one skill and continue solving questions and projects each month? also i would be eligible for beginner or senior role? Thanks!!
bhaiya main bihar se hu .construction me job krta hu 30 age ka hu .last 1 years se data analyst aur basic data engineer ki preparation kr raha hu .bhaiya future ko lekar presaan hu
Kya hua bro mai bhi iski taiyyari kr rah hu is there any job issue ?
@@Mr.Vampire07 bhai job hi ni mil rahi
@@thefacts4u503 aapna study kaha tak ki hai and konsi feild mai ?
@manish_kumar_1 can you Please also suggest what part from book Design data intensive application to focus more for system design round ? (Since you also mentioned you read the relevant part more)
Ek baar aap index dekh ke decide kare ki kon kon sa technology aata hai usi ke hisaab se padhe.
Hi Manish
Can you please make videos on Data profile if you have exposure on this
Hi Sir, I found below missed
1. Basic level system design
2. Is the syllabus mentioned for 2-4 YOE enough for product based, as I heard they ask database design, ETL pipeline design and some in depth questions
Hi Manish, Thanks for the detailed guide! Could you please share the PPT document that was used for the explanation?
1. apache beam
2. hudi
3. iceberg
4. delta
5. nosql
6. kubernetes
this will make roadmap complete.
Sir I have follow your all videos. Content is excellent and informative. I have 2+ years of experience in software development but I want to move in Data Engineer fields. It's possible and how many possibilities? I have skills - advance SQL, basic of python and do some projects in ML/AI.
I think you should book an appointment with him on top mate.
Hi, want to connect with you but your earliest timeslot is of 23 feb, at topmate can you connect early?
Nice Manish
Small question wo azure k playlist are 4 years old kya wo abhi bhi sufficient honge?
I am currently stuck in a mnc with shitty support role and want to transition my career into data science. Help me
Sir SQL leetcode all 'Medium' 'Hard' are so many. Wont it take years for this extensive practice problem completion?
bhai 6-8 years experience k ly what would be market standard salary if targeting product company ?
Hi Manish, ky aap jo bhi books k pdf dikhaye video mein usko share kr sakte hai, that would be very helpful.
Please share any best aws playlist?
Oops and design pattern kha se padhna Acha hoga? Any leads will be highly appreciated
IT domain me 10 years experience hai. Kya hum data engineer me move kar sakte hai courses karke? Thoda details share karo iske regarding
Roadmap me wahi to bataya tha. Aap starting wala portion fir se dekhiye
Bhayya, I've 2 years of experience working with Python, SQL, Django, Pandas, Numpy and Docker. Can I get a good opportunity following this roadmap ?
How to revise effectively?
can we get the pdf of this roadmap
Azure certification is mandatory for azure data engineering interview?
any plans to transition to MLE ?
Manish thanks for all the help. When you suggest us solving leetxode or hacker rank, should we use built in methods or the basic looping and all?
I have seen in many questions, if we write a code in cpp, it takes 15-20 lines of code but in python using built in methods, it’s just 5-6..
You can't use built in methods in interview. Line of code does not show how skilled you are. It's the approach to solve the problem which gives confidence to interviewer
For kafka and spark streaming is it important for us to have industrial experience ?
which python library is preferred for begineer level
Dude, Could you please make one video on end to end project using CI/CD.
Dude, already one project is there in a playlist. CI/CD part is not done. You can do that by watching few videos from youtube. It will take sometime for me to come up with this
plzzzzz start data engineering course
📌 waiting for the video of installation of Spark in docker
बिना ग्रेजुएशन वाला भी कर सकता है डिप्लोमा या सार्टिफिकेट कोर्स
hi sir thanks for the video i am a doing my masters currently can i get a DE job as a fresher with the skills and i been doing spark and aws from last 7 months ? thank you
Hello Manish Bhai,
I'm working as a PL-SQL Developer with 3+ years of experience. Can I follow this roadmap for my career transition to the Data Engineering field?
Sir can you do the same for Data Analyst roadmap?
In JD's nowadays , Data Engineering skills like AWS/GCP/Azure, Snowflake etc are demanding.
I have 2 year of experience in non IT domain can I switch to data engineer or not
Bro i am a fresher started just now learning data engineering
i have learnt python,librarys,sql,powerbi,excel
thinking to learn spark next
pls guide me should i continue or quit and take another path .
IM from tier 3 college
Is good to go with dbt?
Hi Manish I am using Aws Step Function for orchestration at my current role. Do i need to learn Airflow as well for interview??
Nope. As long as you are using any orchestration tool you don't need any other. You can learn the new one in next organization
@@manish_kumar_1 but Job Description mai generally they ask for that and If we did not incldue it our cv we might not even get shortlisted
for AWS we can refer Technical Guftgu.
Can I get your book pdf please . I need that
How can one gain experience without being given opportunities as a fresher?
if i have experience then why follow roadmap ??
Personal experience, you start with other stuff. I started with etl Informatica pwc and sql. Once you have your basics clear, you can learn spark and cloud and just translate your project in the new technologies you learnt...
@@harshu0721Brother can u guide me how to get experience in data engineer if I don't get job as fresher in this field
Means in which role should I get job if I am learning data engineering skills if Im not getting job in data engineer role
Can u specify me roles brother it will be really helpful pls anyone who knows reply me I'm really confused
Manish sir, aapne system desng wale point mai senior aur junior Data engineers bola, toh usme kitne years fo experinece the junior count honge? I have 1.5-2 years of experience so am I a junior DE or not, and system design theoritical book se padh le and company mai kuch khaas accha DE project nhi hai, toh bhi chl jaega kya, I have heard ki system desing round mai practical real life knowledge chahiye jo current company mai mil nhi rhi and next company creck krne mai wo lgegi
🙏
Sir I'm an SQL developer (plsql to be precise) I want to move forward as a data engineer, will my sql experience be considered if im applying for say 2 yoe data enginner role?
❤
🙏🙏Sir kuch guide dijiye mai last year of engineering mai hu data engineer kisi job ya internship available itni nhi hai,kyuki jyada tar fresher ko nhi lete toh mai as data analyst join karu as of now 🙏🙏🙏
What about HADOOP?
I am working as embedded software developer and now I want to move to Data engineering. I have 2 YOE, how do you prefer my switching process. :)
You can do that. But most of the opening in DE is after 3 YOE.
If possible can you make video on how can we search job and ways to apply for a job ? As yk there are many influencer made lots of reel but not working irl.
I think in next week I will upload that too. May be in 2-3 days.
@@manish_kumar_1 Thank you so much , That will help many student as well as professional 😊✨
Bhai I need help in figuring out spark optimisation, I’m clueless whenever I open spark ui if you can create a tutorial regarding that would be really helpful. If you can offer some personal guidance I’m up for that too
Already multiple video me maine baate kari hai. Aap spark fundmental wala playlist dekhenge to mil jayega sab. Kuch aur jo miss ho gaye hai uska kabhi time milega tab bana denge
For 1 year experience may i get interview call?
Very less chance. Just be open for the role and if you get any then just grab that
Airflow kitna aana chahiye basics ki medium level tak?
Basics hi puchte hai
W guy