I'm a freelancer and also self-taught programmer who wrote a lot of Data Scraping scripts using Python for my clients. I have a loyal client who requires me to extract data for them every month. I repeated this manual process for about 3 months until I realized the scraped data was getting bigger in size and dimension. I thought it was not a data scraping job anymore, it was a data engineering job. After doing some research online, I figured out some useful tools I can use to manage all data collected by the data scraper. I start by storing all the data in an SQL database and automating the process using Apache Airflow. That was the first time I thought Data Engineering was the right direction for my career path as a developer. Thanks for your advice, Jay.
Hands done, the best video I have seen on the subject. It's no nonsense, practical and easy to digest (of course after you put in the work). I am just starting out in my journey to become a Azure Data Engineer so I thank you Jay. Thanks for sharing and being so transparent. I'll keep you posted on my many adventures. For what it's worth, I'm 70 years young and still dedicated to life long learning.
What a gem. I am currently pursuing AI/ML. Getting data scientist job as a fresher without experience is almost impossible so had to start with either as data analyst or data engineer. Since now everyone is becoming data analyst so their is demand and supply issue in the job market. Thinking of expanding my domain knowledge to data engineering as well but as you stated “got lost” with the available resources. This video helped me to make a roadmap to pursue data engineering.
Thanks for sharing. I agree it's super difficult to get a DS job fresh out of college. Branching out to DE is a great way to "stand out" from the crowd.
i'm a junior data scientist transitioning to data engineering/MLOps. i'm also based in nyc! i'm recently discovering my interest in filmmaking too! looks like i stumbled upon your content at just the right time. just subscribed!!!
I recently signed up for the data engineering course from Datacamp and I have covered pretty much most SQL courses but I'd love to go back on a few of them again, I work as a data analyst but I feel a career in Engineering would be great.
Hi, I’ve done the complete python track of data analyst, mostly of the sql courses of DataCamp. I’m from Chile, Latin America, do you have any advice for me to find remote jobs as data analyst in the US?
I'm starting my learning journey from here. It's a completely new world for me, both because I'm not a native English speaker and because I'm totally new to all this technology. I've made up my mind: I want to become a Data Engineer, and I'm going to work really hard to achieve that! Thank you so much for the guidance; I’m going to put it into practice now.
I am transitioning from web development to data. Fortunately for me, I learnt development skills from Learnenough. I have gone through their Commandline course and now on Python and Python for Data Science. I am excited to be doing this. Thank you for this content.
00:04 Data engineering is a high-demand skill in America. 01:25 Data engineers need expertise in relational SQL, Python, and workflow scheduling tools. 02:58 Python with third-party libraries and virtual environments is recommended for learning data engineering concepts. 04:23 Learning Python, SQL, and data storage for data engineering 05:50 SQL light is the most widely deployed database in the world 07:14 Project idea: Create an airflow DAC to extract and upload data to a data warehouse 08:39 Build a real-time application with Apache Kafka 10:05 Learning Data Engineering
I appreciate this video. At 3:00 he mentions that "the objective is to learn about data concepts, not computer science." I'm literally a total novice (but getting deeper into this), and I always thought that computer science was partly focused on data concepts. Or, maybe he means that data concepts is more specifically focused, in contrast to computer science which is extremely broad.
This is hands down the best high level introduction to data engineering I’ve come across. Thank you for making this amazing resource guide as well. I work as a BI analyst now and wanting to expand my skill set so I can truly become a data engineer. I appreciate your time and hard work!
@@jayzernI'm a college student , I am currently doing the az900 and then I shall do the data bricks certification, are these enough to guarantee an entry level job in data engineering?
Hello bro ,i completed my undergraduate in actuarial but failed miserable and i have been down,filled with regrets and lack motivation ever since but your has given something to believe and get me to regain my confidence . Today begin a new journey,my journey into teaching myself to become a data engineer hope to come back to this post after a uear to share my progress.
Outstanding video! This video was remarkably well-explained and detailed. It is also structured very well and was easy to follow along with your reasoning. It was a pleasure to watch. I am a Senior iOS Engineer looking to dive deeper into ETL and related topics. This helped me organize what I want to learn. Definitely subscribing.
I have been working with python so I think a few very important things 1) pyspark 2) pick AWS and/or Azure to learn their data engineering stack 3) very solid SQL skills.
Man, this was an awesome video. I'm currently a data analyst and my company has just offered me a junior data engineer role. I have a lot of learning to do and the path you have provided is helpful!
Love the content bro, hope you keep growing. Its cool to see another Malaysian in the US whose also in the field of Data while making RUclips videos on the side.
Thank you, thank you a lot, Jay!! I recently graduated as a Data Scientist but when I see the options of a career path I think Data Engineer fits better with me than others. This video is helpful!! I'm a new subs!
Is an excellent introduction I’m far away from being close to a data engineer. I’m just starting off in learning data analytics, but I enjoyed very much this introduction. I felt like it was well explained, and definitely sets the tone moving forward in my learning.
Thanks for this video! I always loved SQL and manipulating data but was put in the software Eng camp. I hated programming so gave it up and moved to IT/admin now I realize data engineering may have been a fun career choice and this video has given me a lot of interest on things to get back into!
I am a fresh graduate in Information Systems and my internship had me working with the team's data lead using Python, Apache Airflow, and PostgreSQL. I was glad and became more confident from the experience but once I was job hunting, I kept finding data engineering job postings demanding tools like Azure, AWS, Spark, Hadoop, etc. I was discouraged as I only had the three I mentioned. But I was a bit encouraged when you talked about tools as getting more specific on what the task demands, and that foundation through concepts can be better.
I started / enrolled in courses on DataCamp on path to be Data Analyst then will take courses for Data Engineering. I’m a super excited to learn about Data and the career opportunities
Thanks for the mindset part! I got a BS in DS, but shifted into DE because I enjoyed the backend of getting data for the analysis more than the analysis itself. Being new in the field, it’s definitely overwhelming with all the tools. However, Ill try to take your advice and start small by reading into a tool and doing instead of learning. I guess the best way to learn is to build and try? I always overcomplicate stuff and end up not following through because I try to tackle all these tools at once but will tackle it in smaller parts. From there, keep adding on to my project. Excited as this will be my first “DE” project. Thanks!
I do the same thing. It's a blessing and a curse. Scale back. Focus on one tool. Do a project. Join everything together like one complete Lego set. Blog about it. Reach out to recruiters. Wait for the 🎣 & snag a job
Starting Data Technologies course on September, for now I don't think about money, I just think how to survive and not become mad during education because of huge amount of information
I am just starting out in Data Engineering, your video is very helpful. Can you please make a video of a roadmap how to become a data engineer. Thank You
Hello broo, your video was great! I'm an aspiring Data Engineer and what you said about being overwhelm with many ambigous word on the first year was true, sometimes I feel burn out too. Kinda sad but I know this career is great so I keep it up. This video helps me regain my motivation :))
I'm currently transitioning from Data Analytics to Data Engineering. I think DE is only going to grow over the next decade as AI continues to blow up. Thanks for the amazing video jayzern!
About to dive into SQL this weekend. I'm just starting my journey into data analytics though. I don't think I have what it would take for an engineer 😅 but I still like watching videos like this cause I like looking at different things
Sis!!! 🗣️YOU HAVE WHAT IT TAKES. Commit to putting in the work and you can become an engineer. Don’t let self-doubt keep you from reaching your potential. You got this!!
@@LisaEveShow considering I have ZERO technical background, it's going ok lol. I'm starting to get the basics of SQL and about to start Python. I kind of wish I'd spent more time on the hard skills this whole time
This is awesome, thank you for sharing your knowledge. I'm a Software Engineer studying for my Master of Science in Data Analytics, maybe becoming a Data Engineer is in my future? Right now, focusing on my SWE job and learning SQL and Python through school have been my ways of learning and doing and while gaining experience (as well as coding projects on the side when I can).
Nice study resources. Thank you for sharing! Have you seen any data engineers got hired without working experience? Another RUclipsr said this was impossible. He said a data engineer has to have a few years experience in other data roles before taking this position.
Yea I've seen a ton of data engineers hired without working experience personally. IMO it's easier to get data engineer jobs without experience compared to data scientists, and roughly same level of difficulty with landing SWE jobs
Nice Video. May be you can suggest a couple of courses for each section of the video, bcos that is where people have a hard time choosing and taking action.
Thank you for a great introduction! I have been working in data science for quite some time, but over the years, I realized that the analysis part is much less interesting for me than the data part.
Started my journey in learning, I'm currently a systems engineer and it suck lmao, requirements this, requirements that.... anyways, I feel like I found a good solid book to read and some videos on you tube to follow along with for postgreSQL
Hi Jay, is it recommended to have a good overview of certain areas of mathematics before starting to learn the individual stages of data engineering?? Like linear algebra, discrete math, and the like?
Hi Jay! Thank you so much for this video. Can you please make a video about DE vs Cloud Engineer. It seems that CE make more money but require more skills like AWS/Google Cloud; but a lot of skills overlap as well.
Hey, First of all thank you for making this video, Is there any way you could provide the doc you are referring to in the video? That will be so helpful.
Hi Jay, thanks for taking out time to read this. This is my first experience learning Data Engineering. Can i start off with the first material you suggested: Introduction to machine learning with python? would that be taking me out of focus on Data Engineering? Please help me.
in the project youve mentioned in the core data skills section. are the resources that are involved free? as far as I know, airflow is open source. I have worked with databricks and the whole azure ecosystem, but it was my dads companies subscription.
Im 23 and switching careers from aviation maintenance mechanic to the tech industry trying to get into the data field as i’ve heard the phrase “data is king” i was debating between coding and cybersecurity but i feel like data is the best spot and data engineering sounds like my niche question is should i go to a regular community college (BCC for me in south Florida) or a university aswell or can i break into the field through certifications through technical/vocational school and what certifications are baseline i should go for im trying to find the most efficient and fastest way to break in considering my age (23) im fully committed to this aswell
Hi, I am looking into this. I started learning analytics but I want to expand my skillset now to also try for DE jobs. I have two doubts, would appreciate if you have time 1. Can someone make a career transition to data engineer from a non tech background(not having a STEM or CS degree)? Do companies accept? 2. I browsed through some jobs on LinkedIn and almost every job asked for some kind of experience in data or Software engineering. So, my doubt is if there are jobs for an entry level role like Junior Data Engineer or Associate Data Engineer?
10 - 15 years ago , data engineering jobs were much more straightforward - not so much today - there is new technology / platform/ framework released almost every year ! Its insane... Data bricks and microsoft synapse / fabric are the current popular ones. And to constantly learn new things can be a challenge, besides none of the frameworks or solutions have been good all this time !
I'm a freelancer and also self-taught programmer who wrote a lot of Data Scraping scripts using Python for my clients. I have a loyal client who requires me to extract data for them every month. I repeated this manual process for about 3 months until I realized the scraped data was getting bigger in size and dimension. I thought it was not a data scraping job anymore, it was a data engineering job. After doing some research online, I figured out some useful tools I can use to manage all data collected by the data scraper. I start by storing all the data in an SQL database and automating the process using Apache Airflow. That was the first time I thought Data Engineering was the right direction for my career path as a developer. Thanks for your advice, Jay.
how long did you train the skills of Data Scraping and Python to make money out of it?
wow, thanks for sharing this, it's really inspiring me to learn more about data engineering
@@ajdinpipo08 You can sharing resource you learn DE ?
What would you say the size difference was/is?
Where did you go to start freelancing jobs? I've got DE experience and wouldn't mind some side gigs.
Hands done, the best video I have seen on the subject. It's no nonsense, practical and easy to digest (of course after you put in the work). I am just starting out in my journey to become a Azure Data Engineer so I thank you Jay. Thanks for sharing and being so transparent. I'll keep you posted on my many adventures. For what it's worth, I'm 70 years young and still dedicated to life long learning.
That's amazing! Love the attitude on life long learning
Wow, that is so encouraging! Godspeed!
What a gem. I am currently pursuing AI/ML. Getting data scientist job as a fresher without experience is almost impossible so had to start with either as data analyst or data engineer. Since now everyone is becoming data analyst so their is demand and supply issue in the job market. Thinking of expanding my domain knowledge to data engineering as well but as you stated “got lost” with the available resources. This video helped me to make a roadmap to pursue data engineering.
Thanks for sharing. I agree it's super difficult to get a DS job fresh out of college. Branching out to DE is a great way to "stand out" from the crowd.
i'm a junior data scientist transitioning to data engineering/MLOps. i'm also based in nyc! i'm recently discovering my interest in filmmaking too! looks like i stumbled upon your content at just the right time. just subscribed!!!
No way! data engineering/MLOps and filmmaking is such a great niche
I love the way you split the career to basic and advanced skills. I’ve been keep doing that to become the data engineer. Thanks for sharing!
I recently signed up for the data engineering course from Datacamp and I have covered pretty much most SQL courses but I'd love to go back on a few of them again, I work as a data analyst but I feel a career in Engineering would be great.
We are in the same boat
@@holmesketrick7mee to
Exactly. SQL can be voluminous to learn. It takes time and practice to let it sink in
Hi, I’ve done the complete python track of data analyst, mostly of the sql courses of DataCamp. I’m from Chile, Latin America, do you have any advice for me to find remote jobs as data analyst in the US?
hi may I ask, does datacamp teach you how to normalize column tables in excel format to tables in sql / other database formats? thankyou
Beginning my path here, great video with no filler!
After 10 months, how are you now 😁?
I'm starting my learning journey from here. It's a completely new world for me, both because I'm not a native English speaker and because I'm totally new to all this technology. I've made up my mind: I want to become a Data Engineer, and I'm going to work really hard to achieve that!
Thank you so much for the guidance; I’m going to put it into practice now.
I am transitioning from web development to data. Fortunately for me, I learnt development skills from Learnenough. I have gone through their Commandline course and now on Python and Python for Data Science. I am excited to be doing this. Thank you for this content.
00:04 Data engineering is a high-demand skill in America.
01:25 Data engineers need expertise in relational SQL, Python, and workflow scheduling tools.
02:58 Python with third-party libraries and virtual environments is recommended for learning data engineering concepts.
04:23 Learning Python, SQL, and data storage for data engineering
05:50 SQL light is the most widely deployed database in the world
07:14 Project idea: Create an airflow DAC to extract and upload data to a data warehouse
08:39 Build a real-time application with Apache Kafka
10:05 Learning Data Engineering
Great job with your career man, mixing technical and artistic paths, great balance.
Thanks for the kind words man!
I appreciate this video. At 3:00 he mentions that "the objective is to learn about data concepts, not computer science." I'm literally a total novice (but getting deeper into this), and I always thought that computer science was partly focused on data concepts. Or, maybe he means that data concepts is more specifically focused, in contrast to computer science which is extremely broad.
Do I have opportunity to learn together with you mate? I also a total novice but getting deeper 😂
This is hands down the best high level introduction to data engineering I’ve come across. Thank you for making this amazing resource guide as well. I work as a BI analyst now and wanting to expand my skill set so I can truly become a data engineer. I appreciate your time and hard work!
Wow that means a lot! Keep working on expanding your skillsets. Learning is a non-stop journey
@@jayzernI'm a college student , I am currently doing the az900 and then I shall do the data bricks certification, are these enough to guarantee an entry level job in data engineering?
Hello bro ,i completed my undergraduate in actuarial but failed miserable and i have been down,filled with regrets and lack motivation ever since but your has given something to believe and get me to regain my confidence . Today begin a new journey,my journey into teaching myself to become a data engineer hope to come back to this post after a uear to share my progress.
you’ll do well, being a DE is thee best. and you’ll easily work worldwide
I am going back to your comment next year to check you. keep up 🫡
How has your journey been so far?
@@Odin.x.He came bck to reality
How is your journey going did you land a Data Engineering job?
Best roadmap for Data Engineering on RUclips. New sub Here 😊. Greetings from Argentina 🇦🇷
Seriously the best video I have seen! Absolutely love how down to earth and practical these tips are😍Thanks a lot for a great content
Outstanding video! This video was remarkably well-explained and detailed. It is also structured very well and was easy to follow along with your reasoning. It was a pleasure to watch. I am a Senior iOS Engineer looking to dive deeper into ETL and related topics. This helped me organize what I want to learn. Definitely subscribing.
Thank you for such an informative video, I'm a junior data engineer and your channel is an invaluable resource!
Hi, can you share data engineering learning resources? Thank you
Loved your video, it surprised me how you started the second section getting us into the book. It felt like already starting :) thank you!
really great and clear explanation ! please keep this way. I would recommend this video to anybody wanting for learn about data engineering.
I wish I would have seen this sooner! Great and crisp content.
I have been working with python so I think a few very important things 1) pyspark 2) pick AWS and/or Azure to learn their data engineering stack 3) very solid SQL skills.
Why not gcp?
Man, this was an awesome video. I'm currently a data analyst and my company has just offered me a junior data engineer role. I have a lot of learning to do and the path you have provided is helpful!
Amazing man, good luck in your new role!
This is super super SUPER practical and insightful and helpful. Looking forward to your future content and successes!
Thanks for the support Derek
Very useful tips for those who are just starting out. Great content!
Love the content bro, hope you keep growing. Its cool to see another Malaysian in the US whose also in the field of Data while making RUclips videos on the side.
Thanks bro! Awesome to see other Malaysians too
Holy f, brother this is an amazing and quick overview.
You def got potential in this, keep it up!
Thanks man! So much more to come
Thank you, thank you a lot, Jay!! I recently graduated as a Data Scientist but when I see the options of a career path I think Data Engineer fits better with me than others. This video is helpful!! I'm a new subs!
Is an excellent introduction I’m far away from being close to a data engineer. I’m just starting off in learning data analytics, but I enjoyed very much this introduction. I felt like it was well explained, and definitely sets the tone moving forward in my learning.
Thanks for this video! I always loved SQL and manipulating data but was put in the software Eng camp. I hated programming so gave it up and moved to IT/admin now I realize data engineering may have been a fun career choice and this video has given me a lot of interest on things to get back into!
Thank you. Very kind of you to put out this valuable easy to digest and to follow content.
I am a fresh graduate in Information Systems and my internship had me working with the team's data lead using Python, Apache Airflow, and PostgreSQL. I was glad and became more confident from the experience but once I was job hunting, I kept finding data engineering job postings demanding tools like Azure, AWS, Spark, Hadoop, etc. I was discouraged as I only had the three I mentioned. But I was a bit encouraged when you talked about tools as getting more specific on what the task demands, and that foundation through concepts can be better.
thanks for delivering a well produced video centered around DE
Nice briefing,waiting for more knowledge sharing on data engineering
Currently working full time while I get my degree I would love to get a job in Database engineering when I graduate helpful info ty!
Very good explanation. Love it for the conclusion with the complete motivation.
Thanks Jay for posting such an informative and insightful video!!!
Amazing video with to the point information. Thank you Jay!
I started / enrolled in courses on DataCamp on path to be Data Analyst then will take courses for Data Engineering. I’m a super excited to learn about Data and the career opportunities
I've been a data analyst for a few years and I am just starting to learn the material to transition into the DE space.
nice and well structured information, liking this!
Elite video, very informative I’d truly enjoy another video like this put more in depth in whatever area you think is valuable or interesting
Thanks for the support man!
This is a great video. Thanks for providing real value.
Thanks for the mindset part! I got a BS in DS, but shifted into DE because I enjoyed the backend of getting data for the analysis more than the analysis itself. Being new in the field, it’s definitely overwhelming with all the tools. However, Ill try to take your advice and start small by reading into a tool and doing instead of learning. I guess the best way to learn is to build and try? I always overcomplicate stuff and end up not following through because I try to tackle all these tools at once but will tackle it in smaller parts. From there, keep adding on to my project. Excited as this will be my first “DE” project. Thanks!
Amazing! I started out with a Bachelors in DS too
I do the same thing. It's a blessing and a curse. Scale back. Focus on one tool. Do a project. Join everything together like one complete Lego set. Blog about it. Reach out to recruiters. Wait for the 🎣 & snag a job
Great content!
Can you make more practical project tutorials? More specifically, AWS DE projects
Thanks for the useful tips. Short to the points yet very effective !
Starting Data Technologies course on September, for now I don't think about money, I just think how to survive and not become mad during education because of huge amount of information
This was really helpful. Thanks for sharing ❤️
Powerful tips! Thanks a lot for lightning the path!
I am just starting out in Data Engineering, your video is very helpful. Can you please make a video of a roadmap how to become a data engineer. Thank You
Hello broo, your video was great!
I'm an aspiring Data Engineer and what you said about being overwhelm with many ambigous word on the first year was true, sometimes I feel burn out too. Kinda sad but I know this career is great so I keep it up. This video helps me regain my motivation :))
very sincere and very true points, thank you very much
I'm currently transitioning from Data Analytics to Data Engineering.
I think DE is only going to grow over the next decade as AI continues to blow up. Thanks for the amazing video jayzern!
Why u transitioning from data analysis
To data engineering?
Is there any problem with data analysis?
About to dive into SQL this weekend. I'm just starting my journey into data analytics though. I don't think I have what it would take for an engineer 😅 but I still like watching videos like this cause I like looking at different things
Amazing! Good luck on your data analytics journey.
@@jayzern thank you! It's a little intimidating, but so far I'm hopeful ❤️. Kind of wish I'd known about this when I was younger though lol
Sis!!! 🗣️YOU HAVE WHAT IT TAKES. Commit to putting in the work and you can become an engineer. Don’t let self-doubt keep you from reaching your potential. You got this!!
How are you doing on this journey?
@@LisaEveShow considering I have ZERO technical background, it's going ok lol. I'm starting to get the basics of SQL and about to start Python. I kind of wish I'd spent more time on the hard skills this whole time
Hey Jay thanks for sharing all those valuable information with us. I hope to you see soon in NYC! :D thanks again brother!
Awesome video Jay! I will personally be looking through the resources that you've recommended.
Thanks oliver, u the best
Great work on explaining the concepts 👍🏽
This is awesome, thank you for sharing your knowledge. I'm a Software Engineer studying for my Master of Science in Data Analytics, maybe becoming a Data Engineer is in my future? Right now, focusing on my SWE job and learning SQL and Python through school have been my ways of learning and doing and while gaining experience (as well as coding projects on the side when I can).
Thanks for sharing Aaron. A lot of people are in the same boat and can totally relate. Enjoy the ride and good luck in your Masters degree.
@@jayzern thanks I subbed, will be tuning in for more videos!
Best Roadmap ever. Just get started with it.
Thanks a lot!
This is a super high quality video! Cant wait to see this channel grow!
Thanks man!
Thank you so much for this guidance. Really, it's a great help.
Nice study resources. Thank you for sharing!
Have you seen any data engineers got hired without working experience? Another RUclipsr said this was impossible. He said a data engineer has to have a few years experience in other data roles before taking this position.
Yea I've seen a ton of data engineers hired without working experience personally. IMO it's easier to get data engineer jobs without experience compared to data scientists, and roughly same level of difficulty with landing SWE jobs
Great information. Looking for more videos from you. Keep doing .
Nice Video. May be you can suggest a couple of courses for each section of the video, bcos that is where people have a hard time choosing and taking action.
Exactly
Ooo thanks for the suggestion, will include that in future videos
Thank you for this video, it created more of a path for me.
Designing data intensive applications is another great book by O'Reilly
One of my fav books!
Great guide bro...
However the video is a year ago, but I would really like to thank you much for sharing such infos, you put me back to the track
Hugely useful! Subbed!
Thank you for this great video
Thank you for a great introduction! I have been working in data science for quite some time, but over the years, I realized that the analysis part is much less interesting for me than the data part.
It was amazing introduction
and guide line.
Thanks
you earned a subscription because of simplicity and being efficient on teaching!
Thank you for your job!
Thanks a lot for this tutorial
Amazing and informative!!
thanks for notion and great vid
Excellent information man
Very insightful video 👏👏👏
Eye opening, thanks
Great information Thanks Jayzern . Could you make a video on HOW I WOULD LEARN SNOWFLAKE DEVELOPER [if i could start over ]
Well developed and presented informative video. Thanks a lot 🙏👍
Great content.. the screen you have online with the contents where are u reading this from? The book or from a website?
Started my journey in learning, I'm currently a systems engineer and it suck lmao, requirements this, requirements that.... anyways, I feel like I found a good solid book to read and some videos on you tube to follow along with for postgreSQL
Thank you for sharing insights
Great content, keep up the good work!
Thank you!
Hi Jay, is it recommended to have a good overview of certain areas of mathematics before starting to learn the individual stages of data engineering?? Like linear algebra, discrete math, and the like?
Useful information thank you
Hi Jay! Thank you so much for this video. Can you please make a video about DE vs Cloud Engineer. It seems that CE make more money but require more skills like AWS/Google Cloud; but a lot of skills overlap as well.
This video helps me learn how to learn ML and process data.
Hey, First of all thank you for making this video, Is there any way you could provide the doc you are referring to in the video? That will be so helpful.
Hi Jay, thanks for taking out time to read this. This is my first experience learning Data Engineering. Can i start off with the first material you suggested: Introduction to machine learning with python? would that be taking me out of focus on Data Engineering? Please help me.
It deviates from DE a little but it's also my fav book for learning ML! I think it's fine for learning Python the unconventional way
Thank you jayzern@@jayzern
in the project youve mentioned in the core data skills section. are the resources that are involved free? as far as I know, airflow is open source. I have worked with databricks and the whole azure ecosystem, but it was my dads companies subscription.
thank you for the video
Im 23 and switching careers from aviation maintenance mechanic to the tech industry trying to get into the data field as i’ve heard the phrase “data is king” i was debating between coding and cybersecurity but i feel like data is the best spot and data engineering sounds like my niche question is should i go to a regular community college (BCC for me in south Florida) or a university aswell or can i break into the field through certifications through technical/vocational school and what certifications are baseline i should go for im trying to find the most efficient and fastest way to break in considering my age (23) im fully committed to this aswell
Thank you Jayzern
Hi, I am looking into this.
I started learning analytics but I want to expand my skillset now to also try for DE jobs.
I have two doubts, would appreciate if you have time
1. Can someone make a career transition to data engineer from a non tech background(not having a STEM or CS degree)? Do companies accept?
2. I browsed through some jobs on LinkedIn and almost every job asked for some kind of experience in data or Software engineering. So, my doubt is if there are jobs for an entry level role like Junior Data Engineer or Associate Data Engineer?
10 - 15 years ago , data engineering jobs were much more straightforward - not so much today - there is new technology / platform/ framework released almost every year ! Its insane... Data bricks and microsoft synapse / fabric are the current popular ones. And to constantly learn new things can be a challenge, besides none of the frameworks or solutions have been good all this time !
"Death by a thousand microservices" comes to mind lol
Hi, how do you suggest building and presenting a portfolio?