As an absolute clueless beginner, have to say you are doing an incredible job introducing this subject. No fancy jargon, just pure simple English for anyone to understand.
@@SeattleDataGuy I really feel you are taking the channel in the right direction. You have cleared most of the general broader explanatory and introductory stuff and you are now creating in depth tutorials, how to actually use stuff; just keep on with that. Also maybe include some real life case studies and incidents similar to the style of that video where Hussein Nasser delved into that Uber shifting from Postgress to MySQL. And DE, being the tool/skill intensive and ever rapidly changing stream it is, you will never be short of content tbf atleast for the next 5 yrs.
@@SeattleDataGuy Thanks Ben. I'm already on my 100 day challenge. Will mention you once I complete it. You are an inspiration!! ✨. Thank you. Can you please make an updated video on how to search for Data engineering jobs and what roles are good for beginners/ entry level
I'm not sure where else to ask this so here's my shot in the dark; I've been a Data Engineer for 2.5 years now and things are getting bad at work (typical management change issues, favoritism at work causing bad decisions etc) and trying to find another job even just at my own level is depressing. I already got my Azure Data Fundamentals certification and am working towards my DP-203 (Azure Data Engineering) but it doesn't seem to be doing anything for my call-back rate on applications. I feel like the big issue besides years of experience is that my current employer is very narrow-minded with their data so besides Data Factory and T-SQL we don't use much, so I can't list things like Synapse or Databricks on my resume in good faith. All this to ask; How do you get claimable experience with some of these things if your current employer doesn't use them?
If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k
@@SeattleDataGuy I see many companies moving towards Dbt for ELT style of DE. From what I’ve seen online I know they follow software engineering principles to perform the transformations. What I couldn’t find was a good video on how they perform the end to end project . I’m reading this book called Data Engineering with Dbt by Roberto zagni to understand more on DBT. If you could make a video on that it would be great . I understand your explanations much better . Thank you again :)
@@karszn Sounds like an interesting video! I often see dbt at mid-sized companies, i feel like larger companies tend to use more enterprise level tools
I just started a job as trainee in data engineering, I just know Python (Numpy, Pandas, Matplotlib) and Advanced SQL and some concepts like ETL/ELT, EDA etc. I dont know anything about cloud, is it a good time to get a cloud certification as AZ-900 or it is more recommended to learn Spark before? (my company provide a voucher of 100% in Azure certifications)
Do u think that airflow just do orchestration work not relating to compute, and use other external compute tools to do that work, so it's not need to learn more about executor of airflow and implement something like kubernetes cluster to run airflow and scale airflow... because it just do orchestration work.
@1:31 He is a famous actor AKSHAY KUMAR (PHIR HERA PHERI) Movie scene It's comedy movie In india n numbers of memes are there for this movie.... People use this meme to tease someone who is in love or who did anything sexual...or any intimacy topics....which one you have used....
You started nice, but then your moved to name dropping if all the latest buzz words in the industry. Snowflake, DBT, azure, etc. are not tools, those are just different type of technologies you might be using as part of your work
As an absolute clueless beginner, have to say you are doing an incredible job introducing this subject. No fancy jargon, just pure simple English for anyone to understand.
Thank you! I am so glad to hear that, any thoughts on what the next few videos should be?
@@SeattleDataGuy I really feel you are taking the channel in the right direction. You have cleared most of the general broader explanatory and introductory stuff and you are now creating in depth tutorials, how to actually use stuff; just keep on with that. Also maybe include some real life case studies and incidents similar to the style of that video where Hussein Nasser delved into that Uber shifting from Postgress to MySQL. And DE, being the tool/skill intensive and ever rapidly changing stream it is, you will never be short of content tbf atleast for the next 5 yrs.
This video perfectly explains where to start with and where most of the focus should be.
I am glad you found it helpful!
@@SeattleDataGuy Thanks Ben. I'm already on my 100 day challenge. Will mention you once I complete it. You are an inspiration!! ✨. Thank you. Can you please make an updated video on how to search for Data engineering jobs and what roles are good for beginners/ entry level
Hey, fresh subscriber. I think it's gonna be 1 of my favorites. However I have to go refresh my SQL. Gotta find a free usable terminal now
I'm not sure where else to ask this so here's my shot in the dark;
I've been a Data Engineer for 2.5 years now and things are getting bad at work (typical management change issues, favoritism at work causing bad decisions etc) and trying to find another job even just at my own level is depressing.
I already got my Azure Data Fundamentals certification and am working towards my DP-203 (Azure Data Engineering) but it doesn't seem to be doing anything for my call-back rate on applications. I feel like the big issue besides years of experience is that my current employer is very narrow-minded with their data so besides Data Factory and T-SQL we don't use much, so I can't list things like Synapse or Databricks on my resume in good faith.
All this to ask; How do you get claimable experience with some of these things if your current employer doesn't use them?
Maybe working on personal projects or looking around and seeing if you can contribute to open source stuff
You might change your job.
If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k
Can you make a video on how dbt is used to make a software engineering type of transformations? Thank you and everything regarding it
Thanks for the comment. I am curious, what do you mean by software engineering type?
@@SeattleDataGuy I see many companies moving towards Dbt for ELT style of DE. From what I’ve seen online I know they follow software engineering principles to perform the transformations. What I couldn’t find was a good video on how they perform the end to end project . I’m reading this book called Data Engineering with Dbt by Roberto zagni to understand more on DBT. If you could make a video on that it would be great . I understand your explanations much better . Thank you again :)
@@karszn Sounds like an interesting video! I often see dbt at mid-sized companies, i feel like larger companies tend to use more enterprise level tools
@@SeattleDataGuy yes . Same and thank you again for your videos and content . Appreciate it
I just started a job as trainee in data engineering, I just know Python (Numpy, Pandas, Matplotlib) and Advanced SQL and some concepts like ETL/ELT, EDA etc. I dont know anything about cloud, is it a good time to get a cloud certification as AZ-900 or it is more recommended to learn Spark before? (my company provide a voucher of 100% in Azure certifications)
Learning Spark on top of a cloud is the way!
Can you share your LinkedIn please?
Where did you learn Advanced SQL? Any tips on it?
what do you guys think about pentaho data integration ?
Do u think that airflow just do orchestration work not relating to compute, and use other external compute tools to do that work, so it's not need to learn more about executor of airflow and implement something like kubernetes cluster to run airflow and scale airflow... because it just do orchestration work.
can you create a dedicated vedios on datawarehouse concepts,it will be helpfull for me!
I have put together a few around data modeling, is that what you mean or do you mean like data warehouse technology?
What the Kimble means here when you mentioned how you find the difference between data warehouse and databases? Thanks a lot.
awesome video, thank you!
Glad you liked it!
The sound at @15:57 is hilarious. But great video thank you.
hahaha thanks i guess. Glad you enjoyed the video!
On the Open source Big Data Platform Which best tools anyone can consider for data quality, data processing (Spark ), Data storage etc.?
13:38 SSIS instead SSAS ??
What about microsoft fabric? Ur expectation for it!
1:29 ....didn't expect this meme on this channel...lol.
I love this content. Thank you so much
Glad you enjoy it! Any other content you'd like to see!
what about r?
r isn't really great for data engineering. I am sure some people use it, but it's not ideal and is more suited towards DS projects.
This is super helpful!!!
Glad you think so! Let me know if there is any other content you'd like to see!
Can you make video about analyst engineer
so how much sql do I need? I know data science folks need expert level sql
I've liked Prefect over Airflow
I’d love to know why!
Are you planning on covering Palantir anytime soon?
🎉🎉🎉🎉🎉
Wooo hoo!
He is using indian meme in his video...,🤣
wait which one!
@1:31
He is a famous actor AKSHAY KUMAR
(PHIR HERA PHERI) Movie scene
It's comedy movie
In india n numbers of memes are there for this movie....
People use this meme to tease someone who is in love or who did anything sexual...or any intimacy topics....which one you have used....
Ok at this point would it be easier to list skills Data Engineers shouldn't be learning now 👀👀
It’s getting close
Is this really a good video? i feel like he just namedrops every techs
Not clearly explained, just bs .
You started nice, but then your moved to name dropping if all the latest buzz words in the industry.
Snowflake, DBT, azure, etc. are not tools, those are just different type of technologies you might be using as part of your work