Just adding some facts about politics in DS: especially in traditional industries, usually DS won't help with decision making at all, because the reality is: "the boss has already had a decision, and then he asks DS to conduct analysis to glorify his decision as a data-driven highly intelligent and scientific decision." Then the problem would be: when the DS's analysis results contradict the boss's decision, then the boss would get mad, disputing the DS as incapable, saying that the DS's analysis is all garbage, asking the DS to redo the analysis again and again... until the DS adds tricks or even falsifications to make the results conform to the boss's thought. The above scenario happens in academia as well. Many advisors care more about if the results conform to their experience, rather than scientifically right or wrong.
I am very sorry to hear about this experience. I have a high amount of cynicism and disillusionment with academia in general; if you've experienced that there, that is simultaneously regrettable and something that doesn't surprise me one bit. I'd say that one probably has to do with sponsorship/funding bias more than anything. It's true that sometimes decisions, hypotheses, and what people want to do precede the data and the analysis a lot. That makes the DS's work a bit less about leading to new decisions, but rather, confirming that one people are already predisposed to it is a correct one. Then again, I do have a couple friends who've had similar experiences to you with their bosses. They all left within months. I'd do the same, because that is less of a DS problem and more of an overall toxic workplace environment problem.
very interesting perspective. It almost seems the DS's biggest value is to validate the boss's decision and not to add value through an unbiased analysis of the data.
my partner who is a DS quit his job for a myriad of reasons, but mostly the company never knowing what to use him for, while also not having a data dept at all and expecting him to magically pull all of the answers out of his ass the same day, and would nag him hourly, despite his accurate forecasts and quotes for time, the CEO would directly harass him even after he'd tell him he would be done by Friday and it was only Wednesday, and then because it wasn't done 'early', he'd tell him to cancel the project and switch to a new one based on an article he just read. this happened all the time, and when he'd start/propose projects they'd make him stop and do IT and menial tasks like help new call center employees set up their email
Yeah any job where you constantly have manager types requesting that you justify your worth while at the same time trying to assume credit for the data driven decisions that you influenced isn't gonna last long. Too many MBA types are threatened by the quantitative direction of decision making in a lot of companies and it's just a toxic environment.
Richard, I would love to see a video on the top things in your opinion that a company needs to have in place culturally and infrastructure-wise to support data scientists and a data department. Keep up the good work.
Thanks Joel! I'll try and do that within this month. The big data conference here in Detroit last year touched on this topic almost exclusively.... I sense it is a problem that more organizations are aware they are facing. So some cause for optimism. In the meantime.... my latest video is on a topic that I know is very near and dear to you...
Instructors who provide clean data sets beyond the first example or assignment or project are doing a disservice to their students. I enjoy cleaning data, acquiring primary and secondary data, and getting to really understand the data.
Interesting insights, thanks for sharing! Nearly all of the students I went to grad school with ended up going into data science, interesting to get a view on their perspective.
Didnt notice you have so few subscribers. Subscribed! Great content! Constructive criticism : You could improve your narration voice - it's kind of monotone ✌️🙂
Hah, thank you and I appreciate the constructive feedback! My channel has been growing at a rate of 100 subscribers every 6-7 days. We will see what it looks like by end of year, but it's a slow churn. Thanks and welcome onboard!
I am not a Data Scientist but a BI developer. After a couple of years in the field I can say your reflections are 💯 true in my view, specially politics. Through I love BI I must transform myself but I have decided to not dig into Data Science. The reality has scared me away from transitioning into the Data Science field or machine learning field. Also an aspect is BUDGET of the project. Check what investors say about the field. How "stressed" is the company? Many companies want to do great things but they dont give you right resources/tools or time. Always be bold ask for a big project budget. If they dont want to have a Good budget but Still want "datadriven magic", then you know you have to run. Always try to go to a team than be by yourself. You need to be in a team otherwise you will get tasks that are impossible and shitty eg DBA tasks. And shy away from companies that mention too many buzzwords. Waste of time.
Because there is so much effort and passion required to execute the work. It's a legit craft. All for some manager to respond " Oh thats nice to know" High level artists and athletes definitely generate a WOW!!! response. The same is not true for people who work with data.
I did quit. Infrastructure and data driven culture played a HUGE role in me quitting. It was very frustrating to have managers ask us to build pricing models using reinforcement learning algorithms when all I know is how build an xgboost.
I enjoyed the video, and gave it a like, but felt there was a bit of conflation here between people quitting jobs (ie, looking for a better data science job) and people leaving data science for a different career. I expected it to focus on people leaving the field.
I look at machine learning as a subset of data science. It's a valuable piece of the future for sure, as long as it is aligned with business objectives.
The most common, simplest one-word answer I could probably give here would be "politics" -- but I've heard it said that the most complicated problems in the world are the ones that involve humans, and I can't say I disagree!
I have completed MSc in Math and want to choose DS career not because it's a Lucrative career but because I love all the tasks involved in DS itself. In your opinion will it be a good career path? Just give your Opinion no matter what you think.
Yeah, lack of retention - excluding of course the situations when it's due to a position being eliminated - is something that should cause demand to increase.
The misleading part is so to the point. My professor is deep learning evageliat, he can talk about it for hours. But can implement any of the theory. Not motovating
Yes, it can be, though it depends a great deal on the company in question and often the people you're working with - both in your organization and clients. It can lend itself to a lot of work particularly of a creative nature, though in my opinion it's more fun than not!
I totally understand the anxiety about this topic, but I truly do not think we're gonna see a displacement of data science jobs in favor of AutoML at least any time in the near future. See my video on "Will Data Science Die in 5 Years" -- I talk in length about this, but my overall opinion here is that human ingenuity will win out, and we're going to find ways to use these tools to solve the next, most complex problem. So data science will be around.... it just might not be targeting the same problems that it is today.
Hey Richard, thx for the video. I really enjoyed it. I am concentrating on web scraping right now,since I believe for a beginner it is the fastest way to start making money/ get some real on the job experiences with data science. Does it make sense to you? Would you say that there is a better way than web scraping to start with as a freelance data scientist? P.S.: I am learning SQL and Python for nearly a year, so my question is not a about skills data scientists need the most but about where to start getting some real on the job experiences.
You certainly need some more qualities other than just creativity; however, I think there can be as much as 5x-10x difference in delivery between a creative and non-creative data scientist, all else held equal!
Hello richard,i am working as a angular+springboot dev since 18 months and from past 6 months i have been learning ml & dl.i was thinking about transition into DS job roles but this video made me think again.i dont know what to do now.it would be very much helpful if you suggest something.
I am above 50 years old - I have Master's degrees in Business Administration and Statistics . I have also done programming at the initial stage of my career and had been working in the ICT area. Recently I have acquired knowledge of ML with Python. I wonder why I didnt think about being a data scientist earlier ! Anyway I want to be one one now - almost at the fang end of my career.. I have worked in the Middle East and India and in the UK for only 1 year after MBA. I am a very fast learner . Can anyone help me in getting data scientist or analyst or business analyst job anywhere in the world? I know my age maybe a negative point but then I have vast experience of working in different business area.
I m ok with bit of politics I am 18 year and I love mathematics and and want to enter into this field .I am enrolled in Btech CSc (from average college)on campus program and online Bsc degree in data science (from IITM ranked best engineering in India) Should You encourage me to continue
Absolutely; what I'll say is, as you gain experience your preferences and the things you can and cannot tolerate are likely to shift drastically for you (i.e. you may find politics ok now but despise it years down the road); having said this coming from a strong mathematical background will set you up extremely well to be a successful data scientist!
Just adding some facts about politics in DS: especially in traditional industries, usually DS won't help with decision making at all, because the reality is: "the boss has already had a decision, and then he asks DS to conduct analysis to glorify his decision as a data-driven highly intelligent and scientific decision." Then the problem would be: when the DS's analysis results contradict the boss's decision, then the boss would get mad, disputing the DS as incapable, saying that the DS's analysis is all garbage, asking the DS to redo the analysis again and again... until the DS adds tricks or even falsifications to make the results conform to the boss's thought.
The above scenario happens in academia as well. Many advisors care more about if the results conform to their experience, rather than scientifically right or wrong.
I can confirm this has happened to me too.
I am very sorry to hear about this experience. I have a high amount of cynicism and disillusionment with academia in general; if you've experienced that there, that is simultaneously regrettable and something that doesn't surprise me one bit. I'd say that one probably has to do with sponsorship/funding bias more than anything.
It's true that sometimes decisions, hypotheses, and what people want to do precede the data and the analysis a lot. That makes the DS's work a bit less about leading to new decisions, but rather, confirming that one people are already predisposed to it is a correct one. Then again, I do have a couple friends who've had similar experiences to you with their bosses. They all left within months. I'd do the same, because that is less of a DS problem and more of an overall toxic workplace environment problem.
very interesting perspective. It almost seems the DS's biggest value is to validate the boss's decision and not to add value through an unbiased analysis of the data.
same
An amazing answer!! 100% totally true!!
my partner who is a DS quit his job for a myriad of reasons, but mostly the company never knowing what to use him for, while also not having a data dept at all and expecting him to magically pull all of the answers out of his ass the same day, and would nag him hourly, despite his accurate forecasts and quotes for time, the CEO would directly harass him even after he'd tell him he would be done by Friday and it was only Wednesday, and then because it wasn't done 'early', he'd tell him to cancel the project and switch to a new one based on an article he just read. this happened all the time, and when he'd start/propose projects they'd make him stop and do IT and menial tasks like help new call center employees set up their email
Hi Richard, I just want to leave a comment saying that this video is greatly organized and precise. It helped a lot with a great overview. Liked!
Yeah any job where you constantly have manager types requesting that you justify your worth while at the same time trying to assume credit for the data driven decisions that you influenced isn't gonna last long. Too many MBA types are threatened by the quantitative direction of decision making in a lot of companies and it's just a toxic environment.
Richard, I would love to see a video on the top things in your opinion that a company needs to have in place culturally and infrastructure-wise to support data scientists and a data department. Keep up the good work.
Thanks Joel! I'll try and do that within this month. The big data conference here in Detroit last year touched on this topic almost exclusively.... I sense it is a problem that more organizations are aware they are facing. So some cause for optimism.
In the meantime.... my latest video is on a topic that I know is very near and dear to you...
Thank you sir, I find your videos helpful. Appreciate your efforts.
My pleasure! Much more to come!
lack of data, lack of direction, lack of bosses who know what data science actually is
Very true!
Instructors who provide clean data sets beyond the first example or assignment or project are doing a disservice to their students. I enjoy cleaning data, acquiring primary and secondary data, and getting to really understand the data.
I couldn't agree more; it's sad that so many really do confine themselves to completely clean data.
@@RichardOnData im surprised some companies don't have at least one person solely dedicated to data prep.
Interesting insights, thanks for sharing! Nearly all of the students I went to grad school with ended up going into data science, interesting to get a view on their perspective.
Didnt notice you have so few subscribers. Subscribed! Great content! Constructive criticism : You could improve your narration voice - it's kind of monotone ✌️🙂
Hah, thank you and I appreciate the constructive feedback!
My channel has been growing at a rate of 100 subscribers every 6-7 days. We will see what it looks like by end of year, but it's a slow churn. Thanks and welcome onboard!
also PFFT I can't remember the last company I worked for that actually had a real proper HR dept - it's usually some assistant
I am not a Data Scientist but a BI developer. After a couple of years in the field I can say your reflections are 💯 true in my view, specially politics. Through I love BI I must transform myself but I have decided to not dig into Data Science. The reality has scared me away from transitioning into the Data Science field or machine learning field. Also an aspect is BUDGET of the project. Check what investors say about the field. How "stressed" is the company? Many companies want to do great things but they dont give you right resources/tools or time. Always be bold ask for a big project budget. If they dont want to have a Good budget but Still want "datadriven magic", then you know you have to run. Always try to go to a team than be by yourself. You need to be in a team otherwise you will get tasks that are impossible and shitty eg DBA tasks. And shy away from companies that mention too many buzzwords. Waste of time.
how difficult is it to get a job as a BI developer? Can you please tell
Ahhh clean dataset.. I would give everything in my posession to always have a clean dataset.
Because there is so much effort and passion required to execute the work. It's a legit craft. All for some manager to respond " Oh thats nice to know" High level artists and athletes definitely generate a WOW!!! response. The same is not true for people who work with data.
After a whole year of experience as a DS in the industry, I am seriously thinking of becoming a software engineer.
Interesting. What's been your experience and caused you to move in that direction?
Why????
Interesting.
I did quit. Infrastructure and data driven culture played a HUGE role in me quitting. It was very frustrating to have managers ask us to build pricing models using reinforcement learning algorithms when all I know is how build an xgboost.
I enjoyed the video, and gave it a like, but felt there was a bit of conflation here between people quitting jobs (ie, looking for a better data science job) and people leaving data science for a different career. I expected it to focus on people leaving the field.
I think not 'Data Science' but 'Machine Learning' is the future
I look at machine learning as a subset of data science. It's a valuable piece of the future for sure, as long as it is aligned with business objectives.
I certainly liked this video. What do companies want to hire people with expertise, and then do not listen to them?
The most common, simplest one-word answer I could probably give here would be "politics" -- but I've heard it said that the most complicated problems in the world are the ones that involve humans, and I can't say I disagree!
This is painfully accurate
I have completed MSc in Math and want to choose DS career not because it's a Lucrative career but because I love all the tasks involved in DS itself.
In your opinion will it be a good career path?
Just give your Opinion no matter what you think.
I think the running thing is really good advice.
Good content, greetings from Mexico :)
Greetings, thanks for tuning in!
Great into bro
Thanks!
Does that mean there is more of a demand for DS? As in there will be more vacant positions because people are quitting?
Thanks
Yeah, lack of retention - excluding of course the situations when it's due to a position being eliminated - is something that should cause demand to increase.
@@RichardOnData Thanks for your reply. Hmmm I hope many are not being eliminated lol, it wouldn’t be a good sign!
Your hand movements make me uncomfortable
Maybe he's Italian
Hey Richard, thanks for all ur videos, I learnt a lot from your videos. Have you got any experience with Forecasting?
Some, yes, albeit not as much as with non-longitudinal data. Regression with ARIMA errors is an incredibly powerful technique.
Clerk job vs engineering job. Any day engineering job is interesting. Clerk job become boring very soon.
The misleading part is so to the point. My professor is deep learning evageliat, he can talk about it for hours. But can implement any of the theory. Not motovating
Yeah, because it's boring.
I saw a year ago video, and the voice was totally different.... I liked it better from the past 'n.n
Is DS is a stressful job?
Yes, it can be, though it depends a great deal on the company in question and often the people you're working with - both in your organization and clients. It can lend itself to a lot of work particularly of a creative nature, though in my opinion it's more fun than not!
6- expectations about auto machine learning
I totally understand the anxiety about this topic, but I truly do not think we're gonna see a displacement of data science jobs in favor of AutoML at least any time in the near future. See my video on "Will Data Science Die in 5 Years" -- I talk in length about this, but my overall opinion here is that human ingenuity will win out, and we're going to find ways to use these tools to solve the next, most complex problem. So data science will be around.... it just might not be targeting the same problems that it is today.
@richardondata Please do a video on the difference between Data Analysis and Data Science, and what it takes to be a Data Analyst
Hey Richard, thx for the video. I really enjoyed it. I am concentrating on web scraping right now,since I believe for a beginner it is the fastest way to start making money/ get some real on the job experiences with data science. Does it make sense to you? Would you say that there is a better way than web scraping to start with as a freelance data scientist? P.S.: I am learning SQL and Python for nearly a year, so my question is not a about skills data scientists need the most but about where to start getting some real on the job experiences.
Do you think data science is a good job for creative people?
You certainly need some more qualities other than just creativity; however, I think there can be as much as 5x-10x difference in delivery between a creative and non-creative data scientist, all else held equal!
@@RichardOnData
5-10x difference in delivery meaning??
Hello richard,i am working as a angular+springboot dev since 18 months and from past 6 months i have been learning ml & dl.i was thinking about transition into DS job roles but this video made me think again.i dont know what to do now.it would be very much helpful if you suggest something.
The resemblance to Ian Hecox is uncanny here....
Hah! I've never gotten that one before. Interesting.
I am above 50 years old - I have Master's degrees in Business Administration and Statistics . I have also done programming at the initial stage of my career and had been working in the ICT area. Recently I have acquired knowledge of ML with Python. I wonder why I didnt think about being a data scientist earlier ! Anyway I want to be one one now - almost at the fang end of my career.. I have worked in the Middle East and India and in the UK for only 1 year after MBA.
I am a very fast learner .
Can anyone help me in getting data scientist or analyst or business analyst job anywhere in the world? I know my age maybe a negative point but then I have vast experience of working in different business area.
I m ok with bit of politics I am 18 year and I love mathematics and and want to enter into this field .I am enrolled in Btech CSc (from average college)on campus program and online Bsc degree in data science (from IITM ranked best engineering in India)
Should You encourage me to continue
Absolutely; what I'll say is, as you gain experience your preferences and the things you can and cannot tolerate are likely to shift drastically for you (i.e. you may find politics ok now but despise it years down the road); having said this coming from a strong mathematical background will set you up extremely well to be a successful data scientist!
Lol that’s what young and naive people would say until they’re older
First 🤩
Hah! I appreciate the dedication!