Data Science Has Changed - Here's What to Do
HTML-код
- Опубликовано: 16 май 2023
- Thank you for watching the video!
I offer 1 on 1 tutoring for Data Structures & Algos, and Analytics / ML! Book a free consultation here: calendly.com/greghogg/30min
Learn Python, SQL, & Data Science for free at mlnow.ai/ :)
Subscribe if you enjoyed the video!
Best Courses for Analytics:
---------------------------------------------------------------------------------------------------------
+ IBM Data Science (Python): bit.ly/3Rn00ZA
+ Google Analytics (R): bit.ly/3cPikLQ
+ SQL Basics: bit.ly/3Bd9nFu
Best Courses for Programming:
---------------------------------------------------------------------------------------------------------
+ Data Science in R: bit.ly/3RhvfFp
+ Python for Everybody: bit.ly/3ARQ1Ei
+ Data Structures & Algorithms: bit.ly/3CYR6wR
Best Courses for Machine Learning:
---------------------------------------------------------------------------------------------------------
+ Math Prerequisites: bit.ly/3ASUtTi
+ Machine Learning: bit.ly/3d1QATT
+ Deep Learning: bit.ly/3KPfint
+ ML Ops: bit.ly/3AWRrxE
Best Courses for Statistics:
---------------------------------------------------------------------------------------------------------
+ Introduction to Statistics: bit.ly/3QkEgvM
+ Statistics with Python: bit.ly/3BfwejF
+ Statistics with R: bit.ly/3QkicBJ
Best Courses for Big Data:
---------------------------------------------------------------------------------------------------------
+ Google Cloud Data Engineering: bit.ly/3RjHJw6
+ AWS Data Science: bit.ly/3TKnoBS
+ Big Data Specialization: bit.ly/3ANqSut
More Courses:
---------------------------------------------------------------------------------------------------------
+ Tableau: bit.ly/3q966AN
+ Excel: bit.ly/3RBxind
+ Computer Vision: bit.ly/3esxVS5
+ Natural Language Processing: bit.ly/3edXAgW
+ IBM Dev Ops: bit.ly/3RlVKt2
+ IBM Full Stack Cloud: bit.ly/3x0pOm6
+ Object Oriented Programming (Java): bit.ly/3Bfjn0K
+ TensorFlow Advanced Techniques: bit.ly/3BePQV2
+ TensorFlow Data and Deployment: bit.ly/3BbC5Xb
+ Generative Adversarial Networks / GANs (PyTorch): bit.ly/3RHQiRj
I offer 1 on 1 tutoring for Data Structures & Algos, and Analytics / ML! Book a free consultation here: calendly.com/greghogg/30min
The key is to be a "T shaped" person. Deep knowledge in a particular area, but a broad knowledge in the adjacent areas. Goes for almost any job in technology; we're seeing the same changes over on the infrastructure side of the market!
That's really good advice, I like that a lot
I ve read about T Shape people. I agree
Yes absolutely! It's all connected together and in the workplace in the absence of certain employees i would need to fulfil their place somehow, or even help them in the matter of crisis like in cybersecurity.
Incroyable.
It is what it is......
00:00 Data science job outlook is changing rapidly and it's important to know what to do.
00:14 Data science jobs are not getting automated, but the job outlook is changing.
00:29 More data and insights are available, but humans are still needed to interpret and utilize them.
00:57 Exploratory data analysis is not as important as before, but understanding Python code and libraries is crucial.
01:27 Building projects and having skills are more important than just having credentials and spamming projects.
02:21 Data scientists need to have software architecture skills and be able to build full applications.
03:04 Coding is getting faster, so companies will need fewer people to write code.
03:44 Knowing how to put together different components and building actual applications is crucial.
04:12 Traditional analytics is getting easier, but it's merging into building full applications.
05:04 Learning data science and building applications simultaneously is important.
05:32 Being really good at your job and building full applications is essential in the changing data science landscape.
05:40 “… Learn Advanced Machine Learning Architecture …”
simply put "BUILD FULL APPLICATIONS". thanks
Bless you.
Thanks
Here are the six most important points from the video:
• Data science jobs are not dying, but the job outlook is changing rapidly.
• Exploratory data analysis is becoming easier and faster with the help of machine learning models.
• Companies will still need human data scientists to build and put together Lego blocks of data, as chatbots cannot do this yet.
• Data scientists will need to know software architecture skills, libraries, frameworks, and languages to build full applications.
• Traditional analytics is merging with building full applications, and data scientists will need to learn how to do both.
• To stand out in the job market, data scientists should learn advanced machine learning architectures and build their own technologies.
Thank you for the awesome summary codelucky! This is super helpful.
The problem lays in that building simple analytics and simple models are the not the tasks of a "Data Scientist" these are a the tasks of a Data Analyst. So yes, Chatgpt can obviously replace a Data Analyst. In fact there are thousands of jupyter notebook templates that can be used to do this without the need of ChatGpt.
Incroyable.
What's a type 1 error?
@alienboogieman someone who study marketing and its considered a Data Scientist
@@jja7788 Incorrect. It's the level of significance when rejecting a claim when it is true just as "at level of significance of 5%, the true average lies between 10 to 25 in minimum wage" as an example.
@@alienboogieman You confused type 1 error with level of significance, not even Chatgpt makes this mistake :))))
This is super helpful and thanks so so much! Would really love to see a part 2 deep dive into this topic if possible
I don't think I have a direct part 2, but you're super welcome!! :)
Thank you for the video! Everyone keeps talking about how AI is changing jobs, especially in technology, but you are showing us what to do. It would be great if you could make a video about pipelines, etc. Thank you!
Yes absolutely!!
I can help you.
Absolutely agree!
I think abundance of online training sites (DataCamp / Coursera / Udemy) has made good fundamentals of data science fairly easy to find now from a recruiters perspective. I'm doing my AWS Machine Learning Certifcation at the moment and the cognitive leap from visualisations and hyper parameter tweaking to understanding full-on data application architectures and deployments is sort of staggering. The basics is stuff I sort of hope they might need, the big applications is what I know they will need.
What about simpler jobs like business or data analyst. Can you enter the field with less extreme tools?
If you don't mind me asking, for how many years have you been using aws cloud ?
The official site seems to suggest 2 years, but can it be done in less?
@Vincent Adultman Here's the odd thing. I've never actually had a job that uses AWS. All my learning has been done via AWS Skill Builder, Cloud Guru (the game-ified scenario training) and udemy courses. I passed my cloud practitioner about 9 months ago and just passed my Machine Learning Certification there. Very possible to do it in less than 2 years, but you need good data science theory to get the main themes, and I honestly think it's your luck how difficult some of the MLOps questions can be in the exam.
@@rossgo101 thanks a lot dude
@rossgo101 did you skip AWS Solution Architect Exam and went straight to Machine Learning?
I'm sure how good Python is, but few people talk about R as the default option when it comes to performing real data analysis using out-of-box packages.
R is a little weaker in terms of memory management and what not. It's very hard to perform large computations with R.
R has considerable limitations. Last time I used R it was inferior to Python for deep learning. The advanced statistical methods in R, like mixed effects models for intensive longitudinal data analyses, and advanced plots like the CD plots, were my selling points. However, Python is catching up and real data analysis can be done with Python.
If R's syntax remains as hideous and its performance keeps suboptimal, it'll be a tool used only by hardcore statisticians and perhaps in universities.
@aborref8119 my econ professors used only R and STATA. Not a single Data Analyst position has mentioned R as a requirement or beneficial skill. Love that for me
I started with R but switched to python, as R is not good for deployment and databases.
@@rafaeel731 No it is not true. For deep learning Python is hands down better, but for statistics (regression, clustering, multi-level models, time series .....) R certainly wins. Speed ? - use C++ calls (it is not difficult anymore in R), memory ? - > larger than RAM things still not good for Python either (use Spark, Arrow etc.).
Hey thanks for this video. This whole space is really muddy and hard to get a clear idea from someone who actually knows what theyre doing. Thank you for sharing and I really hope to apply this knowledge!!!!!!!
Good stuff as always! Can you give us a spoiler about what problem the startup you are working on solves?
No spoiler!
I have to disagree, my recruiter told me there are A LOT of people getting fired because they are using ChatGPT to get jobs but can’t keep them because lack of skill
In what way are they using ChatGPT to get jobs?
@@dannypakaz uhhhh for the programming involved with data science?
They put skills on their resume they don’t have but that they can “do” with ChatGPT
Hey Greg, thanks for this video. It gives me a little bit of direction in these trying times. I am a industrial engineer graduate who became a software engineer and am now pursuing MS in computer science - but I am struggling to decide if I should take more software engineering type stuff or more analytics. Due to how rapidly the analytics space is changing, I think my best move would be to just focus on becoming a full stack engineer
Thanks Greg! ❤
You're very welcome ❤️
My personal plan is to combine my currently ongoing programming education with my art-school education because I believe there is still a lot of untapped potential there. And I honestly don't even expect to find a decent job with those credentials in this economy, even though I believe combing X with programming e.t.c will be the future for X. I dunno i still suck at programming anyway lol
In order to get into data science field, we should not only grasp basic knowledge, but also the advanced methods and domain knowledge for applications
insightful, thanks!
A top-notch true, dude.
Know the basics of your stacks and be good at prompt engineering
~ Respect
Great stuff, I have been serving two roles for my company for a while, one is business analytics ChatGPT made that so much easier for and now I have time to do my real job.
Sounds great!
that's true! aligning the people in the right direction!
I try!
I’m about to graduate with my bachelors in data science. Definitely needed this
Same here! High 5 and congrats!
@@BrianGrant-Home we did it Brodie 🫡
The main question is - how to get a first job in DS without much experience, even as unpaid intern?
It turns out, that nobody actually wants inexperienced workers. Most of the companies, especially startups, want the job to be done.
It's always tough to get the first one. You'll need to build up your resume and skills as much as you possibly can. Grind!
@@GregHogg Trying to do my best and learning something new every day. Thanks to your great videos, as well... :)
Still, getting it to a professional level is a challenge for me.
Data Science/Business Analytics/Data Analytics/Data Engineering will soon be automated, the sexiest jobs of the 21st century now are - Cybersecurity, Product Management, Program Management, Network Engineer (unexpected, right), Electronics Engineer and Software Engineer
Companies probably don't want unpaid interns, especially in Europe, because soon you are gonna accuse them for slavery. In USA and Canada, I think you can have some luck.
@@univuniveral9713 Actually in Europe the companies are more open towards remote interns from all over the world. In US they want only citizens or green card holders.
I have been learning alot of coding lately. I do learn online alot but I just wanted to point out that you should probably still make a ton of projects. My ideas have got me to learn so many different areas I would not have thought of
Thank you Greg.
I think the Lego bricks analogy you made is very appropriate.
Okay perfect!!
Thank you for this video, Greg!
A thought that struck me right away - can you update the Data Scientist roadmap, having mind the changes you mentioned?
I already have cloud stuff in the roadmap:)
@@GregHoggwhere can I get the roadmap
@@GregHoggwhere can I get the roadmap
@@GregHogg And what courses would you recommend to learn to build full applications?
Thanks for the solid advice.
Very welcome - have a great day!!
Very smart advise.
Thanks!!
Thanks brother got to learn more❤
Good luck :)
Greg is right. I don't see that a lot has changed however. A full stack developer has a lot more opportunities than other folks.
Why how?
@@akihiko99 Where, when, who?
Sure, what company doesn't want to pay 1 salary for 2-3 positions?
@@jcantonelli1 Why not understand the whole system instead of being an assembly line worker?
@@datapro007 That might be possible with smaller, simpler applications at start-ups, etc. - but, at larger organizations no one person can deeply know every single aspect of an enterprise-level stack.
We don't have "full-stack" surgeons for exactly this reason.
Yo I want that Python hat. I view ChatGPT like I do pinyin when I type Chinese. I can't manually write each character, but I do have to know how to read Chinese. The computer/phone gives recommendations as you type in the phonetics of the words. Sometimes the recommendations are really poor, so it's vital to have a strong understanding. It's efficient in the correct hands
Thank you for this.
Great video Gregg!! Thanks! Quick question though, what is the top course you recommend to learn how to build the lego castle first? Cheers!
No problem! And I'm working hard on building it!!
@@GregHogg cool! Let me know!!!
Hello,
I come from a different background in Mechanical Engineering and am pursing a MS in Data Science. It feels like we are learning very superficial Data Science of knowing stats, ML algorithms ,and how to apply the ML algorithms. I worry that I am going to graduate with just knowing baseline models, without making a project of my own.
You had mentioned, instead of this superficial knowledge, to build full applications. But can someone explain what a full application entails and a typical structure/plan for how to build such application?
Much appreciated
The "hire a data scientist to build a full data product" thing only exists when we talk about start-ups, where this is only done to save heaps of money that would have otherwise been sucked up by highly skilled and highly earned developers, architects, and data engineers. It will never exist when we talk about medium-to-large enterprises, where no stakeholder on planet earth will ever trust a customer-facing application built solely by data scientists.
If that's the case, then data scientists are only for analysis and model building. This is tremendously easier than it once was, so there's gonna be a ton of competition. Gotta stand out somehow
Inclined to agree with this.. awareness of a new expanded DS does not apply to small medium/large enterprises yet... my thoughts are better to go from swe + ML + cloud to DS expanded role. Than DS + full stack Swe +cloud+ops....
@@GregHogg can you make a part of this video on this stuff
@@GregHogg then what else were data scitist for exoet analysis and model building
Are data engineers truly engineers though?
Greg's desperation for your attention in quite palpable, but I'm not blaming him, I blame the RUclips algorithm for making content creators plead on their knees for subs and views. I miss the days where it was just about providing useful information.
Honestly, kinda with you. I mean, I certainly tried to provide useful information. Sorry if I didn't. But yeah, you kinda have to be clickbaity these days.
Great video 👍...What advanced ML architectures do you recommend I learn?
Well obviously transformers are pretty popular these days, so that would be a good recommendation
@@GregHogg Great. Thank you for the recommendation.
Pretty good for the current situation, but you are referring to the future. The situation is rapidly evolving and I think its not very accurate to assume that it will stay the same in the near future. Even in the next couple of years things will change.
Right now I am working on a startup that uses GPT-4 to control VMs to install software or write simple web applications which are hosted immediately at unique IP addresses on the internet.
I have also done a contract recently to pay the bills where the user asks a question, GPT-4 writes the KQL for pulling data from a table, and returns the result. It can also (if appropriate) analyze the result table and give an answer in prose, or even generate an arbitrary graph on the fly if that is requested.
You are correct about the limited context, but that is also changing quickly. Anthropic's new model can ingest 100,000 tokens. Its not quite at the coding level of GPT-4 but they will get there. The hardware and software will continue to accelerate, especially over the next couple of years since there is still low-hanging fruit for optimizations of this specific application (GPT) in hardware and model tweaks etc.
Within a few years, and certainly within five years, we should anticipate human-level reasoning at 100 or more times human thinking speed. Available to consumers (if not prohibited by governments). That is how fast compute performance improves. That's not even really speculative given the history of compute efficiency gains.
Incroyable.
Good question
I think you also need to become a pro using pretrained models for solving your problem. It saves you time, money and data size and results are astonishing!
This is also really good advice.
Any pointers to the resources that can help develop this kind of skill sets? Thanks.
I think feature engineering is the main thing you should be learning in DS.
Thanks!
Thank you so much, I really appreciate it!!
Thanks for the amazing content.
I have an intermediate level knowledge of deploying ml apps on gradio and streamlit.
Will this be enough for me to get an entry level role?
You're very welcome! I would learn something REST-based :)
I disagree, if you're a data scientist in charge of prototyping an algorithm that will make critical business decisions and/or potentially affect the lives of many people. I'm sure the last thing you would want to do is to spend your time on JavaScript/CSS/HTML. If ChatGPT can help you make graphics more quickly, that's great. But the world of data analysis, unlike engineering, is something that never ends.
If you're comfortable in an important job already, of course you can focus on that. This advice would be for the folks that have not yet cemented themselves as an important part of a company
One question, where does your expertise come from?
I think this is the right answer. Building things that combine the entire process, front to back, will demonstrate the skills.
Yeah absolutely!!
What's the confidence level of your claim?
What "path" do you recommend people take to become gig worker HR specialists in temporary hiring agencies and clerks in government unemployment lines?
how do you know it can't build the whole castle soon?
So, uh... What do you comsider a full application? I unfortunately only know how to build jupyter notebooks...
Pretty much anything else haha
Thanks, THis is the best advice i've recieved thus far
Thanks for the vid. Straight up no BS.
Thanks so much for doing this video!!!
be an exceptional person, got it. what a tip. nobody would have come up with this grandious advice, definitely worth a 9 min watch about nothing. XOXO
Im a little half way through my data science masters should I i finish or switch out, the field is looking bad, i dont feel like Im learning enough
any update on what you decided to do ?
Any suggestions about what kind of application should I build?
Whatever you enjoy
Great job
Thank you, best of luck to you! :)
It's true that you said but not at this time. Data science nowadays even don't have ChatGPT or any AT tools, juniors need to know a lot of stuff more than just building good models or EDA because everything is much easier than before,
When I first get a job as DS, I need to. know how to do SQL, Python, multiple BI tools, and strong communication skills which is a core thing from my perspective. If someone thinks that Data Science is just only creating the model and EDA, I would know that they aren't even in a field.
I felt really bad for further newcomers because the minimum requirements would be much higher, You need to know something much more than before because ChatGPT could help us and reduce the time and cost to hire a junior and intern.
Yes this is correct.
I studied industrial product design & I work in marketing. What would you think I should do full stack softw or data? I usually work for myself, as a freelancer. And I am looking for more building blocks to build my portfolio & start my own agency.
You will need to figure that one out for yourself! :)
Amazing 😊
Thank you!
Moments ago I decided that this issue (collaboration, and big picture building issues) was my unique skillset giving me an awesome entry point into the field and then I watched your video telling me that same thing minutes later. Boy do I wanna talk to you.
Amazing!!
Does that mean to develop ourselves more as a Machine Learning Engineer, so basically to become more of a software engineer with the MLOps knowledge?
Yes!
This is a really great advice. Hope people get it.
Thank you!
Thank you for the video, now need to become a full stack data scientist for that I just discover Runaway for deployment Could you recommend any one like this?
Never heard of it
I have software that will change how we use computers on a fundamental level. I'm not the only one working on it and I'm pretty sure I can't be the only one working on it with the same intention that I have. Wish I could find some VC and a CEO.
I know of an inventor and programmer that you might need. He also created something new for COVID, but it is not yet implemented. It is published by Cambridge university press. However, to implement it, you guys need to work with biochemical teams of the pharma. It is worth talking to him as this could land you all in billions of dollars. It all depends on how you guys implement and market it. For that, bio and pharma have to sponsor you because you need an income while working on it. I am also interested in this.
@@univuniveral9713 Thank you for recommending someone, but it doesn't sound like we are working on the same type of thing to me.
Thank. Did not regret entering the contract.
It’s nice having a clearance job. ChatGPT is banned. So I’m safe for awhile
Hi Greg, I am kind of confused? From your conversation I hear a lot of generalizations. You mentioned that Data Science has changed and is now drastically changing. Forgive me but I kind of understand what you are aiming at but what you are saying lacks substance. Would it be possible if you could elaborate further in detail and articulate precisely in clarity your overall meaning?
I understand, every RUclipsr state that Data Science is a generalist role depending on the company and their requirements. But when I hear an explanation about what Data Science is, I tend to hear that Statistics is the primary tool, please correct me if I am wrong. I am guessing since statistic is applied then I would be under the impression data is the source and data can only be found within a relational database, correct? Such as a relational database like Azure, AWS, MySQL, Snowflake? How do Data Science provide value to Organizations when the job isn't clearly defined?
Lastly, just out of curiosity? Why are you interested in JavaScript? How does that pertain to your job? Is learning JavaScript a hobby of yours? I mean I can understand learning R? Python? Even possibly Swift for iOS because I hear Swift is an exceptional programming language which is capable of dealing with large datasets.
To be honest if I were aiming at becoming a data analyst even a Data Scientist. I would rather stick with Python because Chris Lattner the creator the LLVM and the creator of the programming language Swift has created a programming language called Mojo. Mojo is similar to TypeScript in a sense but yet it is not. It acts like a wrapper over Python and to my understanding it is a very fast programming language capable of multithreading and other amazing things which make it really good for AI and machine learning. I am not trying to insult or anyone. This is not an attack but merely someone who is very curious and interested in what a Data Science is despite my personal opinion that Data Science will be nonexistent within the next 10 years, I hope I am wrong. Thank you.
You're gonna have to summarize this, this is way too long
Thanks bro 😎, you've earned yourself a new subscriber
Super glad to hear that ☺️
Thank you for adding tracking links
You're very welcome
This just sounds like a token window problem
So you need to know both ML methods and how to build actual applications, rather than just knowing how to do the machine learning model experiment and analyze the data.
In summary, Data Scientists will need to demonstrate tangible value instead of endless tinkering with data.
Yes!
Thank you
You're very welcome!
so did I just watch a 6 minute video to be told that in order to get hired I need to be good at my job
So the advise is to be good at the job to be competitive. I expected a bit more - perhaps before and after?
Pretty sure I said more than that lol
Thought of getting into ds but it is becoming more complicated
What do u say get into it or quit?
Get into it
What does it mean to build data scince applications
Like something outside of just a notebook only
و قل ربي زدني علما و علمني ما ينفعني و انفعني بما علمتني
اللهم افتح بيني و بين مستقبلي فتحا مبينا و أنت خير الفاتحين
Fully agree.. man..fully agree
Thank you!
"Code stitcher" is the new job title
I like it
Data Science/Business Analytics/Data Analytics/Data Engineering will soon be automated, the sexiest jobs of the 21st century now are - Cybersecurity, Product Management, Program Management, Network Engineer (unexpected, right), Electronics Engineer and Software Engineer
thank you!
Very welcome :)
good videos and right to the point
Think stats masters is a good idea rn given the job market?
Virtual assistants will not gonna replace anyone. The more people and faster gonna jump in that trap the better for rest. Cybersecurity and hacking stuff will be so much easier then ever 😂 Web pages and data is safe(at least little bit) because of human factor.
You forgot to change the background screen?
Whoops
Able to putting things together is just matter of time, look at auto-GPT, unfortunately, I think data analytics is no longer a good career as it used to be.
We'll see. There's a lot of errors and extremely particular stuff that goes into full, completely correct and SECURE applications
What do you suggest is i am from cybersecurity background what do you suggest me being at my place, please suggest me a high level roadmap
I disagree. There's just too much thrill about full stack ML apps. You need a deep knowledge in mathematics and this is extremely hard. I simply don't understand how anyone can pretend to build a system on fuzzy concepts.
I am thinking about taking the data science course which one to choose and is it better to take the data science course in 2023
@@ChandlerLaura where we can talk with each other
What's the green screen used for 😂
In my talking videos, not much lol
0:54 I immediately know this guy is on my topic.. 137k views of us!
But how!!!! Giv us something more on the thing of the ending
sorry im new here. what exactly do you mean by "building applications" in data science ?
Apps. Usually web apps that are on a cloud.
Do you mean like building dashboards and platforms? On cloud like AWS or Google cloud?
Im studying data science in Australia right now n i have been enthusiastic about it for about 2 years now n still learning it.
I thing nobody absolutely no body talks about which annoys me is importance of Statistics/ Maths and “ANALYTICAL THINKING”
Anyone can code come one guys if you need to make a difference make use of statistics and be better at analytics
Everyone always forgets AI gets better and better...
Thanks Bro!
Very welcome, thanks for the support! :)
I think depending on organization need data can be customized like government don"t want too much garbage slowcdown their machine or the advertising and marketing need only human behavior and culture fashion , or news media can lead data or follow investigation report in history all need different tactics or instruments which determines privacy. But best storage is DNA in tree or human memory are the infinite
Hi 👋 new subscriber and very much a novice in data science! I liked your honest and direct style. What exactly do applications mean? Or could you point me to one of your videos on the topic? Tried searching and couldn’t find one
Things that do stuff, like outside a jupyter notebook
@@GregHogg thank you!
I mean,I get what you are saying but if I understand the entire picture and can build the whole thing myself, why am I working for someone else instead of building a start up,
Ding ding ding!
@@GregHogg Nice, I see your point
Chat GPT output still needs someone to debug it
Yes, it does