Day in the life of a remote Data Analyst | Using Machine Learning *productive*
HTML-код
- Опубликовано: 11 июл 2024
- A day in the life of a Data Analyst Working from home. Today, I start a new project where I use machine learning, which is always quite fun. This will be my last video on "Data Analytics/Tutorial style videos" since the channel is changing to more opinion based content and finding IRL projects I can speak about is quite difficult!
For the sponsor today, I’ve collaborated with the University of Cape Town and GetSmarter to bring exclusive access to their online courses. Whether you're eager to level up your skills or delve deeper into a subject, GetSmarter has got you covered!
Plus, enjoy a 20% discount on any UCT course. Link is below
2ugetsmarterinfluencermarketingcreator.sjv.io/75ZkEg 
Наука
Those are the videos I love the most: insights into the actual work!! :) thank you!
Thanks for sharing your day in life!!🙏
Great vid and realistic! A lot of tech vids are like “then I go to yoga”, “then I go to the spa” etc and I’m like, no that’s not what we do at all in the real world 😂😂 you make me miss SA so much. Thanks for the video ❤
No I just sit at my desk all day with a sore back and loadshedding
Just diiscovered your channel based on a YT recommendation, i m amazed at the production and the content quality. I m a cloud architect with formal stats background, considering to delve more into ML. Your practical context and insights are very inspiring :)
It's the first time I see someone actually working with this mouse...
brodaaaaaa! I thought I was only that noticed
Since the Magic Mouse has 35% market share (statista) maybe you are not around enough people who work 👀 I’m a coder and use it for 10 years... almost all of the other engineers in our office too. Combined with BetterTouchTool it’s amazing to have 3 dimensions of travel for data sets, code and large digital white boards.
Yeah the mm and keyboard comes with the iMac. It also works, and that’s basically all I care about when it comes to computer accessories
I love the Magic Mouse. It makes scrolling and gestures on Mac *chefs kiss*
thanks for sharing this. pretty interesting to see
Thanks, they was interesting!
Awesome, home office working schedule, love your natural break views. Thanks for sharing, see you on your next video. 👌💪🥰👩💻✨💎💃
A day in the life of a tech employee doing an actual job.
you're on fire with the content! :)
Loved the quip on feature engineering 😮 Havent touched ML and AI in a few years, and the explanation really hit.. like ohhhh, lol 😅
It would be interesting to enrich your dataset with, as an example, information about their client's credit score and, if they (your client) have access, something like gross salary or something related to how much they've made in the last year. That could help the model to make better predictions about their actual ability to repay their (your clients' clients) debt. :)
Very informative video!
Except for you not showing the actual making of your coffee, great vid. Seriously though, it’s good perspective to see how you structure and do the work. Not too far off from some of my days (remote since 2005)
Why wouldn’t you just use the Mac terminal to merge your excel files instead of using python you can merge them all pretty fast without having to write any code. There’s nothing wrong with doing python. I was just curious? personally, I would import all the data into SQL, then clean it up. That’s probably because I’m not a fan of Excel.
As a long-time code slinger (began with 6502 chipset) let me recommend you dramatically improve your ergonomics. In addition to the comments by others on this topic, raise your chair and use a split ergo keyboard (I use Cloud Nine ErgoFS, love it) and ergo mouse that is, preferably, a thumball (love my Logitech MX ERGO). Both may feel odd at first but in less than a week you'll master them and you won't get carpal tunnel, cold hands/fingers, shoulder/neck aches, etc.
Hii im glad i found ur channel, I've always wondered if i could learn Data analytics as software developer, i guess i can seeing That's what you do
Machine learning part is very good. I didn't know anything about it.
Your monitor is not in middle of the desk thats interesting, how is your neck after a day?
i would love a 2nd more technical video on everything you discussed today. would your course on Python for Data Analysis: Projects to Power Your Resume teach me to do what you did in this video? please assist I would genuinely like to know skills tools etc needed to do what you did in this video
Is feature importance similar to parameter tuning?
Would you say it’s worth being a Data Analyst.
What would you say to the beginning Sally salary should be good for
@1:25 As a freelancer, you must be your own project manager. How did I never really get that before? Thanks Dee!
This kind of thing bugs me. The most important feature is age, which probably really is the single biggest factor in predicting whether someone will pay. However, what bugs me is the question what a company wants with that information. Are they going to treat customers differently based on the outcome of feeding their info to this model? We recently had a big scandal here in the Netherlands where a model like this was used for fraud risk assessment and it turned out the single biggest factor was nationality. The problem with this technology is that it's going to lead to decisions based on a person's traits that are only a statistical correlation; not relevant on an individual level, but it will affect individuals.
This is a really interesting question that I think is fascinating as an outsider and from the academic study angle. The problem we see more and more is when these interesting analysis projects are used to drive business decisions vs educate a business about their weak points. Instead of finding solutions, they generally choose to eliminate that risk factor. The risk factor is of course based on data correlating to a demographic or personal attributes or whatever which hits a really bad ethical game show of "Is This Discrimination Illegal or Nah?".
Large studies of people related topics and behaviors always segregate and discriminate but not in a bad way, it's just how you divide and draw conclusions that can be defined. When a company makes money from those decisions is where I see a problem and I think most people do.
I'll give an example. Statistically, women between 20-35 years old have more expensive medical claims than males in that age. The reason of course is pregnancy related claims and child birth. It would be wrong to use that information to increase the cost of insurance for women because there is a huge caveat, not all women get pregnant or want to have kids during that age. So it's wrong and unethical and maybe even illegal to do that depending on where you live. It's interesting for sure to see the data and it's fine for curiosity or research but when money/treatment decisions are made based on it, it crosses a line of some sort.
That bump @0:31
What IDE were you using for the Python development? It wasn’t Pycharm. You were not using Jupyter either which is popular with Data Scientists.
Looks like Visual Studio Code
It doesn’t look like VS Code either, VS Code has a limited tool bar on the left of the edit panes, this app seems to have a much more comprehensive top tool bar. I don’t use VS Code for Python but I do use almost every day for general editing tasks.
@@andrewgrant788I found it, it's called Spider
Interesting! May I note one thing: Your sitting position is not ideal. Your desk and monitor is too high, therefore you always need to bend your arms upwards, also you need to look upwards, which in long term can result in neck strain oder maybe also arm and back pain. I think if you could just use a lower table this could be mostly fixed.
The toughest part to me feels like what to do with the predictions made by the model. Are they going to deny services to customers which the model predicts might not pay within a certain timeframe?
You are correct, that’s often the hardest part. So this model will just help their admin staff communicate with the more risky customers. So the staff won’t spend effort in an invoice that will have a high chance of being paid in a month, they rather spend the time chasing more risky invoices.
Hello, since 15 years i code bots for scraping automation (php or node js etc..) im not using python but i know it is common langage for big data ,my question is " do i am data analyst? i mean i do exactly same as you people except that the data was millions products scraping daily or dozen millions of data wanted by some company but anyway this is just data scraping right ?
Yeah that's right why u didn't play more with data u could have gained a new skill in that time ,gl anyways
Is there a specific reason not using Pandas? Or is it just me cannot imagine looking at data and organise it without it? 😅
Pandas are stupid. The only thing they do is munching on bamboo.
I use pandas all the time, tabular data -> pandas. Always
So I guess this machine learning program works by making correlations? What country are you from anyway? Place looks nice.
Yeah in essence, the underlying structure of the model is a bunch of decision trees. I’m from South Africa 🇿🇦
@@codingwithdee I was thinking you were from India or something!
Did you do any assessment of potential legal or PR risk? I see gender in the feature importance graph. Isn't that a protected category?
wrist pain, back pain, neck pain - hard to watch someone sit and type in such un-ergonomic position.
Then don’t sit like that. Not everyone has your issues 😂
Hahaha. My sitting posture is very bad but I move around my house when I code, I just can’t for a RUclips video because the lighting works well in my office. I also actually had a back issue because I play padel twice a week. I thought I hid it well ☠️
Yes, either chair too low or desk too high. Get a new higher chair or have a carpenter trim part of the legs off the desk. Even a firm chair cushion would help a little.
My first thought was she should sits higher.
As a software engineer I've been thinking about moving into data analytics. However a former colleague who works in data science has said that it's mainly people with PhDs who are competing for jobs. Do you agree with her, or is it more open than that? (I just have a Bsc).
Yeah I think the data science space is quite congested now, I think it’s definitely the more famous field in the data industry. Although you are an swe, if you start skilling up on data analytics, those to skills together are quite wanted.
So what I actually do is build data related applications. The next steps for this project would actually be building an application/pipeline that will send triggers when certain high risk events happen. And there’s a lot of companies who have a solid data analytics team but none of them actually understand the SWE side to create applications that drive the insights to the stakeholder. Anyway, might be something to think about!
@@codingwithdee Thanks for answering, I appreciate it :-)
data analyst doing ml?
That keyboard makes my wrists yell for help. Edit. That mouse also. Ergonomy out of the window.
When you work with ML why do you call yourself a data analyst instead of a data scientist?
Your desk seems a little to high. I think your arms should rest in a neutral position and your wrist should be somewhat straight. I've learned that recently. 🤷
I think you're the perfect woman! 😊I've never met a woman programmer before.😐
Auch! Your setup is so un-ergonomic that it hurts.
Wait was that instant coffee ?
That code will never work
I knew someone will clock the instant coffee! (I actually can’t drink coffee that too strong, I get headaches)
@@codingwithdee Type mismatch: cannot convert :)
I have zero clue as to what you do; I can, however, watch you open a can of tuna for two hours.
I would love to do some machine learning with you.
Love your videos, but please, I hope that is not your real workarea? As others noted, you need to fix that to avoid future problems with your sholders, wrists, elbows and back. It may seem ok now but it will come with a vengeance later on. We want more tech-videos in the future. 😀
Are you actually writing python? I get chatGPT to do about 90% of my python code now.
cutie ❤
ASMR accent 😊
So in the "real version" you do all that while not having your yourself made camera-ready looking like Lebowski?
data analyst doing a job of data scientist...😑😑. I hope that your salary is bigger than a "simple Data Analyst"
You are so beautiful :)
I don't know where you acquired your technical skill sets to build your ML model but with all due respect, that model is garbage and that is being kind. Your information is completely misguided and is grossly inaccurate. I guess that's what happens when you have a Data Analyst exploring the creation of a ML model instead of the appropriate position for which this work is better suited - a Data Scientist. As a Data Scientist, I can't begin to identify how many errors were presented in this video. The presenter did not provide the results which is really what I was hoping to review. Perhaps that was by design that this output was withheld from the video. If I were grading this video on content accuracy I would give it a flat F. Let me briefly explain, correctly, what was egregious about this video. 1) Feature Engineering was incorrectly defined. Your definition is flat wrong. Feature Engineering analyzes the existing variables in a dataset to determine what variables are most impactful to the response variable. What you described was the creation of what are called CALCULATED VALUES, which are new variables that are added to the dataset based on data from other variables therein defined. 2) You did not resolve the issue of overfitting which your model will actually be because you missed a crucial sub-step to prevent it. What you've done is what 99% of new practitioners in the field do before understanding the statistical side of ML modeling. You need to identify the standard error of every numerically defined variable in the samples of your dataset to determine whether the variable sample statistically approximates its underlying population. Beyond that, you actually missed an entire step in your process model when you combined all the data from your spreadsheets. You didn't need to do that and you didn't need all that data to create a model. So what you've done is create a high degree of goodness of fit which was your 85% predictive results for THAT dataset (only) but your overall predictive capability will most likely be less than 50% when your model is tested against data it has never before seen because you have not tested your data for the standard error. 3) Finally, you did not validate the data you used before establishing your training and testing dataset. How do you know the data you are using is correctly defined? How do you know whether or not some of that data was incorrectly defined? You don't and because you haven't provided a test for it you've essentially included garbage data into your model. Consequently, how did you handle missing data within your dataset? You never addressed that issue and it's one that is paramount to a discussion of ML modeling. If you had no missing data that should have been presented to your audience. I could go on for about another hour but I will stop there. This was one of the most misleading videos on ML that I've seen on RUclips in a while. Stop putting misleading, half-baked information about ML out there. This is egregious at best. The final lesson here ladies and gentlemen is NEVER hire a Data Analyst to conduct the work that is unique to, and consistent with a DATA SCIENTIST, who knows how to correctly build a ML model.
Tell me you are not going to get addicted by coffee like NetworkChuck is. :D He is hillarious with the coffee.