R package reviews | dlookr | diagnose, explore and repair your data quick!
HTML-код
- Опубликовано: 12 фев 2021
- In this video, we'll learn how to quickly diagnose, explore and fix problem in your data.
We'll have a deep look at missing values and ourliers and will be able to impute them with fancy machine learning techniques. And of course, we'll make much more.
If you only want the code (or want to support me), consider join the channel (join button below any of the videos), because I provide the code upon members requests.
Music by Vincent Rubinetti
Download the music on Bandcamp:
vincerubinetti.bandcamp.com/a...
Stream the music on Spotify:
open.spotify.com/album/1dVyjw...
Enjoy! 🥳
Outstanding. Tremendous functionality accessible with relatively few, simple commands. Top notch graphics as well. Keep up the good work!
Many thanks for a positive feedback, William! That's why I love R, it makes life easier 😊 really happy that it is useful to more people than just me.
An overall thanks for all the videos uploaded into this channel!
Thanks a lot 🙏 glad it’s useful!
This (specially this!) is marvelous, but also the rest of the series of explanatory videos. Congrats!
Glad you like them! Since they are already a bit old, you might find the more recent videos also useful. Thanks for watching!
Thank you very much for the videos!
These are life changing indeed.
You are very welcome, Sadat! :)
What a great presentation. I love this package. Thank you for introducing it and descdribing it in such an easy to follow presentation.
Thanks, Robert! I also love dlookr. I actually have only done reviews on packages I do enjoy and use everyday. So, you might find my other package reviews also useful. I thing the gtsummary is one of the most capable.
Why didn't I came across you channel till now? This is phenomenal
Thanks a ton, Sandip! That means a lot! My channel is still very young and I hope I'll produce more useful content. Cheers!
Many thanks for your help
You are welcome. Thanks for watching!
I really hope your channel will grow in the future. Your videos are very helpful to me.
I am very glad to hear that! Thanks! And thanks for watching, it's the best support!
Just awesome 👍
Thanks a lot 😊 I love dlookr too! You might also like the gtsummary review, if you did not see my video on it already. Thanks for watching!
I've been using data explorer and dlookr more in my learning journey thanks to you sir.
Glad to hear that! More exiting package will follow! I just do it slowly due to my day job. Thanks for your support and Cheers 😊
This is great stuff!
Thanks for the feedback, Haraldur! You might also like the Deeper Exploratory Analysis Video. It's long, but very dense with lots of nice functions, including and similar to dlookr
Wow, what an amazing tutorial. I've been using R for 5 years, and I've never used the dlookr package before. Your explanation was simple, focused and directed to the point, just as usual.
Thank you so much for your great videos. I really appreciate your work.
😊😊😊😊😊😊.
Thank you very much for nice feedback! I am glad you liked it. I have since then produced reviews on even more useful packages, "gtsummary" is one of the best. You might like it too.
Eso es lo maravilloso del universo de R, uno nunca sabes si te encontrarás con un dinosaurio.
Thanks a lot Dr. Yury. Nice and helpful video.
Thank Ange! Glad it was helpful!
Excelente
Thanks Martin!
Fanatastic video and code examples.
Glad you liked it!
Thanks for this one
You are welcome!
great package, kudos and keep the work!
Thanks a lot Sergio for such a nice feedback! I will continue! Cheers!
Really Really love your tutorial！！
Thanks a lot Yao! Your feedback is very encouraging!
Great! I love your videos. Please cover mixed models 🙏🤓
Thanks a lot, landoska! Noted! Funny enough, I do lots of mixed models in my job - medicine statistician.
Great video and great analysis ! Thank you very much!
I also like package("recipes") and package("vtreat").
You are welcome. Thanks for the tipps, I'll check them out
This a awe·some package.
Thanks David! I hoped this would be helpful not only to me!
Wow! Thank you! So many important informations. I have to watch this video several times. But one question: In which order would you use the packages "janitor" and "dlookr". Would be interesting to teach people how to load and handle "dirty" excel table, fix some excel problems (e.g. date as numbers or entries like "no data" in numerical columns etc) and if those problems are fixed to use "dlookr" to diagnose, explore and repair the data.
Thanks a lot for your nice reply, Hendrik! I would use janitor first and dlookr on top. I guess you already have seen the janitor video on my channel. If not, feel free ;) I also have one video on tidy data, where I show the dirty table, but there is not much of R programming. Thanks for watching!
Yury,
I noticed that your code for imputing outliers in the diamonds data repeats and is thus prime for a for loop, apply or map function (this is roughly 14 minutes into the video). I did not try to get too fancy so I wrote a short for loop to iterate over the methods. The function generates the plots one after the other. I thought I might share this with you and your viewers. Here is my rather crude code:
imp_na_method
Thanks for the for loop, mate! I have to say, nowadays I mostly use missRanger, because it's a very fancy and multiple imputation. I also always check the imputed values and they never disappointed so far.
And thanks a lot for a nice feedback! I am glad videos are useful :)
Yury,
I have duplicated your code and it basically reproduces with a few exceptions. For instance the correlation plot is not a matrix with ellipses but rather a colored chart with the r values. I guess as the package gets updated we will see some variations. Still good stuff - thanks.
P. S. I ran my code in normal r-session with script rather than in RMarkdown.
Sure, usually, they get better. It's just amazing, that this is open source :)
How do you get the results in the same window as your code? And the ability to preview graphs?
Via R-Markdown document instead of R script
Amazing video. This will be a great package for my EDA work. Many thanks.
Is everything ok with your website in the video description? I keep getting a 404 Site Not Found error page. Same result when I try similar links in a few of your other videos.... ?
Thanks, I did a follow up with many more packages „deep exploratory analysis“ .
My blog was shut down because they want me to pay for increasing traffic. I refuse to pay for doing something good for the world in an open source software. So, it might take me some time to find the alternative. But that’s not a problem because the blog is actually the script for videos word by word. And RUclips is still free.
@@yuzaR-Data-Science makes total sense. Have you considered moving your blog over to Github Pages, which is free and should play nicely with your script as blog or code or code on blog pages?
I actually even tried to move it to github pages myself, but something was off, and I could not go online. I am not an IT guy, so I am waiting for a friend to have a look at it and may be help me to solve it. but as you can imagine we both me and my mate have an normal everyday job and life, so the priorities are often not on the blog. anyway, thanks for your support!
8:19 plot_normality()
Is there a way of displaying qq plots too for the log and sqrt transformations? Can we request other fancier transforms such as Box-Cox or Yeo-Johnson?
Separate but related question. What package looks at a predictor and returns the transformation that best normalizes its distribution?
hey, sure, I'd use:
> ggpubr::ggqqplot(log(mtcars$mpg))
> ggpubr::ggqqplot(sqrt(mtcars$mpg))
and yes, sure, there is any transformation possible. Just type "?plot_normality()" and look inside. Hier is an example: mtcars %>% plot_normality(mpg, right = "Box-Cox")
these are transformations: "log", "sqrt", "log+1", "log+a", "1/x", "x^2", "x^3", "Box-Cox", "Yeo-Johnson" possible with plot_normality().
To your last question: I am not aware of any package for that, but there might be one. However, I am not a big fan of transforming the data because you kill interpretability. Log-transform is the most harmless in my opinion. I think it's a better way to use the correct model to fit your distribution.
@@yuzaR-Data-Science Thanks a lot. Look forward to a vid on Decision Trees and Random Forests with R packages... 🧁
Is dlookr the best package for outlier diagnostics and correction?
What we call "the best", depends on a lot ... but, I think, it's certainly useful enough to stop looking for other packages :)
@@yuzaR-Data-Science What's the bestest, most prettiest decision-tree visualization package? Rpart and the like all look pretty industrial.
@@chacmool2581 Don't know, because I don't use decision-trees
I have a question plz, Why did we put “temp” as a predictor to imputate missing values in Ozone variable ?
simply as an example of a predictor
@@yuzaR-Data-Science Am sorry, I cant get it !
sorry, what exactly can't you get?
sorry, what exactly can't you get?
@@yuzaR-Data-Science Do we have to put a predictor to impute missing values in a variable?
Amazing video!! thanks a lot!!!! but the code link is broke D:
Thanks for the feedback! Sorry for that, man! Netlify shut down my blog since they want me to pay for increased traffic. I refuse to pay for doing something useful for the world (without earning absolutely nothing) and since R is open source. But I want to reopen it ASAP, as soon as I find an alternative for Netlify. It'll take some time though, because I am not an IT guy. FYI: my blog is actually the script for the video, word by word, code by code. Thanks for understanding! But if you want the access quicker, consider to join the channel and becoming a member. For members I provide the code immediately. Cheers!
I got an error: "Error in html_paged_target_numerical(reportData, targetVariable, base_family = base_family) :
object 'index' not found" when I ran
airquality %>%
eda_paged_report(
target="Temp",
output_format = "html"
)
I don't know what's wrong with this dataset ... may be it's a numeric variable Temp. It does not work at my computer either. However, I tried iris dataset, and it worked flawlessly:
iris %>%
eda_paged_report(
target="Species",
output_format = "html"
)
If you don't figure it out by yourself, report a bug to the package github page
@@yuzaR-Data-Science Thanks, Yuza.
Happy to help
Doesnt work with Quarto, sadly.
Oh, good to know! I didn’t try it with quarto
What parts (still) do not work with Quarto?