Kaggle: How to Place in the Top 10% (part 1)

This is the first part, in a series where I will discuss some techniques for placing in the top 10% that I've used in two Kaggle competitions using just a laptop and no cloud. I made use of Python, Scikit-Learn, XGBoost, and TensorFlow. I will also be covering some of the Python code that I used.
My Kaggle Playlist: bit.ly/2Evax1Q
My Kaggle GitHub Utilities: github.com/jef...
My Kaggle profile: www.kaggle.com...

[@HeatonResearch]

More videos posted in this series, here is the playlist: ruclips.net/video/tSzrecBdyJc/видео.html

[@ianseward6310]

Can you put this in the description?

[@HeatonResearch]

So, for those of you viewing this... What is your language of choice? These videos will be using Python. Libraries will be sklearn, xgboost, & TensorFlow. Anyone currently in a Kaggle?

[@myguiltybody]

Hi! My language of choice is Clojure (or any lisp rly), and this is probably the 3rd or 4th time I've heard of kaggle. I don't have the skillset to participate, but i wanted to say you make great videos. Keep it up :)

[@HeatonResearch]

Thanks! Clojure is cool, I have not worked with it a great deal. Most of my pure functional experience is in Scheme.

[@brotherlui5956]

Hi Jeff, i followed your Fall 2018 course and the included Kaggle challenge was my first one ever. Unfortunately i missed the start somehow and had only 3 days to create a model. It was quite exciting to see that my model was not that bad as i first thought. In December i was busy doing the daily AdventOfCode challenges and now i'm looking forward to this series of videos which i'll use to dive deeper into Kaggle. I really like to learn from your experiences, you're a good teacher. Best regards, Thomas L.

[@tonym5857]

Python or C# ( TensorFlow )

[@HeatonResearch]

@@brotherlui5956 Kaggle is alot of fun. Takes alot of time. I should announce this semester's Kaggle by March 18th. You (or anyone) is welcome to join. Trying a real Kaggle is great too. The thing to remember with Kaggle, is that scores will not hurt you. So its always a good time to try!

[@suwengu6335]

Thank you so much! Looking forward to the subsequent videos on this topic.

[@HeatonResearch]

Your welcome, they will be coming.

[@leonardommarques]

ok, but how do I grow such massive mustache?

[@HeatonResearch]

Shhh.. Its my disguise. :-)

[@leonardommarques]

@@HeatonResearch :-{)

[@HeatonResearch]

Cool. Just when I thought I knew every ASCII emoticon!

[@KPOPYakuzaz]

I'm a novice Python developer, but work in an environment where i am CONSTANTLY using Python, PHP, JS, Java, C#. I'm interested in learning how to begin these competitions, but have no idea where i can even begin to learn how to use Python in this way - can you point me in the right direction? Awesome video, looking forward to more :)
edit: Just saw the part about titanic and kaggle introduction, whoops :p

[@HeatonResearch]

Titanic is a common suggestion and I agree with it. That was the first Kaggle competition that I started with.

[@bensisakht7076]

I’d also suggest you invest some time in learning about méthodologies of data science and statistics as knowing python is not enough for competing, coursera have some fantastic courses on data science. Good luck 😊

[@parvezmulla3324]

Waiting for your upcoming videos!

[@HeatonResearch]

Working on that now, actually.

[@alexianthefisherman]

Thanks for the explanation i really appreciate it

[@kazanova2337]

+1 , these are all good points.

[@HeatonResearch]

I am honored to have a +1 from one of the greatest Kagglers!!

[@kazanova2337]

@@HeatonResearch The honor is all mine -My first experience with neural networks was through Encog -:), so thank you for that!

[@shankarsuman8801]

Howuch time in years months does it take to reach to a stage of wining cash prize and becoming grandmaster If I start today?

[@npr1m991]

I'm relatively new to kaggle so I wonder why people publish those super features ? Wouldn't them have more chances to win if they keep it secret ?

[@momen_ai]

thank you. so useful.

[@HeatonResearch]

Hope to add the next video in a week or so!

[@ianyang8799]

Hi Jeff, 720p or 1080p quality video might be better I think

[@HeatonResearch]

Agree, error on my part for the encoding of this one.

[@kulasekhar]

wow! watching a 480p video after almost 3 years :O

[@HeatonResearch]

Sorry about that, wrong setting.

[@HeatonResearch]

If you want to have a look at the source code, I've started to upload it here. I plan on covering this in the next video Monday. github.com/jeffheaton/jh-kaggle-util

[@ayeoh47]

dope

[@brunovandekerkhove5688]

Thanks, you explain things well.

[@joaquinmorenoantuna9573]

Hi Jeff, excellent videos. It would be amazing if you can upload a video of the "Classify forest types" for the Roosevelt National Forest. Best!

[@checkpeck]

@Jeff how can I approach you with a specific problem?

[@ashish_neo]

You look like my school principal 😂

[@DannyPhantumm]

He looks like everyone's school principal

[@Q_QQ_Q]

Can we compute on cloud for free ? I have just old laptop with 4GB RAM :( #PleaseReply .

[@HeatonResearch]

Yes, GoogleColab is your best option there. I plan to do a video on how to use my files with CoLab.

[@robindong3802]

try colab.research.google.com or search colab on google, it is free of TPU and GPU.

[@sainath66666]

Or use kaggle kernels

[@ThunderAppeal]

You shouldnt be competing if you dont know how to upgrade your computer.
Youre incompetent by nature until you figure it out.

[@shortmashins4031]

How do i join a team in kaggle,

[@EngRiadAlmadani]

Great sir

[@ProfessionalTycoons]

thank you for this

[@talha_anwar]

Is not it their work? How can we use their work?

[@suryanarayanan5158]

They post it on the forum. So, you're allowed to use them.

[@bensisakht7076]

Thanks for the helpful video. Just as a constructive feedback, perhaps shorter introduction & less repetition would be useful. Cheers

[@gaureesha9840]

so even during the competition we can see other's solution?

[@HeatonResearch]

Only if they post it. Sometimes they will, usually not one of the top solutions.

[@maithiltandel6667]

If I stand in top 10% in the knowledge based competition like titanic dataset, then can I get a bronze model?

[@ammarazlan2919]

I'm interested in the description where you wrote: "..using just a laptop and no cloud.". Would you mind sharing the specs of your laptop and other experimental setups you use like IDE etc. Much appreciate it. Thank you.

[@HeatonResearch]

Macbook Pro 16GB, for tools, mainly PyCharm... Lately I've been using VSCode more. Of course Jupyter too.

[@ammarazlan2919]

@@HeatonResearch Thanks for the reply. Unfortunately, it's too expensive for me. What would you recommend for a student searching for an affordable laptop to do deep learning?

[@HeatonResearch]

@@ammarazlan2919 I would max memory over CPU. Also, you can look at cloud. Also, for free cloud, have a look at Google colab. Or, look at a Kaggle kernels competition (in this type, all computation is done by Kaggle)

[@ammarazlan2919]

@@HeatonResearch Thank you so much for your thorough answer

[@moazzimalibhatti]

Good. But to summarize your talk. You are repeating one thing a couple of times.