Neural Networks from Scratch - P.1 Intro and Neuron Code
HTML-код
- Опубликовано: 18 сен 2024
- Building neural networks from scratch in Python introduction.
Neural Networks from Scratch book: nnfs.io
Playlist for this series: • Neural Networks from S...
Python 3 basics: pythonprogramm...
Intermediate Python (w/ OOP): pythonprogramm...
Mug link for fellow mug aficionados: amzn.to/2xcyfPC
Channel membership: / @sentdex
Discord: / discord
Support the content: pythonprogramm...
Twitter: / sentdex
Instagram: / sentdex
Facebook: / pythonprogramming.net
Twitch: / sentdex
#nnfs #python #neuralnetworks
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
Errata:
~10:26 - The animation depicting the number of tune-able parameters counts biases as if the input neurons have biases. They do not. For example, the 10x8x8x8x2 neural network has 26 biases (8+8+8+2), not 36.
sentdex makes sense
Already did one :p www.kickstarter.com/projects/sentdex/neural-networks-from-scratch-in-python
Sheree yantra show similar structure 3+ 3+ 3+ om om om 🙏🙏🙏
i wanna create a neural network with at least 1000 outputs
but first i got to build up some money cause i only got a intel i5 6400 cpu then again... i could set my gpu's workload to computing my motherboard has a gpu as well so i shouldnt have a problem doing that
Dropping this video is such a big news that even if aliens landed today it would be the second most important thing that happened today
Only because the aliens are already here, obviously. ;)
@@sentdex that's something an alien would say
@@sentdex and nobody cares because this happened
They did land. In fact, they are watching here with me right now. Were having such a good time. They want to know all about Python.
@@sentdex I am the alien oh yeah yeah
“What I cannot create, I do not understand” - Richard Feynman. My motivation for watching this series.
Stfu tool
@@aaronsanders6162 lmfao imagine getting mad at some one making an effort to learn.
@Aaron Sanders
Must be hard to have such a bad case of lil peen syndrome
@@aaronsanders6162 well said.. he thinks quoting an intellectual makes him clever
"I created it, but i do not understand it" - programmers
This is one of the highest quality programming tutorials I’ve seen. Even online courses from universities aren’t this well made made. Thank you for putting this together!
Thank you, that means a lot
@@sentdex no u
@sentdex much love man!!!
Universities typically include scientific papers to read to develop a deeper understanding of why certain things work the way they do. Nevertheless, this is by far the best series on developing fundamental skills in NN programming!
@@acidtears you can get all of that for free on the Internet.
We need more people like you. And you know what? You are making them.
Me: opens youtube.
RUclips: Sentdex- Neural network from scratch.
Me: Opens the link faster than the speed of
light.
The problem with being faster than light is that one can only live in darkness.
@@ndn5656 Best comment of the day award !
You just time traveled
@@ndn5656 that's not true if you are traveling at the speed of light . There still will be light reaching your eyeballs but everything will gonna slow down .
Well guessing from your profile pic, you indeed seem faster than light.
No university in the world beats this top quality of knowledge for free. You go man! You don't know how much your time and effort you put into this has been appreciated. Machine learning and neural networks are hard because of the intense logic behind it, even though I've been programming for 5+ years now. This really does the heavy lifting for many people. Thanks!
almost as if the university and or certs are just for validation
You’re a hero...edit: Update: I’ve been hopping on some friends zoom courses on AI for a pretty elite university(won’t say which one) and you absolutely SMOKE the professor with your ability to teach. I don’t know if people wrote you off when you were starting out but you have certainly proven yourself as one of the GOATS
agreed
I have a solid science background in Physics and mathematics and this is the first time that I came across a lecture which explains why, what , when and how deep learning is from the basics. I am not a fan of memorizing things that I cannot extend in the future. Sentdex, you are on point and these video series have broken the backbone of deep learning for me. Thank you
This is definitely one of the best programming tutorials I've ever watched. Instead of bombarding you with complex math, you take a very pragmatic path and expect minimal knowledge beforehand. This is probably the video definition of "you don't know what you're talking about until you can explain it to a five-year-old".
I'm just 30 seconds into this and paused to tell you how much I appreciate your work, I can't thank you enough or begin to describe how much your content have helped me to better myself as a developer, you're awesome, keep it up! Cheers from the fjords of Norway !
Really love how you’re teaching the mathematics behind deep learning... think this will lead to a lot of innovation
Hope so!
@@sentdexin been searching for a vid going into the math as I don’t understand it yet(still in high school) so I hope this is as good as the comments say
@@sentdex Bro this could've been the trigger for CHATGPT and other ai explosions in 2020+
@@randiaz95no they started long before.
I hope to be able to make a simple network that can identify colours in the future, nothing else. Thanks so much for the tutorial
Can I just say how thankful I am for this?
Most tutorials are either very basic and don’t tell much, overload you with all the math upfront, tell you to import tensorflow and not tell you why it works.
Thanks for this middle ground tutorial, something that is really needed.
this is what ive been waiting for my man , you made me a full stack developer now make me a machine learning developer
Glad I could help :D
"Now this is all very simple maths. But I'm gonna explain it anyway"
BLESS. I'm a uni dropout who installs safety signage for a living, so it is profoundly and deeply helpful when a tutorial doesn't assume that competent beginner programmers know any maths.
Can't thank you enough for all the stuff you put on. I'm actually working as an Dev for two years now and everything started with your Pygame tutorials. Thanks man!
Happy to do it!
- nice tools for visualizing the examples
- easy and complete explanations from scratch
- an instructor with a decent amount of experience
- low requirements to follow the series
- a nice yellow box with a question mark in the background
After running a simple model with the previous variables, the output told me this was going to be the best tutorial on neural networks ever made.
I come from 10 years in the future, we used this tutorial to rebuild society after the Corona Cataclysm!
LOL
Not LOL, i think It's True
Yes.
I come from the past, what's the Corona Cataclysm? Was beer involved?
I come from present....and i am sitting on my couch watching you two play past and future shenanigans
5 years ago, I started programming watching your videos, working a shit job. now I’m a data engineer looking to get into data science. super excited to watch through this series. thanks for what you do.
Perfect utilization of LockDown Period, Revise the things from the person who taught you in the past. Thanks, Sir. Love you 3000..!!
Extremely helpful, I've been finding it frustrating to "just use" certain functions in whatever someone else has already solved without a deep understanding as to why they came to that conclusion of what to use and why. Thank you for taking the time to start this series.
Huge respect for you to come up and make people understand everything in bits and pieces. Recently , I have started working on RNN and i feel it is a complex thing. I also believe there could not be any other tutor who can best explain it like the way you do. The animations are lovely. The coding part is explained in as simpler way as it can be done. Its my sincere request for you to make a video about RNN implementation . Love from India.
Oh man, this series of videos is going to be such a masterpiece.
the wait is finally over...
Except for ... the wait to part 2 :)
@@sentdex how often do you plan to upload?
@@sentdex did you just like my comment? that is the biggest cock block i have gotten in my life
@@sentdex how long should we expect to wait between parts? I'm excited
Most of the videos that I've find here use librarys or frameworks, but learning from scratch is the best option in my opinion since we can adapt better to other tools, so I'm really excited to see this series.
Totally agree!
and more fun
I wrote a Java library for deep ANNs (from scratch) that implemented vectorized forward propagation and backpropagation. It honestly was such a useful thing because I know so much more about neural networks now
Any links to that?
Your timing on this is fantastic. I've been dissatisfied with High-level super-abstracted ML, so I taught myself Calculus and I've been studying neuroscience. My goal is to design ML "circuits" inside of neural networks, and specialize their design in the same way our own neurons are arranged by speciality. I love all your other videos and can already see how this is going to help me bridge the gap between neuroscience and Python.
I already know I'm gonna love this series
le me, starting out in python a week ago: "Oh this looks like a cool video, bet I wont understand it."
me 6 minutes in: "Ok good he knows I don't understand any of it."
...but you will! :)
I like the graphics, reminds me 3Blue1Brown channel! What is the tool that you are using for this?
That's because.... it's built on top of manim ;), which is made by 3b1b. Learn more here: github.com/3b1b/manim
@@sentdex You should do a tutorial on Visualization using manim, that would be great!
@@vijayabhaskar-j Y E S S S S
@@vijayabhaskar-j Hi Vijay !, are you the same person who wrote the Flow from data frame in Keras ??
@@ramakanthrama8578 Lol, yeah. That was a long time ago.
You've become more interactive and it feels as you are connecting more to us than the previous videos. Obviously, no doubt about your teaching since beginning! You've saved me through a lot of Data Science assignments. Love from India ! Please visit India after the pandemic!
"Why would anybody would want to do this to themselves?!!"....lmao,loving this series already!!
This comment is three years from the future :D Finished watching the first video. I just knew the moment you said "from the scratch without any libraries..." this is the series I'm going to learn neural networks from. :D
Awesome 👍 perfect timing as well as we are all stuck at home
This looks simply awsome so far, soooo excited to see the series grow
Omg! The God Of Programming has listened to our requests! Neural Networks from scratch is here! 😍😍
It is a thing for over 3 months now and started as KickStarter campaign and people do already read draft of our book :D
Kudos!
i have been scouring the internet for MONTHS trying to find articles or videos or anything which breaks down the topic and coding structure for building a neural network and yours is the first one I've found. Most of the resources I've found only point to how to create an app using already pre-created code, which I didn't want. so, THANK YOU for your help. And the way in which you explain it so that even a complete numbskull like myself can understand it is absolutely heaven sent
🙇♂🙇♂🙇♂🙇♂
Head's up - If you are starting now, don't even bother. This video series is not complete. They abandoned it.
It's enough to get started, I still recommended it. The fundamentals haven't changed and the way he explains it is very easy to understand. Just for that alone I give it thumbs up
After watching this video I am super excited for the second part. You are among the few people (Andrew Ng, 3Blue 1Brown & Jason Brownlee) who have explained neural networks in such an easy way.
I literally got goosebumps when i saw the notification. Thanks man keep rocking!
I'm incredibly excited for this series. I had the same problem where I've exposed myself to neural networks in the past, but what I know is for very specific tasks. I'm the kind of person who needs to be able to even vaguely "visualize" in my head what's going on in the background to then be able to adjust it to my needs. It's especially important because I'm wanting to use machine learning methods for the lab I'm working in, but I keep hitting road blocks where I feel like I really don't know what I'm doing (which is probably spurred on by the fact that I have no formal education programming or computer science) or how to troubleshoot certain things when problems arise.
Thank you for making these tutorials, I look forward to the rest of the series
I don't have any formal education in programming or computer science either, so we're in the same boat :D
@@sentdex I wasn't aware of that; that's actually fairly encouraging to know. I wasn't exposed to anything programming until an internship about three years ago working in a pediatrics lab. This past year I was accepted into grad school studying bioinformatics; my education was in biology, but I sort of lost interest in it over time and focus more on the computer science side of things. Solving problems is a lot more interesting than reading scientific literature for me. The thing I haven't figured out how to do well is navigating packages. Like trying to go through them and understand how they work. I'm told I'll supposedly get better at it over time. We'll just have to see though.
Thank you so much for this! 🤠
You're so welcome!
Found this channel a week ago with the pytorch tutorial. A fair amount went over my head so can't wait for the rest of this series. keep up the great work!
Welcome aboard!
Can't thank you enough for the great content you are putting out ! I want you to know that you are making a huge difference in people's lives.
My pleasure!
Your ability to explain things simply is nothing short of amazing man.
essence of linear algebra is a pretty epic course here on youtube to clear up your matrix and vector fundamentals
Most tutors don’t have the patience to make stuffs simple but this right here just affirmed, everything should be made as simple as possible, but no simpler. Thank you
The way you structure your videos is so helpful, looking forward to more of your AI/ML content in python. Subbed!
this is such a hype! cant wait for the rest of the series
This has been something I have been waiting for in a long time. Keep it up my guy
Hey sentdex, I'm currently going through your "Neural Networks from Scratch in Python" playlist for the second time, and I wanted to thank you for this awesome resource. Even though some topics are still challenging, your explanations make these concepts so much easier to grasp. Really appreciate the effort you've put into breaking everything down.
"This shouldn't really be over your head"
Me, a high school student who has never coded before: *sweats profusely*
Same dude I just a saw this video in my recommendations and now I’m listening to alien languages.
>high school student
Well there's your problem
I’m in 9th and I code but I still didn’t understand much.
Tornado I’m in 9th and I understood it decently.
@@ifmondayhadaface9490 Do you code?
1 year back, when I was getting started with Data Science. I had a little knowledge on Python (That too was the basics) and a basic understanding of linear algebra. I didn't know how to write a simple function. I was just going through this video and hoping someday that I'd understand what is being written and said by this lecturer.
Now, I have a solid understanding on Python (not an expert, but I understand most of the code that's written and familiar with Object Oriented Programming. Also, I am still learning python) and I have a better conceptual understanding of linear algebra and matrix multiplications. I started learning machine learning recently. Believe me, I was just going through this video skipping most parts, I can actually understand what this lecturer is teaching. I don't know who wants to hear this, if you feel like you aren't making any progress. Don't just give up on your journey! Good things take time! All the best!
Thank you, sentdex. I have learnt a lot from you. Hopefully, I will be learning something new from this series as well.
This is exactly what I want. There are a lot of framework in this field. And each framework has its own advantage and disadvantage. I usually think that if I have to create new framework or have an idea, how I can do that. I would like to know the deeper level in this field.
Thank you very much for creating this series. I will watch every episode for sure.
Hope that you can deploy your own framework too. Hahaha
We're not planning to competitively release a framework, but, we ...could... ;)
YESS! I’ve done this with Daniel from the coding train last year so I won’t be writing along but will definitely follow along. Amazing quality for a tutorial about NNs.
Any idea of how often they’ll be coming out?
Weekly or sooner.
@@sentdex Great! Thanks for the content!
Thank you for making this series. I have always been hesitant to use the frameworks since they don’t show what’s happening behind the scenes. I just have one question. How often are you going to upload these video? (What is your upload schedule?)
No official schedule. Soon as they're ready. Weekly or a few a week is my target, but my main focus is on doing it right rather than adhering to any schedule.
Your course is helping a lot of students in developing countries like me (from India) to improve their Machine Learning skills . Thank You so much
Happy to do it!
this is the greatest thing to ever happen to mankind
For sure lol. I am following along with crystal programming language to learn it too.
I already work in python and I wanted to learn a new lang.
Also, thank you dude. I've been doing a separate little project building out a C# neural network with layers, network, loss functions and all that the last few days. But listening to someone else explaining things from scratch is a great help. That was a very quick 16 minutes too.
I really needed this specially in this lockdown! Thanks a lot (:
Love from India❤️
Happy to share!
Bro I absolutely love your channel. Bumped into it battling with generating tf records due to tensorflow installation for object detection. This is a phenomenal channel, will surely watch a video day from now on. Love from Zimbabwe.
Thank you sir, you are just great. I learned Django from you. I hope this series is as good as that one.
I’ve been trying to learn Python and ML. Thank you for this. As a visual learner, I really appreciate the walkthrough. It helps so much. Subscribed!
JESSE PINKMAN?!
Years ago you were the first person to be able to break through that barrier for me and taught me how to code and made it fun. Glad you're still making vids!
NOTE THIS BEFORE STARTING... although excellent by every measure, the series abruptly stops unfinished. This is a wonderful guided tour out of Hobbiton, but you will have to hike the rest of the way to Mordor alone. The companion book has 666 pages; this video series carries you through to only page 136.
Thank you so much for this! This is a field I recently took interest in and I never really knew where to start, so this series is perfect for me!
Great!
Great video! Loved how you broke down the computation of the loss function and pretty much all the explanations are really easy to follow.
If I make a sentient ai (GLaDOS), I'm blaming Sentdex when the feds ask me.
Quantum Nebula
Bro. Blame me instead. I wish to have such an honor.
@@gaffclant ikr! There's gotta be some legal stuff in Sentdex's course material that explicitly states: "Any and all product from my teaching is you or the product of you by law. Sentdex is thereby exempt from any legal prosecution by any independent, self-aware artificial intelligence you construct under Sentdex's instruction, (This includes marginally corrupt rouge AI's bent on a will to test, and compulsively ridicule test subjects.)"
@@Quantum_Nebula Sendtex™ is not responsible for all the paperclips
hmmm Citations in Court Count, Right"!"
I love the way that the little vein on your temple bulges out when you give the exact versions of the packages that you're using.
Seriously though, love your work. I've watched a ton of your content. You've helped my Python programming immensely.
“It really shouldn’t be flying over your head”
Me who doesn’t understand one bit of it: panik
I decided to create a neutral network for image recognition from scratch, for my A-level NEA and I have been trying to find a starting point for it for quite a while now; searching the internet, reading books and looking at past students NEAs. This has helped so much (relieved alot of stress too), definitely going to continue to follow this series for my NEA. Thanks so much!
Sentdex: "I am speed..."
Dude, I am only 7 minutes in and I know this is the most amazing course I have seen. Better than university!
But the ultimate question remains : What shall I do with the time I lost before finding out your channel ?
A friend just introduced me to you. After about 6 minutes , I am inspired. I use python (also self taught) but don't consider myself a great communicator. I believe I can learn to be more like you and then achieve more traction in my pursuits. Thank you for being you!
i really hope you mean FROM SCRATCH, do build normalization functions and optimizations yourself please
That's exactly what we do. We mean it when we say it :)
Quarantined inside my room. This is just what I needed!❤️
Why would anybody would want to do this to themselves?!!😂😂😂
Looking forward to this series! I plan to do some neural net from scratch tutorials myself :)
Wait is that 'not a flamethrower' over there?
Nah dude it's just something boring
Nailed it
Trust me i was waiting for this since one week. Little bored with cybersecurity and hacking, time to learn something new.
I'm in the same boat :) did pentesting for 10 years, but last three years into product support. but ready to learn something new
"I'm doing x in raw python" sounds like you're doing something you shouldn't
Grateful Harrison! Received my book parcel today in South Korea. Thanks for been so reliable and trusted. Your book is indeed a bomb!!!
wanted to say "first!" but entered a time warp where it appears to have taken me 21 hours to shift from my phone notification to this video.
I just bought the Ebook after watching 3 issues I had, which now I understand and fixed. Keep up the great work.
Hope you enjoy!
I typed in neural network python in youtube hoping to get a decent tutorial on how to learn to program NNs in Python. Instead, I orgamsed multiple times. Wow dude, one of those rare videos you stumble upon not knowing you hit a goldmine of learning content
Finally! Been waiting for this! Thank you so much for releasing this in the quarantine period.
This series is more valuable than pure gold
I am so happy you are explaining the basics of neural networks, I always hated just jumping into tensorflow and not knowing anything about whats going on under the hood. Thanks!!!!
This is better made than most paid online courses available. And the quality on those graphics is amazing !
Since I'm responsible for those animations - thank you! :)
Just ordered the hardcover book online, very excited to start my new journey!!
Just as i wanted to find some tutorials about this it turns out you started a new series about the topic! What a coincidence!
The little detail of ensuring to provide the versions of libraries used, loved it
I love the boring company flamethrower in the backround. So many great things in this video!
Love you sir,
please release the schedule for this series.
The only way for me to have a schedule and adhere to it is to have a buffer, which means you'd have to wait longer most likeyl for videos. I am going to try to do it weekly, but don't hold me to it, and maybe I can even go quicker. We'll see. Some videos will take longer than others to make...etc.
This is the kind of series I've been looking for for years!
Everybody seems to want to make something high-level and abstract, and nobody wants to make anything low-level, it's so infuriating.
I coded my own neural network library in python from scratch and it worked really well. Unfortunately it took about 2 hours to learn mnist dataset... but i now know how it works
OMG sentedex is even better!!! I started to learn python with him 5 years ago, and now I'm back to find even more great content! Thanks a lot!
FINALLY! I've haven't been able to find any good tutorials online on how to make AI, and a lot of the actually good ones are these paid programs. I've resorted to testing out my own things and creating my own AI without a teacher, but now I can finally optimize everything!