Neural Networks from Scratch - P.9 Introducing Optimization and derivatives
HTML-код
- Опубликовано: 1 авг 2021
- Introducing the challenge of optimization and the concepts of derivatives
NNFS series playlist: • Neural Networks from S...
Neural Networks from Scratch book: nnfs.io
Channel membership: / @sentdex
Discord: / discord
Reddit: / sentdex
Support the content: pythonprogramming.net/support...
Twitter: / sentdex
Instagram: / sentdex
Facebook: / pythonprogramming.net
Twitch: / sentdex
#deeplearning #machinelearning #nnfs
At 8:44 shouldn't it be y2-y1 instead of y1-y1?
Yes.
ty captain obvious
@@Kmarad_ Shame you didn't get to spot it first. Eh?
oh so that's why when i did the math it was coming out as zero
You guys in the reply section dont know what its to be a begginer to calculus
Top three unfinished works of all time:
3. The Silmarillion by J. R. R. Tolkien
2. Requiem by Mozart
1. "Neural Networks from Scratch" RUclips series by sentdex
lol I was here for the backpropogation video but :(
@@theoutlet9300 samesies
It seems he wants us to buy his almost 100 dollars book.
@@laboralmail9239 it's worth it
@@laboralmail9239 It's a complicated topic and he's doing a damn fine job of explaining in a way that even a neanderthal like you can understand, plus the ebook version is like $30...
"and I will see you in the next video ...." never was so disappointed :(
Sentdex be mocking us with love reactions :/
please do a part 10, this is the only series that I found on youtube that actually explains the concepts. and its done so well
forbidden tactics to sell the book 🤣🤣
@@TeuPreem libgen
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Yes, wee need part 10-100 or many more
@@noormohammedshikalgarIts not that, there's litterally only one, maybe two, more topic before you have everything you need.
Eventually I figured out the back propagation algorithm on my own, but it took forever
Is part 10 happening? This has been the best series I've been able to find for this sort of thing.
LoL it's been 1 year
@@rifkiardiansyah616 1.5 years actually. Really hope he would continue it someday
@@raccoon_05 add 2 more months to that
It's all a lie. He won't do it. It's just a strategy to force people buy his book. It's quite shameful what he has done.
@@laboralmail9239 shameful!!? dude whatever he has taught (for free) is certainly the best. You must respect that too.
please complete this series it is literally the best one i've ever seen on the internet
Oh my god I just discovered this series today and I was bummed the optimization videos weren’t out yet, and of course, what perfect timing for this video to come out an hour after I finished episode 8, thanks so much!
and now u have to wain an year for ep 10.
P.S. the series is great but I desperately need ep 10
@@ishaankapoor933 he briefed over how to do it, so i would recommend experimenting by yourself for the time being
@@ishaankapoor933 ebook is like 29$ not too bad
@@ishaankapoor933 😐😶 i also want he complete series
@@ishaankapoor933 We need ep. 10
edit: just noticed, someone already mentioned this, but maybe the explanation is appreciated by someone
Cool video, just a little inaccuracy:
You mentioned at 8:16: "this can only be done on a continuous curve, so no sharp corners"
I know this is an introduction but to clarify:
a continuous function can have sharp corners. A better description for a continuous function would be that it should be " connected", so no parts, where it suddenly jumps from 1 to 0 without a connection. Someone gave an intuition like "if you can draw it without lifting your pencil it's continuous. But there's a bit more to it.
what we want to look at are "differentiable function", so functions you can differentiate (or find a derivative).
Does not help with understanding, but these are the functions with no sharp corners.
All differentiable functions are continuous. So it is a stronger statement.
Examples:
f(x) = x^2 is differentiable and hence also continuous
f(x) = abs(x) is continuous, but no differentiable (sharp corner at 0, as the function describes a V shape)
f(x) = 1, if 0 < x < 1; else 0 is not continuous, hence also not differentiable (we have value jumps at x = 0 and x = 1)
This series is absolutely amazing. Your explanations are extremely thorough while still being engaging - something that even my favorite college professors have trouble with. I feel like I'm actually learning ML instead of just copying and pasting code. Can't wait for the next video!
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Thanks for making these. It must be a lot of hard work, we all appreciate it so much. People say that ‘reinventing the wheel is a waste of time’, but learning how the wheel works, what makes it tick, helps us to apply more advanced techniques. Thanks once again 🙏🏼
Love seeing your production value skyrocket over that last few years. Hard work paying off!
I am blown away this video came out so quickly! Thank you so much Sentdex! Your videos are the only reason I understand Neural Networks even slightly!
Thank you for this very helpful book,
I've finished going over it a couple of months ago and I feel a lot of my understanding of the subject since has been easier because of it.
Ahh we were just getting to the good part! This was one of the best tutorials I have ever seen. I hope you can continue the series!
I'm so glad to see this series continued. This is a gold mine of a lesson.
this aged like milk
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
I can't wait for the next video, I also can't wait for my hardcover of the book to be delivered so I can follow along! Love the work you're doing.
Awesome tutorial, sentdex! I love the animations and the "simpler than necessary" cases are very well thought out, help me a lot in deepening my understanding with deep learning. Must have been lots and lots of hours of work!
Start an amazing series based on your book. Leave out the conclusion of the problem. Well played..
Made me wanna get the book
Man this is so awesome, been at this for a week now and my head is splitting open from the amount of information to take in. You've made it much clearer thanks!
When is the next one ?
I've just finished watching all 9 of the videos currently on this playlist. I absolutely cannot wait for part 10. PLEASE CONTINUE.
A very gifted teacher and great content creator. Finally found a teacher who explained the concept so well, but it's a shame that he settled for being a salesman. Was about to recommend this series to my class, but how can I? Left at the most important part. Thanks though, you explained it very well. I hope you decide to complete the series for future learners.
This is the best tutorial series I've found on this subject anywhere. Followed upto this point with no issues. Hopefully you'll continue this someday :(
Thanks so much for the vids so far 👍
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Finally! Its what we all been waiting for!
This series has blown my mind so far! I went from 0-60 real quick, but I feel like I understand it! Can't wait for part 10!
never happening
This series is so amazing! You helped me a lot, thank you so much!
Amazing series! Thank you so much Sentdex for sharing this. I'd love to see the conclusion of the remaining concepts.
This is the best series I have watched on Neural networks. Hope you can upload the videos atleast once a month. Really looking forward for the complete set of videos for back propagation concepts.
A lot of effort put into making this series. Great content. Looking forward to watching the next vid
Turned on my notifications for this one. Great content as always. Keep it up man!
Amazing series, just got the book just because this series is so well put together! Gonna have a bed time story to read, keep up the videos!
Im waiting for part 10! Thanks for the tutorials! keep the grind!
@@aadarshkumar2257 pretty soon 2024
@@joekanaan2548 nah, I heard 2030 bro.
You have to code a NN up and use a prediction model to calculate when he'll release the other parts
Brooo the guy just gave up on the tutorials :(
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
hope to see this series brought back to life, I bought the book first and really like having the videos to confirm what I read.
this series is seriously a masterpiece. the visuals, information, and presentation are just incredible :) i cant wait to see how this series is used in the future
Thanks Spencer!
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Thank you for taking the time to continue making these
been following this series in typescript. I must say its a masterpiece. The fact that I was able to write a neuro network from scratch is incredible. Looking forward to follow up videos. And if there are non to follow this is job well done. Greetings from Tanzania
I am waiting for this series to be updated. It's really awesome!
The first time anyone has made a video there made me understand what's going on in neural networks. I literally can't wait for the following video.
Absolutely learned alot from this series, not just for my technical knowledge in neural networks but also my calculus subjects seems to be getting better. Knowing that the knowledge can be applied.
I´m really learning a lot with your videos. I´m absolutely going to buy your book as soon as I can. Greetings from Uruguay!
Sorry for putting this responsibility on you, but I am addicted to this series and you HAVE to continue with it. Simply love it, ordering a book, but please come back. :)
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Bhai please continue this series!! It's so bloody awesome.
Would love to see videos 10+, your explanations are excellent. Have been and will continue to come back here every few months to see if there is any new content💚
I cant wait for more of your videos...
This was really a helpful introduction to Neural networks.
Please continue this, its a great series.
Is there a part 10 everywhere? I can' t eat my launch without an nnfs episode
Not even sure if anybody will see this, but just wanted to say that your series has helped Neural Networks actually *click* in my head, I'd refer to this series as like the programmer's 3Blue1Brown.
I see a lot of comments here requesting another video and I myself would love another one but please don't rush it! Take your time and I'm sure everyone can agree that it will be excellent.
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Amazing series! Thanks for the good work :)
Definitely the hardest video so far! Took a couple rewatches to understand but worth it!
You are doing an incredible job here, and I can't wait to see the next video(s). Could you please let us know what is the time plan for releasing the next lessons? Thanks a lot!
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Amazing videos. Going to purchase your book. Are you planning on making more videos in this series? Thanks for everything!
Hey @sentdex, I understand if there is not enough interest in this topic to make it economically viable to continue producing this content for free on youtube.
However I'm sure a large portion of the 50,000 people who made it all the way to this video.
Would be happy to support further content production financially using a service like Patreon.
I will purchase the book to support anyways, but honestly, I think a ton of people would just rather pay for video content.
heck I'd pay $250 for a full video course of the content in the book
just got the e-book. For about $29, it will have exp(29) x returns..This is absolute classic in so many ways. It is also allowing me to hone my python skill. Glad i came across this..excellent stuff
Great work, I have been using the book to continue learning after I got to the end of this video. The youtube videos are certainly amazing but you can do so much with the book also. All depends on how fast or slow you go. So far, I think it is the best resource for the development of neural networks.
hello SIr, please complete this video series .. i learnt so much from these .. this is the best one over the internet
Man you cannot cut such an awesome content in the middle like that, pleeeease finish. I never asked you anything, please xP
Great series. I would love to see more.
Will there be parts 10 and more? Even if it's only for those of us who have purchased the book to support you?
The book is very good but these videos are superb for grasping the bigger concepts.
What other topics are left after this?
@@aurkom Back propagation, weight adjustment, bias adjustment, gradient descent, ...
@@ChaosContrl im right with you would love to see the rest of the idea before i set off on my own
@@ChaosContrl yeah, i would love to see some examples of optimized weight adjustments and also how to choose the better neuron architecture for each purpose
@@aurkom lmfao if you see his table of contents in the ebook, we would be on page 180 out of 667.. So quite a bit more
I love this series, I just got into neural networks and was wondering how to make them, this is helping so much!!! Are there going to be additional videos in this series?
6 hours taking notes, writing code, and trying to manipulate the shapes and functions with almost complete understanding, you are amazing!
Please, is this the last video or you will upload more soon?
Thank you.
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Dude. This is fucking awesome!!! We demand the rest of the series!!!
please release the next videos, have been loving this series
We need the rest :( I never would have started this series if I realized it wasn't finished :(
Just finished the series ,That's an amazing explanation .. eagerly waiting for the next one .. thanks man...
still not out :(
Dude! this was powerful! we want part 10 please !
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
My dude just attempted to get folks up to partial derivatives for the gradient in 5 minutes. Admirable.
Why leave us alone at the most important chapter of this book???? Backpropagation is very difficult to understand without its video.
Hey this was an amazing journey which ends absurdly I was really looking forward to understanding complete optimization no matter how simple
just binged the first 9 in an evening hopefully 10 comes out soon
Welcome back, bro!
I love this series, I understand so much more about neural networks now! Is there gonna be an episode 10?
guys i am extending this playlist on my channel, if you are intrested to learn further
ruclips.net/video/DOFA4dL-qAA/видео.html
Thank you for this video series! :)
Average workers: "All the calculus you learn in school won't be used in any way"
Me who is a programmer and following this series: "Hold my beer"
Also, you just made me way more interested in learning calculus than any of my calculus teachers ever did. I'm way more hyped to understand Logarithm and Natural Log than ever.
The y=mx+b equations, The non-linear formulas (x**2), I've learnt all of those.
You made me feel like my braincells is connecting(For real!)
Please finish this series. I can really use the knowledge you shared here for my upcoming Semester's Courses
please continue this great series
once again, i'm so glad I took further maths at A-level (i think it's like AP in america, but for all parts of maths), and I understand calculus well enough to know where this will be going and how it'll work
Maths is useless, nobody needs it, especially not programmers /s
@@maythesciencebewithyou Science is useless, nobody needs it, especially not You /s
I'm glad I bought the book. It was different from what I imagined but I am still glad I paid for a printed copy. Since my purchase I have continued to learn about Neural Networks in general as a problem in convex optimization using gradient descent. Gradient descent works via back propagation of numerical partial derivatives using "Automatic Differentiation" with Dual Numbers. Coding these things up from scratch in Python is a worthy goal.
so i guess i should not expect next videos anytime soon😅
Nope, must purchase the book. It is great
Following along very closely, in Java. Loving the book so far. Looking forward to the next video!
Makes me so happy to hear people are following along in other languages ☺
@@sentdex Follow along in cpp. cpp master race
I just bought the textbook! This is a great series and is just the kind of ground-up education I was looking for.
One thing I'm struggling with in the textbook is that, in preparation for backpropogation, you spend a ton of time on mathematical fundamentals. As someone who hasn't touched calculus in 15 years, it's a slog and the mathematical notation is hard to get used to. I get that it's a necessary evil, but I don't think the textbook does a great job of contextualizing the information. In this video, I followed why random and incremental permutation were too inefficient, but I don't really follow how the derivative is used here.
I think I lack the intuition for what it means that the network can be represented as a single equation (probably because I'm used to thinking algorithmically). So, while I know what a derivative is, I don't really understand what it means to take a derivative of a neural network.
I expect all that will come with time, BUT it's a momentum killer to have all this hard-to-digest theory in an unfamiliar notation without a clear sense of how it will help.
I need more of this. I just ordered the book this morning and I am fucking hyped ! :D
looking forward to next episode!
Loved the videos, really helped me understand Neural Networks deeper. I’m looking forward to buying the book but I also enjoy the videos that you do. Are you going to continue video support for this series or is the book my only way forward from here?
For everyone reading this and wondering when he will finish this series, he said he plans to finish it eventually. In meantime, I would recommend Samson Zhang's video on this. It doesn't have the best explanation, but can get you started.
Ah! About time, thank you
People who don’t understand calculus will have a hard time to grasp the information if you went to learn machine learning on your own. I suggest to learn some calculus, specifically differentiation, before trying to code neural networks. It’s a shame he didn’t finish the series because he wanted people to buy his book. I suggest to watch 3b1b’s essence of calculus and machine learning series. I know calculus but I’m new to machine learning so I have no idea if 3b1b’s machine learning series is good or not. All I know is that he is wonderful at explaining mathematics.
ty
Bought the book, would love to see more videos, would be happy to pay for a video course
Awesome Series!! Awesome Book!!!
Hi! Love the series, explanation, and visualization together really help me understand the concepts. Just wondering how long until the next video comes out and how many parts will there be in total?
Please continue the series !!!
At the beginning of this tutorial, you said you'd hope that this series would remain relevant for 10 years. At this rate, that 10 years will be up by the time you finish this series...
can't wait for next part
Excellent stuff guys :-)
Greatly created content top notch
I have the book (soft-cover), and I love it!. Greetings from Chile.
Thanks for supporting us! Hope you've enjoyed the content
Hey, when will the next videos of the series be? Loved the one so far. Wish there were more. Thank you for the explanations too.
thank you for this episode!!!!
Please Continue. Pls pls pls. The BEST Series. I have been waiting for 1 year. Please
Please make more videos in this series!
Finish this series!
its heart breaking we don't get more of these awesome videos :(
:( :( :(
So interesting. Just bought the book.