We definitely need another online course from Andrew Ng. His course on Coursera is the start of many many people into machine learning. I hope that he at least update his course.
you're right, his course was like the initial kick for me in this amazing world of machine learning, could you imagine one with deep learning, gpu optimization and IoT?
I think he is the starting point of everybody of our generation, studying Machine Learning. The books of Tom Michell or Christopher Bishop were of no help to me atleast. :P
Oh man! I had forgotten just how good Andrew was, but after watching this video I've been reminded thoroughly of that fact. It's been so many years since I took his intro class on coursera, but boy! He still got it!
Andrew Ng is such an inspiring mentor. At the same time, I find it incredible that he gives practical and down-to-earth suggestions -- such as the ones he gave out toward the end of this talk. Only a sincere man who knows his field deep is able to do this.
This section is great: Importance of understanding bias/variance, especially with train/train-dev/dev/test datasets at 49:48 including Ng's workflow for how to address various issues.
Andrew Ng looks so much like Andrew Yang. Soft voice, the ability to convey in simple terms to masses, openness. Awesome Guy Thank you for the upload :)
Respect for Andrew Ng... so much knowledge made available free of cost to all of us. Done a tremendous job to bridge social inequality to make knowledge (almost) equally available to the students and professionals alike.
What a great man show (y)... I love this voice .. I have already gone through various video of Mr .Ng during machine learning course.. his voice inspired me do to lot in AI...
I can say humbly, that if we even understand 10% what he is saying we will be able to open a startup that is going to be successful or we can develop a moderately complex deep network from scratch. One of the top figure in DL and AI who means business.
I missed how the use of end-to-end DL for speech recognition 'proves' that phonemes don't exist. Maybe it just means the DL algorithm had an internal representation of something like phonemes?
READER DIGEST: Prof. Andrew Ng, (a psychologist, prof. of Stanford CS Dept & Chief Sci. of Baidu), gave an excellent and sequential review in Sep 27 2016 in the traditional marker & white board fashion about "Nuts & Bolts of Applying Deep Learning". He has summarized excellently three saturation curves for a small, a medium, and a big architectures of ANN's respectively upward. Furthermore he has summarized current algorithms in four buckets: traditional deep learning (DL), Reinforcement learning, Convolution NN for images, and the unsupervised DL, ICA, etc. He goes on to comment on the phonemes for speech processing (NB: He has dedicate RUclips on speech) and other applications of machine learning. His gave excellent error analysis, e.g. from human 2%, data 5%, bias %, multiple variances, etc at the level of training set, or develop set over fitting or under fitting are quite insightful,before you blame the machine learning. Also, He recommend to build a unified data warehouse is a necessary preparation for novice, He went on to address the complicate exemplar, e.g. 30K hours training set and 10 hours development (for generalization) and test This part of experience could perhaps augment DL error analyses with what a NIH called Double Blind with Negative Control (DBNN)statistical analysis adopted in medical community . While Internet Coursera can be stopped and re-run until one fully understood; but one can not do that in a real time lecture.
The title of this video is partially misleading. It's *mostly* a general machine learning methodology lecture. Still, it was a great inspiring lecture. Thanks for posting.
Very nice, neat and clear talk. I always like his lectures. One thing that I wonder is that how one should initialize the number of hidden layers and number of hidden units of a DNN model when we start training. On the internet, there are many notions but I did not find any standard method.
I didn't understand at 27:23, if the training error is high then it's mean we are having high bias in training error. Please explain this point a little bit. Thanks
Machine learning looks promising. Still pretty narrow AI but it will be interesting to see if general machine learning is invented soon, maybe not in 2020 but who knows. Exponential progress!
Very nice practical lecture. About his rule of thumb though, I don't think our doctors decide on cancer image in 1 second. Is he underpromising, over delivering?
One thing that Deep Learning has failed at is transcribing Andrew's lectures. It's not very clear for non-native speakers and the volume does not help at all. Understanding the video took me like 8hrs
Yup! The main innovation of DL is the ability to chain together and jointly-optimize computational steps. These are called layers. So, making a bigger model means chaining more layers together and, thus, doing more (deeper) computations. On slide 29 in this link, www.slideshare.net/iljakuzovkin/paper-overview-deep-residual-learning-for-image-recognition, you can see a side-by-side visualization of different models built for the same data. The one on the left from has 8 layers. And the one on the right has 152. It's not always the case, but the larger, bigger, deeper model performs much better.
Thanks, Wayne! This makes sense. So when you mention more layers -- this does not mean new architecture? These are Andrew's suggestions in case of high training error: 1. Bigger model. 2. Train longer. 3. New model architecture.
Imagine you have three non-colinear points on a graph. You would be able to achieve perfect fit with a 2nd-degree polynomial. The training error would be zero, but the model would probably not generalize well. If I gave you a thousand additional data points, the parabola would not be able to pass through the data points, but would probably generalize better.
At the begening of the video, Ng Andrew, goes through a couple of steps to attempt to separate him from the lesson. He then Seperates your outgoing attention , theres lots of sound stuff going on , intent of forced listening to gain access to . calculating (setting the tone) what volume is, where action to volume is. multiple sub division. or a spliced wire to be put back together (figured accordingly)TGB He is using the bilingual trick here though, I mean what can you say/whats to say, if you have it, your made not to use it, its going to surface in some platform. Lets make it good Andrew (chineese (Asian)pie sign TGB) I'm gonna put it on a grilled cheese sandwhichTGB But he knows its there, uses it uniformly in a specific place.
Great presentation. Thanks! One suggestion: Use slides--PowerPoint, Prezi, etc. That doesn't preclude writing on the whiteboard over the slides when needed. Stanford, right?
Even if that's so, one can learn to stretch beyond one's limitations and biases for the benefit of more effectively imparting information to one's audience. The material and knowledge is captivating, but it could be delivered more effectively.
I've been going to math conferences for the last 5 years. Every single power point presentation has been bad, no matter how much effort people put in it.
I was there. My impression was that he 'winged' it. But Andrew Ng can get away with that. You have to understand he has a young kid, he's a prof, heads up Baidu's AI lab. He doesn't have any time to do slides. Presentation was still great, even if his writing is illegible.
Look, I enjoyed and learned from Ng's presentation. His knowledge and perspective are very impressive. I guess I should attend a math conference, Roy, so that I could witness such presentations first-hand. In my experience, though, most science and math lecturers just don't want to take the time (or really know how) to communicate effectively in writing (and more often than not, women make the best math lecturers, but that's another story). So many math, physics and engineering books are so often incomprehensible as reading rather than reference material. There are notable exceptions, of course, which demonstrate that more effective communication is possible. To that I would point to Eli Maor (especially e: The Story of a Number) or more to the point, Richard Feynman's physics lectures. Because I wasn't fortunate enough to see Feynman lecture, I'm left wondering if he redrew on the board each of his diagrams for each lecture. It seems like that would have been a waste of time, but it doesn't have to be that way. Andrew Ng, was basically copying from his hand-written notes onto a white board, Jim, rather than thinking aloud. (Unfortunately, there doesn't seem to be an option in this window for me to drop a sample whiteboard element from the lecture for illustration.) However, it would be a simple matter to turn the notes into slides by first having typed them rather than having written them by hand (although at times, even for simple lectures, I use all three steps). Of course, it is spontaneity and the absence of forethought that makes whiteboarding effective with an audience. The beauty of slides, of course, is that they can be reused as needed. Say one has a 100-slide lecture, and a 15-slide segment (especially one that defines terms, etc.) might be ideal for another audience, even with some modification. BTW, I recall little in the Ng lecture regarding complex mathematical formulae that would have made slide-making a challenge. And there's nothing precluding augmenting a lecture by writing on a whiteboard with prepared slides. Finally, I should mention collaboration with someone who could help. Stephen Wolfram wrote and self-published an interesting book a few years ago (A New Kind of Science, 2001). I'm not acquainted with him in any way and I occasionally demonstrate the efficacy of Wolframalpha.com, but it would have been a much better book -- and maybe 50 or so pages shorter -- had I (or someone) edited it. People do collaborate, but more effective collaboration is possible. Imagine how much more effective Ng could be if using good slides. Not only could he expand his audience, shorten his lecture (or include more material) and save himself later work, but he could spend more time making effective eye contact with his audience. Etc. (Yes, I'm a little garrulous.)
He’s truly energetic! Campaigning during day time and teaching programming at nights. Respect!
pro tip : watch movies on kaldroStream. I've been using it for watching loads of movies lately.
@Keith Cayden Yup, been using kaldroStream for since november myself :)
@Keith Cayden Definitely, been watching on KaldroStream for since december myself :D
@Keith Cayden definitely, I have been using kaldroStream for since november myself =)
I can't tell if this is a joke or you're confused between Andrew Yang and Andrew Ng XD
Such an amazing talk, and he kept it so simple. You cannot help but admire this guy.
We definitely need another online course from Andrew Ng. His course on Coursera is the start of many many people into machine learning. I hope that he at least update his course.
Andrew's machine learning on Coursera is one of the best courses, online or in classroom, I've ever taken.
you're right, his course was like the initial kick for me in this amazing world of machine learning, could you imagine one with deep learning, gpu optimization and IoT?
this is exactly what I have been thinking about today
The world needs a deep learning course taught by him.
I am doing the ML course right now. Really loving it.
HE has taught me so much on Machine Learning. We all owe him :)
I think he is the starting point of everybody of our generation, studying Machine Learning. The books of Tom Michell or Christopher Bishop were of no help to me atleast. :P
Oh man! I had forgotten just how good Andrew was, but after watching this video I've been reminded thoroughly of that fact. It's been so many years since I took his intro class on coursera, but boy! He still got it!
Andrew Ng is such an inspiring mentor. At the same time, I find it incredible that he gives practical and down-to-earth suggestions -- such as the ones he gave out toward the end of this talk. Only a sincere man who knows his field deep is able to do this.
Hereby I declare myself as the biggest fan of Ng. He has this amazing capability of simplifying the most complex equations. Hats off to you sir. !!
this man is pure genius..so well spoken ..i hung on to every word and was blown away ! fan boy now ..more Ng classes please :D
finding your channel back in the day changed my life. thanks Lex.
Andrew Ng is the best at teaching us machine learning , deep learning all the stuff. GURU
Soft voice, the ability to convey in simple terms to masses, openness. Awesome Guy
He seriously needs to create a DL Coursera course. PLZZZZ. You are the best professor I have ever had. We all hope you create a new course. Thank you.
He just did. It's even better than his original Machine Learning course. deeplearning.ai
He did. Part 3 of the deeplearning.ai course is exactly this talk.
@@sksqhubham is this open source ?
This section is great: Importance of understanding bias/variance, especially with train/train-dev/dev/test datasets at 49:48 including Ng's workflow for how to address various issues.
Andrew Ng looks so much like Andrew Yang.
Soft voice, the ability to convey in simple terms to masses, openness. Awesome Guy
Thank you for the upload :)
Mr. Andrew Ng inspired me to get into Machine Learning. He now inspired me to get into Deep Learning. Simple!!
Respect for Andrew Ng... so much knowledge made available free of cost to all of us. Done a tremendous job to bridge social inequality to make knowledge (almost) equally available to the students and professionals alike.
understanding & tuning of bias variance between stages is my big takeway from this (not to detract from any part...) this was an incredible lecture.
Such an inspirational and humble speaker. Thank you so much for uploading this.
Andrew Ng looks so much like Andrew Yang.
I was expecting this kind of comment lol
He is basically Andrew Yang without the "Ya" 😂
not really just both asian lol
I am curious about what were the genes that make them very influential individuals.
They are actually 2nd cousins
Great great lecture. When ever i see any video of Andrew sir I get very excited for ML.
Andrew Ng is my ML sensei
Such an amazing video! Made me finally clear about bias/variance and also higly motivating.
What a great man show (y)... I love this voice .. I have already gone through various video of Mr .Ng during machine learning course.. his voice inspired me do to lot in AI...
We are all in debt to Andrew Ng
I can say humbly, that if we even understand 10% what he is saying we will be able to open a startup that is going to be successful or we can develop a moderately complex deep network from scratch. One of the top figure in DL and AI who means business.
4 years later this content got through me- deep learning is still not good enough :)
Your content is great Lex.
Very motivating statements in the last few minutes. Thanks a lot for uploading.
who dislikes this video !! this man is literally an awesome person !!
Ingenious, very comprehensive grasp of the subject
Very amazing talk.. Words are simple but profound. Provide us the methodology to understand and learn to understand machine learning.
Amazing talk! He is a researcher one can look up to! ❤
I missed how the use of end-to-end DL for speech recognition 'proves' that phonemes don't exist. Maybe it just means the DL algorithm had an internal representation of something like phonemes?
READER DIGEST: Prof. Andrew Ng, (a psychologist, prof. of Stanford CS Dept & Chief Sci. of Baidu), gave an excellent and sequential review in Sep 27 2016 in the traditional marker & white board fashion about "Nuts & Bolts of Applying Deep Learning".
He has summarized excellently three saturation curves for a small, a medium, and a big architectures of ANN's respectively upward.
Furthermore he has summarized current algorithms in four buckets: traditional deep learning (DL), Reinforcement learning, Convolution NN for images, and the unsupervised DL, ICA, etc.
He goes on to comment on the phonemes for speech processing (NB: He has dedicate RUclips on speech) and other applications of machine learning.
His gave excellent error analysis, e.g. from human 2%, data 5%, bias %, multiple variances, etc at the level of training set, or develop set over fitting or under fitting are quite insightful,before you blame the machine learning.
Also, He recommend to build a unified data warehouse is a necessary preparation for novice, He went on to address the complicate exemplar, e.g. 30K hours training set and 10 hours development (for generalization) and test
This part of experience could perhaps augment DL error analyses with what a NIH called Double Blind with Negative Control (DBNN)statistical analysis adopted in medical community .
While Internet Coursera can be stopped and re-run until one fully understood; but one can not do that in a real time lecture.
Thans for sharing. This talk is full of insights and best practices. But its word is very simple and concise.
Thanks a lot for the upload, I learned many good practices on working with DL.
The great Andrew Ng.
The title of this video is partially misleading. It's *mostly* a general machine learning methodology lecture. Still, it was a great inspiring lecture. Thanks for posting.
I'm going to start training the neural networks in my brain on machine learning weekend after weekend for 52 weeks from today on.
Thanks for posting - excellent materials for all the speakers - will attend next year !!
incredible useful and so amazing,one of the best speech I ever heard...
15:06 the fact that phonemes n audio to transript is harder than generation of new images
Although I am a Bernie guy, I have much respect for Andrew nYang ;)
SO many great people working in deep learning. And so many great youtube videos:D
Very nice, neat and clear talk. I always like his lectures. One thing that I wonder is that how one should initialize the number of hidden layers and number of hidden units of a DNN model when we start training. On the internet, there are many notions but I did not find any standard method.
I didn't understand at 27:23, if the training error is high then it's mean we are having high bias in training error. Please explain this point a little bit. Thanks
太精彩了，andrew老师讲的很有启发性 赞
I didnt know andrew yang was this involved in AI
Hunter Newberry this is a different Andrew
@@ArturHolanda91 I know lol
Tks for sharing the videos. These talks are amazing.
great video from Andrew Ng. I particularly liked the closing bit @ 1:10 mins on the advice on how to get good at ML.
In the middle 1st row, in red shirt (with laptop) is Andrej Karpathy.
Thank you Lex!
Thanks for uploading, but I had to don some headphones to make out what he was saying.
What a super dude. Thanks Andrew.
Excellent talk, highly recommend !
Great Teaching and Thanks for Professor Ng
How to become a great ML researcher: 1:12:31
Machine learning looks promising. Still pretty narrow AI but it will be interesting to see if general machine learning is invented soon, maybe not in 2020 but who knows. Exponential progress!
Looking forward to his next textbook.
Very nice practical lecture. About his rule of thumb though, I don't think our doctors decide on cancer image in 1 second. Is he underpromising, over delivering?
Ty for sharing!! Andrew is great man.
If you are here for some advice on how to build a ML/DL career, start from here 1:10:25
I
Thank you for the upload :)
Andre Yang going brazy..MATH
Amazing talk, but what if the test set is unlabelled will it be still be useful to use some part if it as validation set?
One thing that Deep Learning has failed at is transcribing Andrew's lectures. It's not very clear for non-native speakers and the volume does not help at all.
Understanding the video took me like 8hrs
Also I hate that the volume is super low, maybe needs a GAN to improve it's amplitude 💀💀
God damn, his hand moves so fast when he's writing on the board
Andrew mentions "Bigger model". What does it mean? Can a model be made bigger with the same amount of training data?
Yup! The main innovation of DL is the ability to chain together and jointly-optimize computational steps. These are called layers. So, making a bigger model means chaining more layers together and, thus, doing more (deeper) computations.
On slide 29 in this link, www.slideshare.net/iljakuzovkin/paper-overview-deep-residual-learning-for-image-recognition, you can see a side-by-side visualization of different models built for the same data. The one on the left from has 8 layers. And the one on the right has 152. It's not always the case, but the larger, bigger, deeper model performs much better.
Thanks, Wayne! This makes sense. So when you mention more layers -- this does not mean new architecture? These are Andrew's suggestions in case of high training error: 1. Bigger model. 2. Train longer. 3. New model architecture.
very great talk.
Thanks for sharing this :)
Thanks!
great talk indeed!
Love him
Thanks for the great lecture :)
Not to promote stereotypes but... Andrew Yang ? Is that you
God damn, Andrew Yang's a math teacher too!??
that was a pretty good talk
This guy is a genius.
How can I attend these talks in person ?
Love this❤️
brilliant
I thought that getting more data for a high variance model doesn't fix much?
Didn't he himself say that in his Coursera course?
Imagine you have three non-colinear points on a graph. You would be able to achieve perfect fit with a 2nd-degree polynomial. The training error would be zero, but the model would probably not generalize well. If I gave you a thousand additional data points, the parabola would not be able to pass through the data points, but would probably generalize better.
is that pieter abbeel the front row wearing the blue t-shirt?!
Why this man keep mentioning Baidu? What's the relationship?
At the begening of the video, Ng Andrew, goes through a couple of steps to attempt to separate him from the lesson. He then Seperates your outgoing attention , theres lots of sound stuff going on , intent of forced listening to gain access to . calculating (setting the tone) what volume is, where action to volume is. multiple sub division. or a spliced wire to be put back together (figured accordingly)TGB He is using the bilingual trick here though, I mean what can you say/whats to say, if you have it, your made not to use it, its going to surface in some platform. Lets make it good Andrew (chineese (Asian)pie sign TGB) I'm gonna put it on a grilled cheese sandwhichTGB
But he knows its there, uses it uniformly in a specific place.
Is the knowledge shared here still relevant after 6 years?
yes it is
Great talk! He should make a NN that writes notes on the board for him though ;)
The human error rate on rating this video as of right now is 10 / (1300 + 10) = 0.7%.
Andrew Y. Ng needs to log out of My Cellphone and My Google Account !
Topnotch.
Gilbert Strang of Deeplearning 👌
AdHoc, not very impressed. I was hoping for a proof on say optimizing design, not some words of advise. Meh..
Bravo!
How to generate subtitles
Damn, I thought the title was "Nuts and Bots!"
,Er der noget der kan lære mig at bruge internettet?
Sweet
2:10 Hi 👋
Great presentation. Thanks! One suggestion: Use slides--PowerPoint, Prezi, etc. That doesn't preclude writing on the whiteboard over the slides when needed. Stanford, right?
He prefers using whiteboard instead of ppts in stanford
Even if that's so, one can learn to stretch beyond one's limitations and biases for the benefit of more effectively imparting information to one's audience. The material and knowledge is captivating, but it could be delivered more effectively.
I've been going to math conferences for the last 5 years. Every single power point presentation has been bad, no matter how much effort people put in it.
I was there. My impression was that he 'winged' it. But Andrew Ng can get away with that. You have to understand he has a young kid, he's a prof, heads up Baidu's AI lab. He doesn't have any time to do slides. Presentation was still great, even if his writing is illegible.
Look, I enjoyed and learned from Ng's presentation. His knowledge and perspective are very impressive. I guess I should attend a math conference, Roy, so that I could witness such presentations first-hand. In my experience, though, most science and math lecturers just don't want to take the time (or really know how) to communicate effectively in writing (and more often than not, women make the best math lecturers, but that's another story).
So many math, physics and engineering books are so often incomprehensible as reading rather than reference material. There are notable exceptions, of course, which demonstrate that more effective communication is possible. To that I would point to Eli Maor (especially e: The Story of a Number) or more to the point, Richard Feynman's physics lectures. Because I wasn't fortunate enough to see Feynman lecture, I'm left wondering if he redrew on the board each of his diagrams for each lecture. It seems like that would have been a waste of time, but it doesn't have to be that way.
Andrew Ng, was basically copying from his hand-written notes onto a white board, Jim, rather than thinking aloud. (Unfortunately, there doesn't seem to be an option in this window for me to drop a sample whiteboard element from the lecture for illustration.) However, it would be a simple matter to turn the notes into slides by first having typed them rather than having written them by hand (although at times, even for simple lectures, I use all three steps). Of course, it is spontaneity and the absence of forethought that makes whiteboarding effective with an audience.
The beauty of slides, of course, is that they can be reused as needed. Say one has a 100-slide lecture, and a 15-slide segment (especially one that defines terms, etc.) might be ideal for another audience, even with some modification. BTW, I recall little in the Ng lecture regarding complex mathematical formulae that would have made slide-making a challenge. And there's nothing precluding augmenting a lecture by writing on a whiteboard with prepared slides.
Finally, I should mention collaboration with someone who could help. Stephen Wolfram wrote and self-published an interesting book a few years ago (A New Kind of Science, 2001). I'm not acquainted with him in any way and I occasionally demonstrate the efficacy of Wolframalpha.com, but it would have been a much better book -- and maybe 50 or so pages shorter -- had I (or someone) edited it. People do collaborate, but more effective collaboration is possible.
Imagine how much more effective Ng could be if using good slides. Not only could he expand his audience, shorten his lecture (or include more material) and save himself later work, but he could spend more time making effective eye contact with his audience. Etc. (Yes, I'm a little garrulous.)
"Humans are pretty good at computer vision" @54:30
lol