The moment we stopped understanding AI [AlexNet]

Поделиться
HTML-код
  • Опубликовано: 21 дек 2024

Комментарии • 1,6 тыс.

  • @WelchLabsVideo
    @WelchLabsVideo  5 месяцев назад +156

    Thanks to KiwiCo for sponsoring today's video! Go to www.kiwico.com/welchlabs and use code WELCHLABS for 50% off your first month of monthly lines and/or for 20% off your first Panda Crate.

    • @samuelspace101
      @samuelspace101 5 месяцев назад +3

      AI today is growing exponentially, just curios, do you think we will ever hit a peak where innovation on AI will start to flat out, or hit a wall, and if so where and when do you think AI will hit its peak.
      You kind of skimmed over this in the end, i just wanted a bit of a longer explanation.

    • @KWifler
      @KWifler 5 месяцев назад

      Oops, I farted 4o

    • @michaelmangion6187
      @michaelmangion6187 5 месяцев назад +1

      Was really keen to sign up for a crate for my daughter, but after 10 minutes of glitches on their system I just gave up. Not your fault of course, but you might want to let them know that their system is pants.

    • @jeffg4686
      @jeffg4686 5 месяцев назад +1

      could have done it with ASIC a long time ago.
      Just living out THEIR best life possible first...

    • @ryvyr
      @ryvyr 5 месяцев назад +1

      These days non-adsense being placed some ways into video, rather than with mutual consideration at very front/back/both where plenty people and myself would still watch, is instant skip/cliff off - though I wish success

  • @EdgarVerona
    @EdgarVerona 5 месяцев назад +8717

    30 years ago, I used to work with an older guy who retired from IBM. I was barely out of high school, and he used to tell me that neural networks were going to change the world once people figured out how to train them properly. He didn't live to see his dream become reality unfortunately, but he was totally right.

    • @spartaleonidas540
      @spartaleonidas540 5 месяцев назад +255

      Same except the guy was at Toronto and his name was Hinton

    • @EdgarVerona
      @EdgarVerona 5 месяцев назад +412

      @@spartaleonidas540 guy I knew was named David Modlin. I wonder how many folks who had their prime years in the 60's and 70's saw this coming? I wish they had been able to see it. I suppose some of them might have lived to see it. Crazy to think about.

    • @squamish4244
      @squamish4244 5 месяцев назад +173

      @@EdgarVerona Hinton's most important contributions came from the 80s onwards, but he has lived to see it, for one. He was working on neural nets in the 70s as a postdoc. It's all happened well within a human lifetime, is the crazy part.

    • @EdgarVerona
      @EdgarVerona 5 месяцев назад +84

      @@squamish4244 Nice, that is very cool. Glad to hear he's still kicking! The guy I knew helped create handwriting recognition software in the 70's. It is crazy to think that someone could see basically the dawn of modern computing and also its progression to this crazy time we're in.

    • @squamish4244
      @squamish4244 5 месяцев назад +54

      @@EdgarVerona Ray Kurzweil did too, but he's obsessed with mind-uploading, cryonics and resurrecting digital copies of his father etc. which is distracting, and he has trouble with being challenged on the practical implications of his predictions. He was right about the computing revolution but he's also a very strange dude. Hinton was running circles around him recently in a debate when both of them were onstage.

  • @somnvm37
    @somnvm37 5 месяцев назад +3271

    "one way to think about this vector, is as a point in 4096 dimentional space"
    give me a minute, I now gotta visualise a 4096 dimentional space in my head.

    • @autohmae
      @autohmae 5 месяцев назад +139

      Are you done yet ? 🙂

    • @Muhammed_English314
      @Muhammed_English314 5 месяцев назад +183

      "One way to think about a point in 4096 dimensional space, is as a vector"

    • @adamrak7560
      @adamrak7560 5 месяцев назад +132

      high dimensional spaces are crazy. A hypercube with the sides size=2, would have absolutely enormous surface and volume in 4096 dimension.
      size = 1, volume: 1
      size = 1.01, volume, approx 501587856585103488.

    • @RandomGeometryDashStuff
      @RandomGeometryDashStuff 5 месяцев назад +34

      @@adamrak7560what does volume mean for non-3d thing?

    • @thenonsequitur
      @thenonsequitur 5 месяцев назад +62

      Easy, image a 3-dimensional space and pretend it's 4096 dimensions.
      I mean, that's basically what the visualizations in the video are doing.

  • @JustSayin24
    @JustSayin24 5 месяцев назад +2401

    That real-time kernel activation map was life-changing.
    If, whilst editing these videos, you've ever questioned if the vast amounts of effort are worth what amounts to a brief, 10s clip, just know that it's these moments which have stuck with me. Easy sub

    • @prabhatp654
      @prabhatp654 5 месяцев назад +32

      Ikr, shows the hard work of this guy and that is something I respect.

    • @JoseJimeniz
      @JoseJimeniz 5 месяцев назад +25

      I wanted to say this too. You actually did it, you make that animation. That is an amazing thing you've done, you've really added to the sum of human knowledge.
      The amount of effort must have been phenomenal. Really: thank you. Nobody else has done this. I know the effort of huge, but I'd love more even on just Alexnet. Animations on creating the node activation image generation.
      I'd love one of Resnet

    • @amarissimus29
      @amarissimus29 5 месяцев назад +2

      The irony being, of course, that the script popped out of ChatGPT in about three seconds, editing by submagic slightly more, and images by stable diffusion in much less. But I agree, those few moments are worth it.

    • @kapsi
      @kapsi 5 месяцев назад +3

      @@JoseJimeniz While I'm sure it took a lot of work, someone else already did most of the work for the Activation Atlas.

    • @marklorinczy4412
      @marklorinczy4412 5 месяцев назад +3

      Same, this was truly eye opening

  • @drhxa
    @drhxa 5 месяцев назад +1653

    I've been in the field for 10 years and never had anyone describe this so clearly and visually. Brilliant, thank you!

    • @TheStickofWar
      @TheStickofWar 5 месяцев назад +9

      same here (9 years)

    • @TheRealMcNuggs
      @TheRealMcNuggs 5 месяцев назад +8

      Would you say it is still worth it going into the field (studying AI) even after progress is made so incredibly fast nowadays that after the maybe 3-4 years of studying everything could have already changed again?

    • @drhxa
      @drhxa 5 месяцев назад +12

      @@TheRealMcNuggs I say, if you love it (or have a strong interest) then absolutely! It's been changing quickly since I started, but the underlying fundamentals stay the same 👍

    • @GRIM787
      @GRIM787 5 месяцев назад +14

      3blue1brown made a whole gen ai series which goes much deeper and visualises things better, I do recommend to have a look, really interesting stuff

    • @ineedpills
      @ineedpills 5 месяцев назад +1

      Im still confused 😭

  • @samuelspace101
    @samuelspace101 5 месяцев назад +2199

    Most people think AI is a brand new technology, while in reality there have been studies on Computer Neural Networks all the way back in the 1940s, that's insane.

    • @louis-dieudonne5941
      @louis-dieudonne5941 5 месяцев назад +253

      But the real issue is that only now has computing power become strong enough to support everything, allowing research ideas to be realized into reality, and truly transforming these ideas into technologies with such astonishing effects.

    • @samuelspace101
      @samuelspace101 5 месяцев назад +150

      @@louis-dieudonne5941 makes you think, what are we studying now that will only be possible years in the future because of the lack of resources.

    • @empathogen75
      @empathogen75 5 месяцев назад +43

      It’s new in the sense that neural networks are relatively inexpensive and for the first time broadly applicable to a wide range of tasks.

    • @davidaugustofc2574
      @davidaugustofc2574 5 месяцев назад +21

      @@empathogen75 Its just a popularity phase, RUclips paid for itself when it was rapidly gaining users, we'll have Adobe level subscriptions in the future.

    • @gljames24
      @gljames24 5 месяцев назад +18

      ​@@louis-dieudonne5941Not just hardware, but data as well.

  • @khanghoutan4706
    @khanghoutan4706 5 месяцев назад +484

    Fun fact, the kernels used in vision models work pretty much the same way as how our retinas perceive objects. In a similar structure, our eyes have cells that perceive edges at certain angles, then as shapes, then as objects in increasing abstraction.

    • @PallasTurrets
      @PallasTurrets 5 месяцев назад +105

      only edge detection occurs in the retina, anything more complex than that happens higher up in the various visual areas of the brain

    • @pyropulseIXXI
      @pyropulseIXXI 5 месяцев назад

      They don’t at all; you are confusing a low level explanation for how our eyes really work
      Humans don’t work like the kernel at all; biology is far more efficiency and works in ways we don’t even understand yet

    • @khanghoutan4706
      @khanghoutan4706 5 месяцев назад +60

      @@PallasTurrets Whoops I forgot to mention but yeah, more complex stuff still occurs in the brain. Thanks for correcting me

    • @ВалентинТ-х6ц
      @ВалентинТ-х6ц 4 месяца назад +7

      Their similarity is less than between an airplane and a bird.

    • @schok51
      @schok51 3 месяца назад +2

      ​@@ВалентинТ-х6цmeaning?
      Do you have a more detailed understanding of human vision to share to compare and contrast ?

  • @kellymoses8566
    @kellymoses8566 5 месяцев назад +1127

    Computers not being fast enough to make a correct algorithm practically usable reminds me of Reed-Solomon error correcting codes. They were developed in 1960 but computers were too slow for them to be practical. They went unused until 1982 when they were used in Compact Discs after computers had become fast enough.

    • @jimktrains0
      @jimktrains0 5 месяцев назад +99

      RS codes were used on the Voyager probes in 1977. CDs were the first large scale usage. Your basic point is still true: it took a while for computers to be complex enough to use them.

    • @T3sl4
      @T3sl4 5 месяцев назад +55

      Bayesian models have followed a similar path; the basic idea is so fundamental as to be trivial, but actually using it in practice requires a high level (uh, I don't know what the big-O complexity is -- quadratic? worse?) of detail and thus computation to truly harness. The parameters might be trivial (individually, or conceptually), but there are so many of them for a problem of modest scale that it's only recently we've made much use of it.

    • @kellymoses8566
      @kellymoses8566 5 месяцев назад +14

      @@jimktrains0 I should have specified first wide-spread use.

    • @ron5948
      @ron5948 5 месяцев назад

      Logix programming same prediction, eill be viable in a yeR year and I will do it ???😮❤😂🎉🇨🇭😘💶💶💶🍆🍑🍆🥑⛔⛔⛔🪬🤣😅🏳️‍🌈✡️💪🏾👯♂️♂️🔯✡️🔯👬🕎♀️⛔

    • @afterthesmash
      @afterthesmash 5 месяцев назад +21

      It has always been an easy decision tree. Will the interesting case fit in system memory at all? It not, wait for the next system refresh. Can I tolerate the latency? Predicting tomorrow's weather a week from now is a good example of not being able to tolerate the latency. If it fits in memory and I can tolerate the latency, am I willing to pay for the computer time?
      I recall hearing stories in the 1980s about a power station with an entire Vax 11/780 devoted to running an FFT kernel on generator shaft vibration. There was no legal way to ship a replacement shaft. They had barely been allowed to truck in the first one over existing roads. Hence they spent the moon looking after the one they had.

  • @ernestuz
    @ernestuz 5 месяцев назад +255

    I was working with deep neural networks at the university during the late 90s, the main issue that stopped all progress was the use of a kind of functions between layers (the sigmoid as activation function), this effectively stopped the learning backpropagating from the output layers and limiting how many layers you can use (the problem is called the vanishing gradient). Once people rediscovered ReLU (it was invented in the early 70s, I believe, but I think the inventor published it in Japanese, so it went unnoticed) deep neural networks became possible. High computation needs were only a problem if you wanted real time or low latency, those days we used to leaving the computer calculating during nighttime to get something next day.

    • @chiyembekezomaunjiri3278
      @chiyembekezomaunjiri3278 5 месяцев назад +18

      Thank you for all the work you did.

    • @dest5218
      @dest5218 5 месяцев назад +3

      Thank you for all your work, cant imagine doing all this back then

    • @yannickhein9788
      @yannickhein9788 5 месяцев назад +2

      While this video perfectly explained how the networks work during recognition, I don't understand how they are actually training all the layers. Does anyone have a similar good source about teaching neural networks / backpropagation?

    • @The_Quaalude
      @The_Quaalude 5 месяцев назад +2

      Bro was working on a toaster 😭

    • @ernestuz
      @ernestuz 5 месяцев назад

      @@yannickhein9788 Hi, the most common algorithm used today, backpropagation, is based on propagate the "error" (the difference between the neural network, now on nn, prediction and real value) backwards, from the output to the input. One way of seeing it is for every layer in the nn (though not all nn can be divided in layers, but lets simplify) the error at its output is transformed to an error at its input, having into account the contribution of each neuron to the result. Performing a search in YT, there are two videos on top:
      ruclips.net/video/Ilg3gGewQ5U/видео.html&pp=ygUZYmFja3Byb3BhZ2F0aW9uIGV4cGxhaW5lZA%3D%3D
      ruclips.net/video/IN2XmBhILt4/видео.html&pp=ygUZYmFja3Byb3BhZ2F0aW9uIGV4cGxhaW5lZA%3D%3D

  • @michaelala4924
    @michaelala4924 5 месяцев назад +565

    Awesome video! Funny how the moment we stopped understanding AI also appears to be the moment it started working lol

    • @andybaldman
      @andybaldman 5 месяцев назад +47

      An astute observation.

    • @MrAvaricia
      @MrAvaricia 5 месяцев назад +80

      It works like the brain. And like the brain, the moment the results are interesting is when they have enough oomph

    • @ObjectsInMotion
      @ObjectsInMotion 5 месяцев назад +133

      "If the brain were so simple we could understand it, we would be so simple that we couldn't"
      The same is true for AI.

    • @Sqlldude
      @Sqlldude 5 месяцев назад +19

      AI cant verify the truth of the answers it gives. It often gives shit answers.. So saying it works is a bit of a reach

    • @ObjectsInMotion
      @ObjectsInMotion 5 месяцев назад +34

      @@Sqlldude Humans can't verify the truth of the answers they give either. Both need an external source.

  • @4.0.4
    @4.0.4 5 месяцев назад +47

    The visualization is what takes this video from good to fantastic. It's very evident you put a lot of effort into making this visually engaging, which is very didactic!

    • @grapehool
      @grapehool 20 дней назад

      i had to search what didactic means

  • @frostebyte
    @frostebyte 5 месяцев назад +478

    I really appreciate how well you communicate non-verbally despite using very little A-roll. You're expressions are clear yet natural even while reading, enunciating and employing tone, and there's no fluff; you have a neutral point for your hands to signal that there's no gesture to pay attention to.
    I couldn't find anything to critique in your vids if I tried and this seems particularly easy to overlook. Thanks for every absolute banger!

    • @MathGPT
      @MathGPT 5 месяцев назад +5

      @@frostebyte he is truly a master teacher we can all learn from

    • @sntslilhlpr6601
      @sntslilhlpr6601 5 месяцев назад +4

      The vocal fry is annoying. A shame, because his vids are such fantastic quality otherwise. But I've literally just noped out of his vids before because it grates me so heavily. Use your lungs, my good man!

    • @StormTheSquid
      @StormTheSquid 5 месяцев назад +18

      Half of these comments read like they were written by chatgpt lmao

    • @codycast
      @codycast 5 месяцев назад +16

      @@sntslilhlpr6601I don’t know what “vocal fry” is but his voice doesn’t sound annoying to me.

    • @JorgetePanete
      @JorgetePanete 5 месяцев назад +2

      Your*

  • @talkysassis
    @talkysassis 29 дней назад +7

    My dad graduated around 2009. One of his teachers (that was my teacher at Computer Science too) said at the time neural networks would thrive if they find their place in practical applications, but at the time most computer work as analogue to human work, but we've been learning how to abstract everything and work from there. That was when programs designed for computers instead of digital versions of physical media got very very popular. As he said: the tools were already there, we just needed to know how to use them, and we would shift from designing computer programs from real world to design the real world around computers. This teacher is still one of the smartest people I've ever seen so far.
    To put that into perspective, digital document control until a few years ago was very tied to how we handled them with paper and programs for dedicated digital control were a massive change of paradigm when they got adopted. We now tie small databases with documents, link them to project files, communicate projects with attributes... What a good place for something like NNs.

  • @optiphonic_
    @optiphonic_ 5 месяцев назад +98

    Your visualisations helped a few concepts click for me around the layers and activations Ive struggled to understand for years. Thanks!

  • @crownoffyre894
    @crownoffyre894 Месяц назад +9

    2:40 dude this single picture right here the way you described it this way literally was like the the thing that truly helps me understand how this all worked thank you

  • @siddharth-gandhi
    @siddharth-gandhi 5 месяцев назад +180

    Stellar video, you’re gifted at communication. Keep at it!

  • @TheEVEInspiration
    @TheEVEInspiration 5 месяцев назад +17

    It is pretty amazing that these systems consume their own output to set the next step in their "reasoning".
    This implies that much of the true decision of the final output is actually already made in the first pass-through.
    And that the extra passes are just needed for us to extract the output in a way we can process.
    It also implies there is a "hidden" boundary of how far the first pass through can "reason", any cycles beyond that are "improvisations" of the path the output was already set on.

    • @clinttube
      @clinttube 3 месяца назад +2

      Very astute observation. And it gets to my biggest concern with any kind of recent AI model I encounter whether at work or in the wild: "what was this trained on"? Much like if you get a group of highly trained but inexperienced students together, the range/frequency of potential answers to a question near their field of expertise is likely to be a lot lower/tightly clustered than if you do the same with a less trained group. That initial lens through which the question gets passed (the training) can severely limit novel outputs.
      There are deeper connections between concepts it theoretically can make that humans may not, which is super cool, but fundamentally it's synthesizing. Various permutations and combinations of + - , * / , powers/square root, derivatives, and vectors.
      Another question I always have that is much harder to answer simply (if at all): "out of the various potential modes/models tested, what was it about this one being used that made it get selected for production". Haven't gotten a good answer yet; I'm sure if I dove deep I'd find some, but at least at work our AI folks aren't capable of explaining it.

  • @emrahe468
    @emrahe468 5 месяцев назад +282

    Amazing intro with scissor and carboards 👏

  • @Aofekiko
    @Aofekiko 5 месяцев назад +12

    The visual aid in this video is unlike I've seen anywhere else, it really helps grasp the ideas presented easily, wonderful video!

  • @CollinReinking
    @CollinReinking 5 месяцев назад +69

    The amount of work you must put into videos is mind boggling. Thank you for making them.

  • @beautifulsmall
    @beautifulsmall 5 месяцев назад +8

    A great learning experience i had was to deep dive into bitmap format and multiply greyscale images with 3x3, 5x5 arrays with simple patterns, ie all zero with a -1 in the middle. Different array patterns highlight edges or remove edges. it was a really eyeopening experience any software person should try that shows these fundamental operations. Great video.

  • @michaelm358
    @michaelm358 5 месяцев назад +60

    Very clever and nice vizualisations! Excellent as usual.
    Thank you!

  • @Sam_Saraguy
    @Sam_Saraguy 5 месяцев назад +448

    I stopped understanding AI around the six minute mark.

  • @AlvingGarcia
    @AlvingGarcia 5 месяцев назад +4

    I've been studying AI for the past year and the first 2 minutes was the best explanation I have see of how Transformers and ChatGPT works so far. Ive studied everything from Andrew Ngs Coursera courses, to Andrej Karpathy and more. Thank you for this great video!

  • @TheSoylentGreen
    @TheSoylentGreen 5 месяцев назад +4

    GREAT video. Your crystal clear script and visuals make a very complex topic approachable and your production values are top notch. Kudos!

  • @ben9089
    @ben9089 5 месяцев назад +32

    This was an incredible introduction in just 18 minutes. I continue to be blown away by this channel.

    • @WelchLabsVideo
      @WelchLabsVideo  5 месяцев назад +3

      Woohoo!

    • @thevulture5750
      @thevulture5750 22 дня назад

      ​@@WelchLabsVideothe KJV Bible is mathematically encoded by God

  • @SeanStClair-cr9jl
    @SeanStClair-cr9jl 5 месяцев назад +110

    It's rare to find an AI video this informative and interesting. Great pacing great focus, this is wonderful.
    I'm a particular fan of the sort of stop-motion / sped-up physical manipulation of papers on your desk with that overhead lighting. Very clean and engaging effect. Seeing the face-detecting kernel emerge after so few blocks was also mind-blowing!

  • @iccuwarn1781
    @iccuwarn1781 5 месяцев назад +66

    Fantastic presentation on the inner workings of machine learning!

  • @coscostan3334
    @coscostan3334 5 месяцев назад +3

    I've never seen AlexNet this way with a live preview of what happens inside. I'm sure it required a lot of time and programming: great job!

  • @manic-pixie
    @manic-pixie 5 месяцев назад +10

    I was literally talking to my roommate about this last night!! Thanks for the excellent video

  • @martinchakarov
    @martinchakarov 2 месяца назад +1

    The quality of this content is remarkable - great job! Looking forward to watching more awesome videos like this one.

  • @JulianKingman
    @JulianKingman 5 месяцев назад +6

    This is such a good explanation of AI, and the production value is so high. I'm bookmarking this so I can show it to my friends who ask me if I think AI is developing sentience.

  • @elektrolysid
    @elektrolysid 3 месяца назад +1

    What a remarkably well thought out, well organised, well created video! Just stumbled upon this channel and glad i did !

  • @denrober
    @denrober 5 месяцев назад +6

    Love this video. First one where I finally understand how gpt-4 works. Thank you.

  • @robert75019
    @robert75019 5 месяцев назад +2

    You were there 7 years ago when i started my AI journey in images….back at it when i start in language. Truly great work….love your content 👏👏👏

  • @PunmasterSTP
    @PunmasterSTP 5 месяцев назад +3

    The stop-motion and animation, including visualizing AlexNet's activation, were incredible!

  • @TheBooker66
    @TheBooker66 4 месяца назад +2

    Great video! I've been subbed ever since I've watched your amazing series on imaginary numbers, and the quality hasn't dropped and even improved. Looking forward to your next videos.

  • @zhoudan4387
    @zhoudan4387 5 месяцев назад +7

    It is not that the neural networks magically “understands” what is important. It is that the information is not random, so can be synthesized into smaller chunks. The synthesis process is what creates patterns, thus understanding.

  • @kurtu5
    @kurtu5 5 месяцев назад +1

    I have been having this conversation for the last 2 years. Thanks for putting it in video form and expanding the conversation past layer 1, as most people are totally lost on layer 1.

  • @alexvisan7622
    @alexvisan7622 5 месяцев назад +5

    Wow, so much effort has been put into the animations. Subscribed.

  • @johnconner4695
    @johnconner4695 3 месяца назад +2

    Dude your videos are amazing and that office space looks amazing.

  • @machinate
    @machinate 5 месяцев назад +147

    hehe, "hotdog / not hotdog".

  • @Paul0937
    @Paul0937 3 месяца назад +2

    Best dynamic illustrations yet. Using highlights on physically printed research papers is a wonderful story telling technique.

  • @DavidAspden
    @DavidAspden 5 месяцев назад +2

    You nailed it with this one. I'd love to see how much of your video was 'effect' and how much was real computation and composition. Seeing the third layer change with the video on that angle was very impressive.

    • @WelchLabsVideo
      @WelchLabsVideo  5 месяцев назад +3

      I try to "show the real thing" whenever I can, only thing that was really "effect" i think was showing the 96 kernels learning. I thought about actually doing a training run, but didn't have the time!

    • @DavidAspden
      @DavidAspden 5 месяцев назад +1

      @@WelchLabsVideo thanks for that honest insight. I'd put this above Grant's effort, which is a rock solid series. I really enjoyed the whole pace and b roll inserts of historical research.

  • @alessi4249
    @alessi4249 5 месяцев назад +4

    The amount of work that went into that visualisation i would love a behind the scenes video!

  • @AdvantestInc
    @AdvantestInc 5 месяцев назад +2

    This video is a fantastic resource for anyone interested in AI. Your ability to explain the intricate workings of AlexNet and GPT is commendable. Keep up the great work!

  • @Riksia
    @Riksia 4 месяца назад +16

    There will be some point in time, when people stop call statistic models an AI, but it will not be today for sure.

    • @tim40gabby25
      @tim40gabby25 4 месяца назад +1

      I bet on 6 months after fusion solved...

    • @waylandsmith
      @waylandsmith 2 месяца назад +2

      I finally gave up about a year ago in trying to hold the line on the definitional shift of "AI".

    • @givingin2G4Courtz
      @givingin2G4Courtz Месяц назад

      @@waylandsmith yep, it's seriously tiring. And frustrating beyond belief. And depressing.

    • @VampireFlutist
      @VampireFlutist Месяц назад

      I believe that might happen when AGI becomes possible and more widespread, and the distinction between today’s AI and the next evolution into AGI becomes necessary.

  • @GalaxyTracker
    @GalaxyTracker 5 месяцев назад +1

    Earned a sub for sure. The visualizations, and especially those of the real time activation maps are just incredible tools for a better understanding. Got into DL out of a hobby and now I am using it for my research in my scientific field, especially image processing. Visualizing exactly how they map data to vectors in each layer was eye opening.

  • @tommartens1722
    @tommartens1722 5 месяцев назад +3

    Fantastic video. I appreciate the time spent to create it

  • @bradleyfallon6847
    @bradleyfallon6847 3 месяца назад +1

    Very informative video. Thanks!! It's great to see content that actually includes in depth research and knowledge and not just enthusiastic speculation.

  • @amarokorama
    @amarokorama 5 месяцев назад +10

    Great video, insightful visualizations! Regarding your footnote at 6:15, though: the myth that mantis shrimp have great color vision has been debunked long ago. They're worse at it than we are. Just because they have many photoreceptor types doesn't mean they combine them in a way similar to humans or other animals. Shouldn't have been too surprising, given their lack of brainpower.

    • @kellymoses8566
      @kellymoses8566 5 месяцев назад

      Yes. Human vision takes up a huge part of our brain.

    • @bubbleman2002
      @bubbleman2002 4 месяца назад +1

      Say that to a mantis shrimp's face, I dare you.

  • @RussellDeming
    @RussellDeming 5 месяцев назад +1

    I absolutely appreciate the way in which you present this information in an easy to consume and understand format. Brilliant my friend.

  • @woolfel
    @woolfel 5 месяцев назад +25

    feature activation visualization aren't interpretable and there's papers that have addressed this issue. Even before Alexnet, researchers couldn't interpret the weights of a deep neural network. There wasn't really a moment when we stopped understanding neural networks, we never really understood them.
    We understand back propagation and the frameworks (tensorflow, pytorch, tinygrad), but we don't understand the weights.

    • @logandarnell8946
      @logandarnell8946 3 месяца назад +2

      thats why neural nets are a thing to begin with, manually programming things that specific and abstract is not a job for a human brain, way too complex, too many moving parts, too much trial and error. its likely impossible for a single human to ever actually understand the step by step process of a neural net after training data.

  • @stratfanstl
    @stratfanstl 5 месяцев назад +34

    Fantastic visualizations. It is very appropriate to try to think through this transformation process as you illustrate to first see how the algorithm first reorganizes info as we perceive it into info optimized for the algorithm to recursively refine. Once you see this first iteration, then "lose sight" of the next abstraction, it becomes apparent how impossible it will be for any human to identify and correct a "flaw" in an AI model. The only approach for "correcting" a flaw in "learned data" is to somehow feed the AI more data. That assumes an imperfect system WON'T become MORE imperfect by consuming more input. This defies logic.

    • @backwashjoe7864
      @backwashjoe7864 5 месяцев назад +6

      How does that defy logic?

    • @stratfanstl
      @stratfanstl 5 месяцев назад

      @@backwashjoe7864 Round #1 of the example showed that the algorithm is capble of creating flawed "links" or probabilities that lead to "incorrect" information being spit out for a given set of inputs. All of the inputs processsed in round #1 aren't "right" or "wrong," they just ARE. If the solution to (data)===> (partially incorrect output) is to feed more data in, there's no reason to expect round #2 to ELIMINATE the type of probabistic mistake encontered in round #1. It might REDUCE it but NEW errors can creep in, creating new errors in output, either for the original topic or some other prompt given the system.

  • @MedellinTangerine
    @MedellinTangerine 5 месяцев назад +1

    By far one of the best explanations of these mechanics in a video intended for laymen. Congrats + subscribed

  • @ar4hm4n
    @ar4hm4n 5 месяцев назад +6

    Visualization was just wonderful, but what attracted me more is the way of delivering the information.
    Excellent work! Keep it up!

  • @TinkerRaw
    @TinkerRaw 2 месяца назад +1

    Been trying to learn and implement CNNs in my PhD research and work, this has been the best video for helping me visualize each step of the process in my head :) Going to be trying to replicate these visualizations for my presentations as I think they're great ways to show how these 'black box' models function. As mentioned, the real time kernel mapping is super helpful. Wonderful work.

  • @raxirex6443
    @raxirex6443 5 месяцев назад +2

    A math professor of mine actually worked on many of the papers coming out of AI lab at MIT and he also worked on AI to play Minecraft. At the time it was really interesting to me as a sophomore, many years after I can write my own GPT, how the times haves changed!

  • @1ceBreaker99
    @1ceBreaker99 5 месяцев назад +1

    Truly amazing video, really great explanations and way of telling these hard to understand concepts. This got me more exited to learn more about this than an entire year of ai at university

  • @Ben-nx7yy
    @Ben-nx7yy Месяц назад +12

    I stopped understanding this video at about 2 minutes

  • @behnamjabbarizadeh4536
    @behnamjabbarizadeh4536 5 месяцев назад

    I presented Alex Net in my ML class and I am surprised at how good and intuitive you explained it.

  • @rotors_taker_0h
    @rotors_taker_0h 5 месяцев назад +22

    Great video. The only nitpick is with title: we haven't stopped understanding AI at AlexNet (and video clearly shows that we only getting better at understanding since that moment), we finally had working "AI" starting from AlexNet. All those "expert handcrafted" AIs before were no simpler to understand (if not harder) despite being handcrafted. And they largely didn't work and it was very hard to understand why.

    • @Anonymous-df8it
      @Anonymous-df8it 5 месяцев назад +1

      Why didn't they work?

    • @rotors_taker_0h
      @rotors_taker_0h 5 месяцев назад +2

      @@Anonymous-df8it too simple and brittle to capture the real world, I think. I started working on computer vision right after deep learning started to solve problems one by one but was not yet commonly accepted. So for some time people tried to use old and new methods and every single time classic methods only worked with toy versions of the problem and broke apart in real world when anything changed that you as human don't even notice, like different lamp temperature or some reflection.

    • @Anonymous-df8it
      @Anonymous-df8it 5 месяцев назад

      @@rotors_taker_0h Why would it be difficult to understand how they "work" or why they didn't? Also, what were the 'classic methods' and could people in the soft sciences who know programming create an image identifier or chatbot that actually thinks like us (which should work since people can do those things, and the code should be intuitive since it's our own thought processes)?
      I don't know about you, but I don't remember multiplying giant matrices together a bunch of times when thinking about how to respond to you (you could argue that I did but aliens wiped my memory or something, but that would be unfalsifiable), and whilst glare and monochromatic light sources would probably make it hard to see things, those are extreme cases, and I can certainly handle sunlight vs indoor lighting

    • @gilbert2720
      @gilbert2720 Месяц назад

      @@Anonymous-df8ityou didn’t do any math in your head, but your brain did all sorts of calculations behind the scenes.

    • @Anonymous-df8it
      @Anonymous-df8it Месяц назад +1

      @@gilbert2720 Like what?

  • @raizdesamauma8607
    @raizdesamauma8607 3 месяца назад +2

    I'm only 2 minutes into the video and already impressed by the best explanation I've ever seen about how models like gpt work

  • @svendtveskg5719
    @svendtveskg5719 5 месяцев назад +65

    The moment I stopped understanding a single word: 0:01.

    • @nicholascasey6473
      @nicholascasey6473 4 месяца назад +10

      "This is an activation atlas"
      Understandable, have a good day.

    • @nicholasn.2883
      @nicholasn.2883 4 месяца назад +2

      it’s not that hard you’re not doing any math just concepts

    • @logandarnell8946
      @logandarnell8946 3 месяца назад +2

      @@nicholasn.2883which means without prior knowledge you will not understand much of it. at least with math, it can be applied pretty universally except at extremely high levels.

  • @ptiwaridotin
    @ptiwaridotin 8 дней назад

    I took two Masters level courses in Machine Learning and Computer Vision to develop the understanding this video provides in 17 minutes!! Kudos for the excellent video

  • @huub_al
    @huub_al 5 месяцев назад +11

    Fun fact: Neural Networks are based on Neurons in the brain (hence the name), which we also do not know a lot about. Theory suggests that the neurons in our brain work very similar compared to a neural network in combining millions upon millions of simple transformations into something meaningful. This is also why research in fields like Cognitive Psychology go hand in hand with AI research. Very interesting to see where both fields are headed, because the key to understanding human intelligence is in understanding the unthinkable depths of neurons.

  • @Gabriel_CeS
    @Gabriel_CeS 5 месяцев назад +1

    Hey, thanks for this presentation. This video is a great example of how to teach about neural networks and their development. From the audio to the level of detail, everything was top notch. I hope you make more videos. I wish you much success, health and knowledge.

  • @JamesTiberius-pw1du
    @JamesTiberius-pw1du 3 месяца назад +5

    As a very young engineer I got involved in NN with the publication in the Signal Processing IEEE journal an article on the MLP by Lippman. I also worked at a small company for the president who was at Cornell name Frank Rosenblatt. It became my job to integrate NN into our product. I developed a cool way to deal with regularization and realized how critical regularization was as we had very little data. Did not have a billion images of cats. I wrote early FORTRAN code for back propagation that ran in a Sky Warrior array processor. No one knew what would become of the field and the history of Rosenblatt v Minsky. I am sad that Frank never lived long enough to see the word 'Perceptron' on t-shirts. He won. Minsky is a foot note.

  • @Hiyori___
    @Hiyori___ 4 месяца назад +1

    incredible video.
    I'm working with a ResNet on a project and sometimes, focusing on minor bugs and programming headeaches, I tend to lose the perspective on the amazing tool I'm dealing with. This video was a pleasure to watch

  • @AvastarBin
    @AvastarBin 5 месяцев назад

    This was very well explained. And the fact that you took all the images of the video and put it in a nn just to show a visualisation for a few seconds shows how much you care about the quality of your videos

  • @demolle87
    @demolle87 5 месяцев назад +30

    so basically AI sees in LSD

    • @moggadah
      @moggadah 2 месяца назад

      Yes, I've never taken LSD, but I understood that it is similar when I saw so called deep dream images ca 2015. Holy s*** I thought, if AI can hallucinate like that it must be working like the brain.

    • @invven2750
      @invven2750 14 дней назад

      It's possible that LSD screws some higher order of operations indeed

  • @johndaviddeatherage2232
    @johndaviddeatherage2232 5 месяцев назад

    thank you for taking the time to make this video. What a great explanation!

  • @thicksteve1761
    @thicksteve1761 Месяц назад +3

    1:04 the moment I stopped understanding this video

  • @seriousbusiness2293
    @seriousbusiness2293 5 месяцев назад +1

    Perfect mix of engaging and informative. And im not saying that losely, compared to other creators you actually showed in depth workings of the networks that dont always get mentioned but you also managed to make it understandable for novices, i could really see that with your visuals you set up.
    subbed

  • @jonanderirureta8331
    @jonanderirureta8331 3 месяца назад +3

    After seeing how much computational power is needed for a language model to correctly identify an elephant, for example, one has to wonder about the computational power of the human brain, since a toddler can do the same.

  • @ubiquitous9105
    @ubiquitous9105 4 месяца назад

    This is a fantastic video. Thanks for visualising the kernels so well. I enjoyed every minute of it. I've re-watched a couple of times now :)

  • @kellymoses8566
    @kellymoses8566 5 месяцев назад +5

    The 3D visualizations of the neural network activation is incredible. What did you use to do it?

    • @WelchLabsVideo
      @WelchLabsVideo  5 месяцев назад +3

      Really crappy VPython code I wrote.

    • @jvo1464
      @jvo1464 5 месяцев назад +2

      ​@@WelchLabsVideo it's not crappy if it works!

  • @radupopovici118
    @radupopovici118 5 месяцев назад +2

    absolutely amazing video, with great explenation and visuals. Keep up the good work!!

  • @Qstate
    @Qstate 5 месяцев назад +3

    What an insanly high production value.

  • @bradleypeterson7255
    @bradleypeterson7255 Месяц назад

    This is the absolute best video I’ve ever watched on CNNs. The clarity of the explanations AND visuals… Seriously impressive. Wow.

  • @ViralKiller
    @ViralKiller 5 месяцев назад +9

    What people fail to explain is, the training has 2 core chunks. The first stage is 'pre-training' when it is fed millions of words, to understand general relationships between them. No strcuture just words and letters. The second stage is secret but we can speculate this is the 'fine-tuning' stage where data is provided as a JSON file containing a question and answer parts. I mean this is how they would do it if smart. There are also other 'experts' like code maths etc....

    • @rahul_siloniya
      @rahul_siloniya 5 месяцев назад

      Why is it secret now? Can't we look at Llama and check what that "secret" step is?

    • @jcm2606
      @jcm2606 5 месяцев назад +2

      @@rahul_siloniya Because OpenAI and Meta keep their training datasets and procedures secret. We can't learn anything meaningful about how LLaMA or GPT was trained by looking at the model, as the model is just a set of seemingly random weights with no indication of how the weights were arrived at or what the weights actually mean. Anthropic are trying to reverse engineer the weights to figure out what they mean, but that still leaves us in the dark regarding how these models were trained.

    • @tyruslawhorn
      @tyruslawhorn 2 месяца назад

      The second step probably involves virgin sacrifices, demon summonings and an assortment of scented candles from Bath & Body Works

  • @90whiterabbit
    @90whiterabbit 4 месяца назад

    Phenomenal video!! Love the cut out approach to showing how the different elements of the algorithms come together!

  • @tommyshobalongdong
    @tommyshobalongdong 5 месяцев назад +11

    “No one told Alex what a face was, we just forced it to see millions of them over and over again, crazzzyyyyy!”

    • @tommyshobalongdong
      @tommyshobalongdong 5 месяцев назад +5

      “What’s even crazier is that the math from a photo is similar to other similar photos, crazyyyyyyyy”

    • @zelda_smile
      @zelda_smile 2 месяца назад +1

      How can we make Alex unlock 100% of its intelligence?

  • @alegian7934
    @alegian7934 5 месяцев назад +1

    i really like the direction this channel has taken the last few months! keep it up :)

  • @theskinegg9168
    @theskinegg9168 5 месяцев назад +16

    alr why does the right poster 17:38 look like Africa

    • @HomelessPank
      @HomelessPank 2 месяца назад

      Pre continental drift heatlands

  • @pcp351
    @pcp351 5 месяцев назад

    The video -> layers -> activation map animation is one of the best clarifying animations I have seen describing this process.

  • @ligz.3437
    @ligz.3437 5 месяцев назад +3

    2:24 no way you actually asked if it was mad 💀

  • @wii3willRule
    @wii3willRule 5 месяцев назад +2

    This is amazing! Excellent presentation, this also illuminated for me what exactly a "latent space" is. Thank you!

  • @roycohen.
    @roycohen. 3 месяца назад +5

    The more I learn about this so called "AI" (while completely amazing, don't get me wrong), the more I realize the hype is a crock of shit. It cannot reason.

    • @frentiz9209
      @frentiz9209 2 месяца назад

      They boost the hype to get money from stupid investors

  • @Just4Growers
    @Just4Growers 5 месяцев назад

    Two minutes into this video, I decided to subscribe. ❤ Awesome presentation. Thank you!

  • @djayjp
    @djayjp 5 месяцев назад +3

    13:53 Curious how logic operations look so much like the brain's own neural network....

  • @morteneriksen4589
    @morteneriksen4589 5 месяцев назад

    The production quality of your videos nowadays is unbelievably good. The graphics, animations, transitions, and how you tie everything to the script is just massively impressive, way better even than almost all mainstream professional educational and documentary productions.
    (And the content is mindblowing, but that goes without saying.)

  • @iceboorg9737
    @iceboorg9737 2 месяца назад +6

    Man im too dumb for these videos so sad

  • @dahahaka
    @dahahaka 5 месяцев назад +1

    The visualization at 5:11 is absolutely amazing, kudos!!

  • @MathGPT
    @MathGPT 5 месяцев назад +4

    Is the hotdog a reference to Silicon Valley?

    • @Horopter
      @Horopter 5 месяцев назад +3

      It's also a reference to NOT hotdog 🌭

    • @aronsandstedt6055
      @aronsandstedt6055 5 месяцев назад +1

      Like Shazam for food!

  • @unflexian
    @unflexian 3 месяца назад +1

    been following since the early early days, and gotta say that it's criminal that you haven't gotten an award for science communication yet.

  • @Not_a_Pro360
    @Not_a_Pro360 5 месяцев назад +10

    Yes, Ai is literally about creating programs that are too complex for a human to understand.

  • @Manwith6secondmemory
    @Manwith6secondmemory 5 месяцев назад +11

    Humans will never accept that we are not special... life/sentience is probably special and very rare but the way living systems process things is the same, humans are just a level above, an abstraction higher than other intelligent animal species who are an abstraction above less intelligent species.
    In terms of raw intelligence, orcas, dolphins, apes, and crows are not that far behind humans. Human language allows us to communicate much better and organize our experience of the world and build upon our organizations of knowledge. Remember our species has been around for 300,000 years. Our hominid ancestors started to appear 2 million years ago. Complex civilization didn't exist until 6000 years ago. Civilizations were mostly agrarian, powered by human and animal muscle until about 300 years ago

    • @ChocoRainbowCorn
      @ChocoRainbowCorn 5 месяцев назад

      I highly doubt that sentience is anything special but I do agree with what you are saying otherwise. I think we will eventually be able to figure out what constitutes sentience first and eventually consciousness itself and following that, slowly reproduce it in next years, piece by piece.

    • @Manwith6secondmemory
      @Manwith6secondmemory 5 месяцев назад

      @@ChocoRainbowCorn true there are probably aliens. Its just fermi paradox begs the question.
      Our solar system pattern is rare. We orbit a medium sized yellow dwarf. Most solar systems are red dwarfs or multi star systems. Red dwarfs are small and prone to flares and planets around them are tidally locked (not rotating). Multi star systems will mess with planet orbits and gravity and climates, these systems have a low potential for life.
      Another thing is that most systems have hot Jupiter’s, where a large gas giant orbits close to the sun with rocky planets on the outside. But our Jupiter sits on the outside, it does a good job of shielding us and does not obstruct us from the habitable zone of our system. This is a rare configuration though.
      There is also evidence that life formed almost immediately on Earth as soon as it cooled down. Think about Earth now, it oozes life. Nuclear war would destroy humanity but even if we detonated every single nuke it would barely impact Earth. Many species would go extinct including us, but surviving species would evolve starting another cycle.

    • @tim40gabby25
      @tim40gabby25 4 месяца назад +2

      If octopuses also lived 70 years..

    • @ChocoRainbowCorn
      @ChocoRainbowCorn 4 месяца назад +1

      I also wanted to add, only sort of related to this - The intelligence thing is exactly why I find vegans/vegetarians annoying when and if they do preach online or in reality. Animals on same level as us be damned, I don't believe that for a second. Alright, we don't have an advantage in nature for many things - So? What out of it? We make up for it exactly thanks to our vastly superior intelligence, and it is my reason to believe that animals, except for the cases we have specifically chosen, should keep being treated as food source, regardless of the health of it. You want meat because it tastes good? Go for it! Want veggies and plants? Sure. Just don't push it on me, don't compare me to murderers or other such people. We, as humans, are and will always be above animals, and we, as well as other animals that possess similar intelligence and brain power capacities to us, like our pet dogs or cats, dolphins or monkeys should also be kept alive and treated well for this reason while others, remain food.

    • @Manwith6secondmemory
      @Manwith6secondmemory 4 месяца назад

      @@ChocoRainbowCorn domesticated animals were domesticated by us. They did not ask to be here we breed them into existence .
      Domesticated animals implicitly trust/rely on humans and are not hostile towards us. At the very least we should not cause them undue suffering. Eating them is fine but modern factory farming is brutal. Billions of animals suffering under this system.
      Even if they are not as intelligent, they still suffer/feel pain and experience the world similar to us, at least mammals do.
      Really do not understand your point, vegans are a small minority

  • @dfbgsdfh1
    @dfbgsdfh1 5 месяцев назад

    Wow, the clearest video I have seen, the editing was great to clearly show the concepts quickly.

  • @Will_Forge
    @Will_Forge 5 месяцев назад +8

    So the Mayan calendar predicting that there would be the start of a new age the Mayans couldn't comprehend in 2012 was, in a way, accurate? The AI age started with AlexAI in 2012?

    • @DizGaAlcam
      @DizGaAlcam 5 месяцев назад +2

      Ur onto smth

    • @Will_Forge
      @Will_Forge 5 месяцев назад +2

      @@DizGaAlcam Yeah, and it's certainly just a coincidence that my animal brain is seeing as a pattern, but still! Concerning for animal brain reasons. 😅