I think you are the perfect combination of beauty and intelligence. I have never in my life binge watched anything but I have watched everyone of your videos and I cant wait until your next release. I used to work for IBM and we have similar educational backgrounds. You make the most interesting deep dives into all these different technologies. Thank you.
Always enjoy your enthousiasm on sharing your views on these technologies. How would you scale the Dojo (V! or V2) to the Northpole version (thinking about massive data training like video?)
I am ecstatic, and very proud to be part of the team that developed the chip (top right in the suit with long hair :-), also co-author of the paper in the Science Journal. Thank you for beautifully describing our invention and efforts.😮
Correction: IBM has not "released" this chip - it is an experimental chip to be used for research and development purposes rather than high volume production.
I remember when memresistors came out, but then radio silence. This is pretty exciting. Thanks for your expertise and breakdown. Been looking forward to commercial neuromorphic chips for a long time.
Yeah memristor had a lot of hype both for industry and HP(E) as a company, but after that total radio silence as u write... clearly memristor seems not as viable and commercially possible as originally expected...
Yeah I know... this "thing" that's a 100x faster... and you can't buy it anywhere, I know a lot of these things existing, but sadly if you can't buy the thing and use it.... it doesn't matter how fast it is.... and it doesn't matter if it exists at all.
Memresistors are a thing, but manufacturing them at scale is still way beyond current technology which is still getting a grasp on nanotechnology. It is likely that an upcoming technology such as doping silicon with graphene will render them moot.
In my life, I have seen computers with perforated card input...and AI processors...the incremental rate of technology progress is beyond words. Thanks very much for your outstanding channel.
you really hold my attention. i'm 79 and i want to keep living just to see all this stuff come into our world, functioning in our world. so weird. i worked in computers starting in 1980. then in networking in 1990, at nasa mt. view, wide area with sterling software. it just doesn't stop.
Thank you, Anya. Neuromorphic computing is a fascinating field, and IBM achievement is truly a revolutionary progress in enabling general AI robotics development. Great vlog for acquainting us, your viewers, with this new and exciting technology. Thank you, bunch.😊
Possibile che non capiate che tutta questa "AI" nata per migliorare la nostra vita servira poi per imprigionarci?. Anche l'atomo era studiato per scopi benefici, guardate come è andate a finire. Svegliatevi!!!!
Hello! First off, I absolutely love your videos - they’re informative and super engaging! Your latest piece on anamorphic CPUs was a thrill to watch. Can’t wait for what you have in store next. I’ve got a suggestion that might add even more depth to the conversation. While the hardware insights are phenomenal, I believe delving into the software aspect could be incredibly enlightening. Understanding the interplay between software support and compilers is crucial for leveraging the full potential of these innovative hardware designs. It would be awesome to see content that bridges this gap, highlighting how software ecosystems evolve to keep pace with hardware advancements. Keep up the fantastic work!
chip makers have leaned on node for so long but starting in 2030, the leaders will be based solely on creative design. we are gonna see some insanely smart design that never would have been thought of if they could have continued to rely on node shrinks. this is ESP true in graphics and ai. cant wait to see how they solve problems when they cant go smaller
Moore's Law definitely made the chip design industry lazy and complacent, they let processor architecture innovations stagnate while they made the fabs do all the hard work of actually making computers faster through node shrinks, even though the limitations of Von Neumann architecture have been known for literally decades.
I think we are gonna see incredible things in out of the box thinking and I have my fingers crossed we will see more opensource designs and design tools once node changes arnt changing the rules ever 2 or 3 years. @@nekomakhea9440
As someone who has started my IT career in IBM Mainframes Z/OS, COBOL & CICS, I'm ecstatic too. IBM, once a behemoth sort of lost its footing. Im happy its clawing its way back and that too in hardware, rightfully so.
This sounds a lot like state machines. The fact that there is no distinction between the computing function and the results storage is new. It is long over due as an approach for sure.
Thank you for another excellent video. I'm excited to see what type of performance IBM will get out of this technology when they scale it to the 4nm world.
Thank you for shedding the light on this amazing ground breaking (and mind blowing) achievement from IBM. Needs more PR from IBM. Seems like nVidia is all rage these days with their specialised AI chips.
Fabulous presentation, giving race car feel for the exciting chip race that is now on going. NorthPole is super impressive, but unclear to me how far away it is from becoming a commercial product. As is it seems way too big for edge applications and although it could presumably be made in 3 nm or similar it would still be large, suggesting IBM will have to find ways of making it much smaller to use in edge applications. Super interesting yes, but not yet a practical competitor to Nvidia as far as I can see. Thanks for sharing!
Is Northpole more efficient than Brainchip's chip, do you think? I can't find any specs on brainchip's Akida chips, but they've been claiming to be more energy efficient than anyone else for a while. Edit: I wrote this comment before I got to the part where you covered Brainchip's Akida chip... But still the question stands.
It’s hard to foresee how much potential this holds for AI. I mean if it is already 20 times faster then the H100, what might be possible if we go down to 5 nm or less. People always speak about the end of Moores law, I think for AI mores law (at least in its classical formulation) isn’t all that relevant any more. Nvidia already says they manage a factor 1000 in 5 years when it comes to AI. Given that we may be rather close to AGI (many see it happen 2025 -2029, Kurzweil once used to be called an optimist) let’s see how it can improve it’s own hardware (not to mention it’s software). I think we have exponential times ahead that dwarf Moores law
I've seen now a lot of your Videos and i must tell you that i like your unique dialekt very much. The first time i watched your Videos it sounds a little bit strange to me, but this is what makes you special. After all, i like your Videos because one can see that your research is profound. ✌🏻
I worked with the previous generation and it's extremely interesting to train on. It performs off-chip learning so your data is stored off the processing server. Intel's Luihi are an on chip solution
Lovely breakdown, thank you for sharing not just IBM's breakthrough but also the potential applications of others and the different paths of AI chip development that companies are taking. I think it's interesting regarding about real-time applications versus other applications. I could see a future where both are used in conjunction to create a very adaptable AI robot or sentience.
This is a great video explaining the new IBM AI chip, NorthPole, that uses a brain-inspired architecture to achieve faster and more energy-efficient AI inference. I’m impressed by how it can perform multiply-accumulate operations within the memory, eliminating the von Neumann bottleneck that plagues traditional chips. I wonder how it compares to other analog AI chips that use phase-change memory or resistive non-volatile memory devices. Thanks for sharing this informative and engaging content ty Bing AI
I think that for better power efficiency spintronics related technologies seems to be more promising like for example Intel MESO concept that seems well suited both for digital logic and neuromorphic computing…
the thing i love the most about this chip is how it's only using 74w which means it can be powered fully through a PCIe slot, the biggest obstacle i can see tho is software support. we can already see that with AMD GPU to an extent, they're very capable compute machines and often time have more VRAM than Nvidia counterparts at the same price but since they're not supported by tensorflow or pytorch they can't be used for AI easily
It seems like everyone is getting into the game of chips. While exciting to see, it looks akin to "throwing mud at a wall" to see what sticks and there are ALOT of variations that are "sticking", so much so that other teams are learning from what "seems to stick" and growing in unexpected ways. Pretty soon I'm certain that the right mixture is going to be revealed and start the next tech revolution.
IBM has been at this a really long time. They just have a problem with executing and profiting at scale on some of their best ideas, which subsequently get swiped by others. Case in point RDBMS (swiped by Oracle), affordable PC (swiped Microsoft) etc the list is endless.
This is NOT a complaint! I noticed your vernacular shifted after your description of IBM's chip. Oh it is subtle but words are my craft. As a native English speaker it seems to me that your words were different when talking about IBM and then they returned to your own after that. Could be due to your method of relating the information to us! Not like you read the segment about IBM. I would never accuse you of that! I admire your technical expertise greatly! It's more like your presentation is based on a particularly vivid memory of your research into the new IBM chip! I study/speak a different language than English myself... I suck at it!! Your English is fantastic! Always good to raise the bar! Best wishes!
Thanks for the (as usual) very informative video! Question: how does it compare with the chips in Dojo? Maybe Tesla should become a good IBM customer 😀
@@AnastasiInTech maybe Tesla can give a push to IBM to put them in production as soon as possible, as you said it's based on scalable and mature technology 👍
Anastasiia before i comment on Chip design... I would like to say... You look amazing... AI is being held back by conventional compute parameters and constructure material availability AI has to have full control on all phases of the chip design and once AI stacks overall design parameters from material to automation building consensus. But AI is bottlenecked... So only AI can solve all concepts of the Designing. Then and only then will life changing AI chip design be revolutionary. Great topic ty Anastasiia ❤️❤️❤️
I remember the North Star from yesteryear. Great to see IBM at the forefront again. I have always thought that bending GPU ray tracing pipelines around to do more general computing is fundamentally inefficient. That 25x efficiency multiple underlines that point. The spiking NN without central clock signal should yield more savings still. Great segment, Anastasi! Thank you.
Very interesting presentation. Combining memory and computing power makes me think about the concept of "work space" where you put your code and your data in the same RAM memory. This is going a step further by putting the parallel processing power as close as possible to the code and the data. Thank you.
Hello, I am an also Electrical Engineer and Ilike your interesting content and it is very useful to progress my knowledge in hardware, computing and AI! Thanks for this content!
Thanks for the review. Excellent. A limitation of the Neuromorphic chips in the data Center is that most of them are not that impressive in terms of Training, even more so if one wants to test different models to see which performs better.
Great summary and great news that IBM is leading the way in neuromorphic processors. Maybe the ultimate solution will be a combination of analog and digital technologies.
@RajGandhasri Congrats!, as an outsider may I know what the inspiration was for designing and fabricating NorthPole with Compute+memory matrix, seem like distributed computing is the trend for systems design. Thanks, Anastasi, for covering it in a video.
What IBM is doing there, as you describe it, seems to me surprising and impressive. The first processor I ever got inside of was a discrete logic 11-bit instruction word, dedicated unit. So much progress.
It is important to put the "should do" between "can do" and "will do." That was the difficulty IBM had in the past... and they chased a lot of "can do" without knowing the formula of "should do" and profitability.
IBM is on streak right now being more successful than ever. They have successfully transitioned into 21st century model focusing on SW, consulting, infrastructure with plenty of patents, research, their own processors and tech including Quantum. IBM is growing!👍
honestly, 2 of these chips have the same through put as a H100 GPU being 12x more energy efficiency, so I think these could genuinly be used in a scaled up system to make an industrial large neural network training computer
Another amazing video thank you for sharing. Thank you for the summary of other neuromorphic compute solutions. I've been following Sony's work on neuromorphic camera sensors but wasnt as aware of these developments.
What are the drawbacks of neuromorphic chip comparing to the "standard" architecture, the one used on the Nvidia chips? Why does the Intel specialist say the "standard" architecture will be adopted for a long time in cloud computers ? Is it because of the price only ?
I've been looking for something to do with language models for a while now, like llama2. But as far as I understand, no matter how good the models are, unless there are algorithms that will make the learning process better, this work will take a lot of energy, money and time. The complexity and amount of input required to train the model is enormous. The models take the human brain as an example, but I don't think humans need that much data to learn. The problem here is still not understanding the human learning process or not being able to make algorithms accordingly. Most people live 70-80 years using about 800 words a day and knowing 20-30 thousand words :)) I think analog chips can create a new opportunity. good posts.
This reminds me on SyNAPSE, a backronym standing for Systems of Neuromorphic Adaptive Plastic Scalable Electronics. A DARPA Project for asynchronus neuromorphic computing of the early 2000s.
As a retired ATT systems technician, I worked on cord switchboards in the seventies supported by electromechanical 701 step by step switch gear. By 1986 we were putting in System 85 with timed multiple switching and fiber optics connected the remote nodes between buildings A RAM board was 64k and when we saw a 256k board come out soon after we would tell people it was the FM system (Fucking magic). Lol. Fast forward to today and the exponential advancement in technology is unbelievable. Good Job IBM. I’ll bet you guys did all this not even wearing a suit and short haircut they demanded back in the day.
This is great and all, but the power consumption for TrueNorth was a puny 65 mW. So NorthPole has increased perf by 4,000x while increasing power consumption by over 1,000x too. Making the actual gain closer to 4x in perf/watt over TrueNorth. That is still a gain for sure, but all the same it doesn't seem like as big of a gain as I would have hoped for considering they were known to be researching replacing SRAM with MRAM, PCRAM and likely other types of persistent non volatile memory that could make a neuromorphic chip far more efficient (and potentially smaller too).
Thanks ! Memory inside the GPU , we have to see the chip in action . Definitely a solution. Is it open source the software stack ? Or Proprietary closed source like Nvidia? So far Intel and AMD are open source for their new GPU's. In my opinion very important . Are they planning RISC-V ?
Disappointing that more Neuromorphic chips weren't included in the report... Really feel this should be an open source data gathering project to provide an honest benchmark against all manufacturers.
@@MuantanamoMobile There really aren't many neuromorphic chip manufacturers with real products out there... Not yet at least. When do you stop? In research, til no stone lies unturned.
I have no doubt about that's the decade of first emerging and growth of AI and any tool related to! I'm amazed of how different things will starts to become.
Well ....look like Gradient might be a good start for me but as I reorganize I to am switching/starting something new this quarter. I want to begin on AI projects now because their pillar-like position is key in business for one app/web site I am working on. Gradient offers an independent build but I wanted a more professional hands-on help with this build. But it's on my mind.😅 Thanks
So many steps involved… Design Build Pilot production Commercial production Chips are now requiring software suites to support the various functions Does IBM have the software suite to support the new high speed more efficient chips, or is this still a ways off in the future? Other chip manufacturer presentations highlight the suite of software that supports their customers… Nice presentation Anastasi, thank you! 😃
2:15 - video says the chip can be scaled down from 12nm to 3 nm. But if the design contains SRAM cells, the memory itself will not scale down. Or has this issue been resolved?
That's interesting and logical that IBM would be developing and researching new software AI frameworks as it tries to catch up to TSMC in the hardware foundry side. But, I suspect that not all is roses with what IBM is doing. AFAIK one of the strengths of the nVidia ecosystem and the AMD support for CLI is that they're framework agnostic except for their chosen language support so support practically all coding and applications. I'm guessing this isn't the case for IBM's approach... If IBM's product history is a guide, IBM is likely achieving those efficiencies and performance by hard coding long functions into silicon, and that typically means that only certain software coding and software functions are supported which would likely mean that certain types of applications can benefit from IBM chip features. An example of this is in the CPU world where IBM has long been a leader, IBM CPUs are generally best for machines used for business and typical business applications and less for scientific research and raw number crunching which is why GPU computing and graphics have been largely built on ARM chips. It's for this reason I wonder if the described IBM chips would be suitable for machine learning and neural networks despite the brief mention of neural networking in this video... If neural networks might be supported, I'm guessing only in a very specialized field. I suspect that if I'm right about IBM's chip architecture that these chips might require a lot of handcrafted AI created by human AI scientists. Or, IBM would have to create its own meta layer sort of like the .NET runtime that translates human coding into optimized machine language.
In your most humble opinion, what would you say is the "next best" currently or soon to be interconnect? I rabbit holed into gen-z consortium projects and am intrigued.
Asynchonous or "no synchronous" is an important aspect to consider. It can say compress a century of high speed sample data to produce and "answer" in a bit or a byte or bytes. It allows so many assumed rules to be discarded or alleviated towards progress to quality of information in efficient time domains. To include "feedback loops" to both sides of the underlying way a program can re-program itself, while including ways to incorporate "vectors" in time and in space. Any mix/match use of these concepts at any level will drive efficiency, and produce answers to create better questions. It is better to know an answer before one need consider the questions. Gr8! Peace ☮💜
I would hope this is the start of something new in the industry. I am glad IBM is still a R&D juggernaut. When IBM makes advances we all benefit from it.
Can you post an update video on this subject? IBM stock has been setting new highs since August and so I am wondering if they may have something in the pipeline to compete with NVIDIA.
Even though I only understand about 25% of what she's saying. I'm thinking about how proud her mom and dad must be. To have such a smart daughter. Thank you for what you do. I look forward to these presentations. Because I get a little something out of all of them.
Great video! This IBM chip reminds me of the good’ol Transputer chip architecture available in the late 80’s, rebuilt using current digital technology advances. Reckon Neuromorphic chips will totally smash GPU chips (and obviously CPU chips) in the AI domain. Also, I first saw a software demo of Spiking Neural Nets by its inventor in 2000, and this tech is astounding. Just wait and see, although seriously don’t think BRN will lead this…. definitely far too slow. It’ll be someone else yet to emerge.
It wouldn't smash anything, I hardly believe IBM are more advanced than Nvidia in AI chips. Google tensorflow chips were also impressive and faster than anything Nvidia had, when they were first introduced but Nvidia did catch up and surpassed them. Nvidia H100 is the fastest AI chip at present on the market. On another note Google does not sell their chips, they offer them only through their "cloud solutions" .
@@Slav4o911 Nvidia is definitely up there atm, however learn about Cerebra’s chip, which uses a Neuromorphic architecture. Already higher performance than Nvidia for AI. Nvidia uses a general architecture which is great for matrix maths, but Neuromorphic architecture is a custom built for neural network processing which is in some ways different. However, neither of these chips are custom built for SNN processing, which is different again.
I have trouble conceiving all the internet on one chip. I am talking about the memory being on the North Pole chip. It can pull this from memory much faster like the cache used to be on the main processor. How many tokens is that? The comparisons from all the different chips and chip companies is looking good on the charts, I just don't know what the values represent.
This is the second time IBM has come up with revolutionary chip design. Most might not remember the Cell Architecture that was suppose to bridge the gap between a CPU and GPU and was used in Sony PlayStation 3. But it failed because it required totally new approach to software development. With the kind of working that this chip is presented with, we should expect a similar complication in software development for this chip as well along with some other problems that we might have not predicted. Anyways, it is amazing development none the less.
This is such awesome technology. To a degree, I almost see (for example) AMD moving more memory on-chip. Obviously it's not exactly the same as having the memory right near the compute elements in the design, but I guess what I'm getting at, is that there seems to be a trend towards moving memory more "on chip" so you don't require external busses as much or the "hop" to increase latency is being reduced by bringing more of it on die/at least in a nearby chiplet with a much faster interconnect than is possible. You know, for reasons like signaling integrity, error correction and latency. AMD has done a lot more of this with the much larger L3 caches on some of the new EPYC chips. Apple is also doing this with their Apple Silicon line of CPUs by bringing the DRAM package directly onto the CPU package, and their own custom interconnects for ultra high bandwidth/low latency. I suspect we're going to continue seeing a shift towards more and more memory being crammed directly onto the CPU package, or even into the die itself. For now, I suspect "external" DRAM (external to the CPU package I mean) is still going to be around for quite some time because of physical limitations. I'm not sure 3D stacking technology is a proper solution just yet because the more layers and the increased density create problems with feeding power into the package, and also with thermals. What would be cool is to hear about any further developments with on-die liquid cooling. I remember a while ago it was being explored to have layers cut directly into the silicon die for water channels, apparently the thermals were far superior to anything that can be done with traditional cooling methods. If this is done in a 3D stacked manner, then perhaps being able to cram gigabytes (or even terabytes) of DRAM directly into a CPU die or onto the package using a ultra-high speed interconnect would be far more possible.
"memory compute" is a little misleading there are no computations done in the memory but because it is co close to compute unit it can perform much faster and energy efficient because it can move data faster and spend less energy on doing it
hey Anastasi, great work. Would love to see a video comparing how these chips compare to the one George Hotz is creating and how do you see chip manufacturing getting accessible to smaller firms & teams. sort of a democratization of hardware biz.
Check out Gradient: gradient.1stcollab.com/anastasi
💓
I think you are the perfect combination of beauty and intelligence. I have never in my life binge watched anything but I have watched everyone of your videos and I cant wait until your next release. I used to work for IBM and we have similar educational backgrounds. You make the most interesting deep dives into all these different technologies. Thank you.
Anastasi!🌷
Always enjoy your enthousiasm on sharing your views on these technologies. How would you scale the Dojo (V! or V2) to the Northpole version (thinking about massive data training like video?)
@@avos1786sorry, I didn't see you already asked about Dojo 😀
I am ecstatic, and very proud to be part of the team that developed the chip (top right in the suit with long hair :-), also co-author of the paper in the Science Journal. Thank you for beautifully describing our invention and efforts.😮
Wow! Rajamohan, thank you for the comment. You, guys, did a great job! You inspire the next generation of scientists and engineers
Congrats bro. Thank you for the effort and bring such an innovation to us.
Awesome!
@@AnastasiInTech Thank you!
@@leosmi1 Thank you!
Correction: IBM has not "released" this chip - it is an experimental chip to be used for research and development purposes rather than high volume production.
Yes, it is more like proof of concept/architecture
@@RajGandhasriwhat’s the failure rate for these chips?
@@RajGandhasri how it can be ordered?
@@RajGandhasri how much, do you sell the motherboard as well.
@@RajGandhasriDo you have an URL for further info?
I remember when memresistors came out, but then radio silence. This is pretty exciting. Thanks for your expertise and breakdown. Been looking forward to commercial neuromorphic chips for a long time.
what do you think about brainchips Akida?
Yeah memristor had a lot of hype both for industry and HP(E) as a company, but after that total radio silence as u write... clearly memristor seems not as viable and commercially possible as originally expected...
Dear god. dont breed!! PLEASE!!
Yeah I know... this "thing" that's a 100x faster... and you can't buy it anywhere, I know a lot of these things existing, but sadly if you can't buy the thing and use it.... it doesn't matter how fast it is.... and it doesn't matter if it exists at all.
Memresistors are a thing, but manufacturing them at scale is still way beyond current technology which is still getting a grasp on nanotechnology. It is likely that an upcoming technology such as doping silicon with graphene will render them moot.
Retired chip designer here. :) excellent! Kudos to the design team. & thanks for the very well done chip vid.
In my life, I have seen computers with perforated card input...and AI processors...the incremental rate of technology progress is beyond words. Thanks very much for your outstanding channel.
Things are moving fast these days. When the chips are ready, someone will knock on the door for $7 trillion of chips to be delivery yesterday.
you really hold my attention. i'm 79 and i want to keep living just to see all this stuff come into our world, functioning in our world. so weird. i worked in computers starting in 1980. then in networking in 1990, at nasa mt. view, wide area with sterling software. it just doesn't stop.
z80 ftw.
@@JeffreyBenjaminWhite and a machine language almost identical to intels 8 bit cpu.
Take Dr Sinclairs pharmaceutical concoctions in a few years. That should buy u the additional time u need
Thank you, Anya. Neuromorphic computing is a fascinating field, and IBM achievement is truly a revolutionary progress in enabling general AI robotics development. Great vlog for acquainting us, your viewers, with this new and exciting technology. Thank you, bunch.😊
Possibile che non capiate che tutta questa "AI" nata per migliorare la nostra vita servira poi per imprigionarci?. Anche l'atomo era studiato per scopi benefici, guardate come è andate a finire. Svegliatevi!!!!
A direct comparison of IBM's northpole and brainchip's Akida would be very exciting. I would be very grateful if you could do that.
Hello! First off, I absolutely love your videos - they’re informative and super engaging! Your latest piece on anamorphic CPUs was a thrill to watch. Can’t wait for what you have in store next. I’ve got a suggestion that might add even more depth to the conversation. While the hardware insights are phenomenal, I believe delving into the software aspect could be incredibly enlightening. Understanding the interplay between software support and compilers is crucial for leveraging the full potential of these innovative hardware designs. It would be awesome to see content that bridges this gap, highlighting how software ecosystems evolve to keep pace with hardware advancements. Keep up the fantastic work!
chip makers have leaned on node for so long but starting in 2030, the leaders will be based solely on creative design. we are gonna see some insanely smart design that never would have been thought of if they could have continued to rely on node shrinks. this is ESP true in graphics and ai. cant wait to see how they solve problems when they cant go smaller
Completely agree!
will the smart design be a thing beacuse of the people or cause AI designing it?
Moore's Law definitely made the chip design industry lazy and complacent, they let processor architecture innovations stagnate while they made the fabs do all the hard work of actually making computers faster through node shrinks, even though the limitations of Von Neumann architecture have been known for literally decades.
I think we are gonna see incredible things in out of the box thinking and I have my fingers crossed we will see more opensource designs and design tools once node changes arnt changing the rules ever 2 or 3 years. @@nekomakhea9440
Moore's Law has been outpaced by Huang's law already, people don't give Nvidia engineers enough credit.
Your Awesome Anastasia, thank you for your research and presentations. Your much appreciated by many.
And she's very beautiful as well 😍 🙂
@@bass305-HCCA the simps in the comments section is insane
@@julesvern-u4e so are the trolls
It’s extremely impressive thank you for the insight and information.
As someone who has started my IT career in IBM Mainframes Z/OS, COBOL & CICS, I'm ecstatic too. IBM, once a behemoth sort of lost its footing. Im happy its clawing its way back and that too in hardware, rightfully so.
Great presentation. Is there a reason Akida Brainchips neuromorphic chip was not marked against North Pole?
This sounds a lot like state machines. The fact that there is no distinction between the computing function and the results storage is new. It is long over due as an approach for sure.
Thank you for another excellent video. I'm excited to see what type of performance IBM will get out of this technology when they scale it to the 4nm world.
Thank you for shedding the light on this amazing ground breaking (and mind blowing) achievement from IBM. Needs more PR from IBM. Seems like nVidia is all rage these days with their specialised AI chips.
IBM have got industry customers, and they own the industry on a multi decade lock in process.
pretty exciting stuff, thanks for the excellent video Anastasi!
Woah, soo cool. Taking inspirations as a fresh ee graduate from these videos.
Fabulous presentation, giving race car feel for the exciting chip race that is now on going. NorthPole is super impressive, but unclear to me how far away it is from becoming a commercial product. As is it seems way too big for edge applications and although it could presumably be made in 3 nm or similar it would still be large, suggesting IBM will have to find ways of making it much smaller to use in edge applications. Super interesting yes, but not yet a practical competitor to Nvidia as far as I can see. Thanks for sharing!
Is Northpole more efficient than Brainchip's chip, do you think?
I can't find any specs on brainchip's Akida chips, but they've been claiming to be more energy efficient than anyone else for a while.
Edit: I wrote this comment before I got to the part where you covered Brainchip's Akida chip... But still the question stands.
It’s hard to foresee how much potential this holds for AI. I mean if it is already 20 times faster then the H100, what might be possible if we go down to 5 nm or less. People always speak about the end of Moores law, I think for AI mores law (at least in its classical formulation) isn’t all that relevant any more. Nvidia already says they manage a factor 1000 in 5 years when it comes to AI. Given that we may be rather close to AGI (many see it happen 2025 -2029, Kurzweil once used to be called an optimist) let’s see how it can improve it’s own hardware (not to mention it’s software). I think we have exponential times ahead that dwarf Moores law
Well said mate. Completely agree.
Well it's not 20x faster just yet but it would be under the same conditions
I watched your review on IBM's NorthPole chip and not only liked it but believed it. I hope you're a genuine person and wish you well.
I've seen now a lot of your Videos and i must tell you that i like your unique dialekt very much. The first time i watched your Videos it sounds a little bit strange to me, but this is what makes you special.
After all, i like your Videos because one can see that your research is profound. ✌🏻
I worked with the previous generation and it's extremely interesting to train on. It performs off-chip learning so your data is stored off the processing server. Intel's Luihi are an on chip solution
Who are the engineers? They are lightyears ahead. Congratulations.
Great video as usual! How do you think NorthPole compares with what is known about Tesla's Dojo?
NorthPole vs Tesla D1 chip.
Dojo is the whole computer.
Lovely breakdown, thank you for sharing not just IBM's breakthrough but also the potential applications of others and the different paths of AI chip development that companies are taking. I think it's interesting regarding about real-time applications versus other applications. I could see a future where both are used in conjunction to create a very adaptable AI robot or sentience.
This is a great video explaining the new IBM AI chip, NorthPole, that uses a brain-inspired architecture to achieve faster and more energy-efficient AI inference. I’m impressed by how it can perform multiply-accumulate operations within the memory, eliminating the von Neumann bottleneck that plagues traditional chips. I wonder how it compares to other analog AI chips that use phase-change memory or resistive non-volatile memory devices. Thanks for sharing this informative and engaging content ty Bing AI
I think that for better power efficiency spintronics related technologies seems to be more promising like for example Intel MESO concept that seems well suited both for digital logic and neuromorphic computing…
the thing i love the most about this chip is how it's only using 74w which means it can be powered fully through a PCIe slot, the biggest obstacle i can see tho is software support. we can already see that with AMD GPU to an extent, they're very capable compute machines and often time have more VRAM than Nvidia counterparts at the same price but since they're not supported by tensorflow or pytorch they can't be used for AI easily
It seems like everyone is getting into the game of chips. While exciting to see, it looks akin to "throwing mud at a wall" to see what sticks and there are ALOT of variations that are "sticking", so much so that other teams are learning from what "seems to stick" and growing in unexpected ways. Pretty soon I'm certain that the right mixture is going to be revealed and start the next tech revolution.
IBM has been at this a really long time. They just have a problem with executing and profiting at scale on some of their best ideas, which subsequently get swiped by others. Case in point RDBMS (swiped by Oracle), affordable PC (swiped Microsoft) etc the list is endless.
This is NOT a complaint! I noticed your vernacular shifted after your description of IBM's chip. Oh it is subtle but words are my craft. As a native English speaker it seems to me that your words were different when talking about IBM and then they returned to your own after that. Could be due to your method of relating the information to us! Not like you read the segment about IBM. I would never accuse you of that! I admire your technical expertise greatly! It's more like your presentation is based on a particularly vivid memory of your research into the new IBM chip! I study/speak a different language than English myself... I suck at it!! Your English is fantastic! Always good to raise the bar! Best wishes!
Thanks for the (as usual) very informative video! Question: how does it compare with the chips in Dojo? Maybe Tesla should become a good IBM customer 😀
Good one! It is more for an edge device while Dojo is for a supercomputer. Though, in Dojo memory is also close to computing units.
@@AnastasiInTech maybe Tesla can give a push to IBM to put them in production as soon as possible, as you said it's based on scalable and mature technology 👍
The first thing I wondered looking at the comparison. Thank you for asking!
Tesla is working on a newer chip (D2 ?).
Well constructed, informative and substantial piece. A pleasure to view and follow the presentation.
Anastasiia before i comment on Chip design... I would like to say... You look amazing... AI is being held back by conventional compute parameters and constructure material availability AI has to have full control on all phases of the chip design and once AI stacks overall design parameters from material to automation building consensus. But AI is bottlenecked... So only AI can solve all concepts of the Designing. Then and only then will life changing AI chip design be revolutionary. Great topic ty Anastasiia ❤️❤️❤️
As always great video Anastasi!
Glad you enjoyed it!
I love your content, but I missed the rolling credits of your donors with a cool music at the end
I remember the North Star from yesteryear. Great to see IBM at the forefront again. I have always thought that bending GPU ray tracing pipelines around to do more general computing is fundamentally inefficient. That 25x efficiency multiple underlines that point.
The spiking NN without central clock signal should yield more savings still.
Great segment, Anastasi! Thank you.
Very interesting presentation. Combining memory and computing power makes me think about the concept of "work space" where you put your code and your data in the same RAM memory. This is going a step further by putting the parallel processing power as close as possible to the code and the data. Thank you.
Thank you, Anya！
Full dive VR, here we come.
I had nothing to do with this but was a proud IBM employee for 6 years. Great company.
Our brains are analog; the concept has already been tried and tested.
Hello, I am an also Electrical Engineer and Ilike your interesting content and it is very useful to progress my knowledge in hardware, computing and AI!
Thanks for this content!
Thanks for the review. Excellent. A limitation of the Neuromorphic chips in the data Center is that most of them are not that impressive in terms of Training, even more so if one wants to test different models to see which performs better.
I’m thinking can you make a list of these neuromorphic architecture chips makers? Or maybe I can google it somewhere🤔
Great summary and great news that IBM is leading the way in neuromorphic processors. Maybe the ultimate solution will be a combination of analog and digital technologies.
@RajGandhasri Congrats!, as an outsider may I know what the inspiration was for designing and fabricating NorthPole with Compute+memory matrix, seem like distributed computing is the trend for systems design. Thanks, Anastasi, for covering it in a video.
Great video, thank you!
What IBM is doing there, as you describe it, seems to me surprising and impressive. The first processor I ever got inside of was a discrete logic 11-bit instruction word, dedicated unit. So much progress.
I'm sure I only understand half of what you're saying but the half I do understand is fascinating. Thanks!
I always learn so much in your presentations, you provide information DENSE thoroughly researched and totally understandable videos.👍 kudos.
Yeah i was looking at the tsp and thought that was dope too.
It is important to put the "should do" between "can do" and "will do." That was the difficulty IBM had in the past... and they chased a lot of "can do" without knowing the formula of "should do" and profitability.
IBM is on streak right now being more successful than ever. They have successfully transitioned into 21st century model focusing on SW, consulting, infrastructure with plenty of patents, research, their own processors and tech including Quantum. IBM is growing!👍
You m'lady, earned an instant sub. This video was exactly what i wanted to find and worth more than one view
Great video! Do you have any clues about comparison against Dojo?
honestly, 2 of these chips have the same through put as a H100 GPU being 12x more energy efficiency, so I think these could genuinly be used in a scaled up system to make an industrial large neural network training computer
Another amazing video thank you for sharing. Thank you for the summary of other neuromorphic compute solutions. I've been following Sony's work on neuromorphic camera sensors but wasnt as aware of these developments.
Worked for IBM for years and retired in 2016. Nice to see...
What are the drawbacks of neuromorphic chip comparing to the "standard" architecture, the one used on the Nvidia chips? Why does the Intel specialist say the "standard" architecture will be adopted for a long time in cloud computers ? Is it because of the price only ?
Wow how did they knock this out of the park so far compared to everyone else!!!
I enjoy your presentations.
Ricky from IBM
I've been looking for something to do with language models for a while now, like llama2. But as far as I understand, no matter how good the models are, unless there are algorithms that will make the learning process better, this work will take a lot of energy, money and time. The complexity and amount of input required to train the model is enormous. The models take the human brain as an example, but I don't think humans need that much data to learn. The problem here is still not understanding the human learning process or not being able to make algorithms accordingly. Most people live 70-80 years using about 800 words a day and knowing 20-30 thousand words :)) I think analog chips can create a new opportunity. good posts.
IBM is also the future of Quantum Computing, Annealing etc too as far as I understand. ❤
This reminds me on SyNAPSE, a backronym standing for Systems of Neuromorphic Adaptive Plastic Scalable Electronics. A DARPA Project for asynchronus neuromorphic computing of the early 2000s.
As a retired ATT systems technician, I worked on cord switchboards in the seventies supported by electromechanical 701 step by step switch gear. By 1986 we were putting in System 85 with timed multiple switching and fiber optics connected the remote nodes between buildings A RAM board was 64k and when we saw a 256k board come out soon after we would tell people it was the FM system (Fucking magic). Lol. Fast forward to today and the exponential advancement in technology is unbelievable. Good Job IBM. I’ll bet you guys did all this not even wearing a suit and short haircut they demanded back in the day.
This is great and all, but the power consumption for TrueNorth was a puny 65 mW.
So NorthPole has increased perf by 4,000x while increasing power consumption by over 1,000x too.
Making the actual gain closer to 4x in perf/watt over TrueNorth.
That is still a gain for sure, but all the same it doesn't seem like as big of a gain as I would have hoped for considering they were known to be researching replacing SRAM with MRAM, PCRAM and likely other types of persistent non volatile memory that could make a neuromorphic chip far more efficient (and potentially smaller too).
Thanks ! Memory inside the GPU , we have to see the chip in action . Definitely a solution. Is it open source the software stack ? Or Proprietary closed source like Nvidia? So far Intel and AMD are open source for their new GPU's. In my opinion very important . Are they planning RISC-V ?
Looking forward to seeing some application. 😊
Disappointing that more Neuromorphic chips weren't included in the report... Really feel this should be an open source data gathering project to provide an honest benchmark against all manufacturers.
These are the major players. Also at what number do they stop? the paper has to have a limit.
@@MuantanamoMobile There really aren't many neuromorphic chip manufacturers with real products out there... Not yet at least. When do you stop? In research, til no stone lies unturned.
I have no doubt about that's the decade of first emerging and growth of AI and any tool related to! I'm amazed of how different things will starts to become.
Always love your seeing your videos!
Well ....look like Gradient might be a good start for me but as I reorganize I to am switching/starting something new this quarter. I want to begin on AI projects now because their pillar-like position is key in business for one app/web site I am working on. Gradient offers an independent build but I wanted a more professional hands-on help with this build. But it's on my mind.😅 Thanks
So many steps involved…
Design
Build
Pilot production
Commercial production
Chips are now requiring software suites to support the various functions
Does IBM have the software suite to support the new high speed more efficient chips, or is this still a ways off in the future?
Other chip manufacturer presentations highlight the suite of software that supports their customers…
Nice presentation Anastasi, thank you!
😃
a whole different architecture, when i became a computer engineer pentium 4 was new
Thank you Anastasi! You are a good teacher!
Fascinating video!
I assume the memory on chip is quite limited. How would it scale for for daily things like LLMs?
'My brain has no chip memory....at least for now.' 🙂
2:15 - video says the chip can be scaled down from 12nm to 3 nm. But if the design contains SRAM cells, the memory itself will not scale down. Or has this issue been resolved?
That's interesting and logical that IBM would be developing and researching new software AI frameworks as it tries to catch up to TSMC in the hardware foundry side.
But, I suspect that not all is roses with what IBM is doing. AFAIK one of the strengths of the nVidia ecosystem and the AMD support for CLI is that they're framework agnostic except for their chosen language support so support practically all coding and applications. I'm guessing this isn't the case for IBM's approach... If IBM's product history is a guide, IBM is likely achieving those efficiencies and performance by hard coding long functions into silicon, and that typically means that only certain software coding and software functions are supported which would likely mean that certain types of applications can benefit from IBM chip features. An example of this is in the CPU world where IBM has long been a leader, IBM CPUs are generally best for machines used for business and typical business applications and less for scientific research and raw number crunching which is why GPU computing and graphics have been largely built on ARM chips.
It's for this reason I wonder if the described IBM chips would be suitable for machine learning and neural networks despite the brief mention of neural networking in this video... If neural networks might be supported, I'm guessing only in a very specialized field. I suspect that if I'm right about IBM's chip architecture that these chips might require a lot of handcrafted AI created by human AI scientists. Or, IBM would have to create its own meta layer sort of like the .NET runtime that translates human coding into optimized machine language.
In your most humble opinion, what would you say is the "next best" currently or soon to be interconnect? I rabbit holed into gen-z consortium projects and am intrigued.
Asynchonous or "no synchronous" is an important aspect to consider. It can say compress a century of high speed sample data to produce and "answer" in a bit or a byte or bytes. It allows so many assumed rules to be discarded or alleviated towards progress to quality of information in efficient time domains. To include "feedback loops" to both sides of the underlying way a program can re-program itself, while including ways to incorporate "vectors" in time and in space. Any mix/match use of these concepts at any level will drive efficiency, and produce answers to create better questions. It is better to know an answer before one need consider the questions. Gr8! Peace ☮💜
So, is IBM Northpole clip still has competitive advantage when GB200 is out?
New subscriber!
Wow! This is a good news!
I would hope this is the start of something new in the industry. I am glad IBM is still a R&D juggernaut. When IBM makes advances we all benefit from it.
Can you post an update video on this subject? IBM stock has been setting new highs since August and so I am wondering if they may have something in the pipeline to compete with NVIDIA.
Even though I only understand about 25% of what she's saying.
I'm thinking about how proud her mom and dad must be. To have such a smart daughter. Thank you for what you do. I look forward to these presentations. Because I get a little something out of all of them.
Thank you ☺️My Mom read your commend and sent me the screenshot
@@AnastasiInTech Thanks 🙏 I appreciate you taking the time to respond.
Blessings to you and your family...
I remember Clive Sinclair talking about processing in memory chips back in the 90s.
Great video! This IBM chip reminds me of the good’ol Transputer chip architecture available in the late 80’s, rebuilt using current digital technology advances. Reckon Neuromorphic chips will totally smash GPU chips (and obviously CPU chips) in the AI domain. Also, I first saw a software demo of Spiking Neural Nets by its inventor in 2000, and this tech is astounding. Just wait and see, although seriously don’t think BRN will lead this…. definitely far too slow. It’ll be someone else yet to emerge.
It wouldn't smash anything, I hardly believe IBM are more advanced than Nvidia in AI chips. Google tensorflow chips were also impressive and faster than anything Nvidia had, when they were first introduced but Nvidia did catch up and surpassed them. Nvidia H100 is the fastest AI chip at present on the market. On another note Google does not sell their chips, they offer them only through their "cloud solutions" .
@@Slav4o911 Nvidia is definitely up there atm, however learn about Cerebra’s chip, which uses a Neuromorphic architecture. Already higher performance than Nvidia for AI. Nvidia uses a general architecture which is great for matrix maths, but Neuromorphic architecture is a custom built for neural network processing which is in some ways different. However, neither of these chips are custom built for SNN processing, which is different again.
I have trouble conceiving all the internet on one chip. I am talking about the memory being on the North Pole chip. It can pull this from memory much faster like the cache used to be on the main processor. How many tokens is that? The comparisons from all the different chips and chip companies is looking good on the charts, I just don't know what the values represent.
Nice Accent, even better knowledge. Now i have some tech to look into. After HBM and Ryzen, chips got kinda boring for us outsiders
This is the second time IBM has come up with revolutionary chip design. Most might not remember the Cell Architecture that was suppose to bridge the gap between a CPU and GPU and was used in Sony PlayStation 3. But it failed because it required totally new approach to software development.
With the kind of working that this chip is presented with, we should expect a similar complication in software development for this chip as well along with some other problems that we might have not predicted.
Anyways, it is amazing development none the less.
This is such awesome technology. To a degree, I almost see (for example) AMD moving more memory on-chip. Obviously it's not exactly the same as having the memory right near the compute elements in the design, but I guess what I'm getting at, is that there seems to be a trend towards moving memory more "on chip" so you don't require external busses as much or the "hop" to increase latency is being reduced by bringing more of it on die/at least in a nearby chiplet with a much faster interconnect than is possible.
You know, for reasons like signaling integrity, error correction and latency. AMD has done a lot more of this with the much larger L3 caches on some of the new EPYC chips. Apple is also doing this with their Apple Silicon line of CPUs by bringing the DRAM package directly onto the CPU package, and their own custom interconnects for ultra high bandwidth/low latency. I suspect we're going to continue seeing a shift towards more and more memory being crammed directly onto the CPU package, or even into the die itself.
For now, I suspect "external" DRAM (external to the CPU package I mean) is still going to be around for quite some time because of physical limitations. I'm not sure 3D stacking technology is a proper solution just yet because the more layers and the increased density create problems with feeding power into the package, and also with thermals.
What would be cool is to hear about any further developments with on-die liquid cooling. I remember a while ago it was being explored to have layers cut directly into the silicon die for water channels, apparently the thermals were far superior to anything that can be done with traditional cooling methods. If this is done in a 3D stacked manner, then perhaps being able to cram gigabytes (or even terabytes) of DRAM directly into a CPU die or onto the package using a ultra-high speed interconnect would be far more possible.
I think the more competition. Breeds innovation. And that's the exciting part Technology.
"memory compute" is a little misleading
there are no computations done in the memory but because it is co close to compute unit it can perform much faster and energy efficient because it can move data faster and spend less energy on doing it
That’s right!
hey Anastasi, great work. Would love to see a video comparing how these chips compare to the one George Hotz is creating and how do you see chip manufacturing getting accessible to smaller firms & teams. sort of a democratization of hardware biz.