From my understanding: You need to design physical chips that perform similarly as brain neurons where each neuron has its computational part and memory colocated?
nah, this can be done using flash memory where electrons/charge is stored in a potential trap, just instead of using discrete values like in 3d-nand, you use continuous charges to represent varying values of G
but how are the G Variables with their different values set into the matrix? I guess they need to be modified when training the neural network so they can`t be set by lithography when making the chip. But how is it done afterwards when working with the chip?
this is done using flash memory where electrons/charge is stored in a potential trap, just instead of using discrete values like in 3d-nand, you use continuous charges to represent varying values of G
Yes, it is RAM. During the forward propagation process (and backwards likewise) computations are performed -as usual- by the ALU, so it implies fetch-decode-execute steps that inevitably rely on the buses mentioned in the first diagram. Some limitations can be overcome by GPUs thanks to heavier loads of data and parallel buses between memory and the streamlined multiprocessors. Still way slower than doing everything locally though.
As far as I understand, using flash memory for this, you can only use positive charge to increase resistance, so you would need to normalize to remove negative values
wait, is she actually writing in reverse or it's being reversed in real time or something? My apologies if it's something common, there were no digital boards back when I was in school.
🤔 I am thinking about maybe it is possible to replace those resistors by digital potentiometers in rapid prototyping circuits to expirment with them, as a hybrid between a fpga with resistors, or include already the part of the predesigned circuit: with transistors and these to the resistors. The rest how the memory, demux, D/A, A/D, etc circuit would be designed by the engineer or student.
A digital potentiometer has typically 5 to 8 bits controlling 31 to 255 resistors respectively, that is 0.3922% resolution for 8 bits, and analog potentiometers offer 2%. In this way AI systems can be designed for home use or small applications at a very low cost. If they are made of 10 bits would control 1023 resistors that would have a resolution of 0.0978%.
Probably not, a nice trick is having she write the normal way, but then flip the video horizontally, if you notice it *seems* she's using her left hand, which is statistically un-probable. but if the video is flipped horizontally then she's writing with her right hand. in this other ibm video this person also "seems" to be using his left hand, but if flipped it would be his right hand. so.. ruclips.net/video/b61DPVFX03I/видео.html
Watched a doctor explain sugars & honey just a hour ago using clear glass{?} & various colors markers writing backwards like this Chancellor Lady,,,man i want to know what is the trick
Very late reply, but the video is probably mirrored left to right. You can see that her ring is on what would be her right hand if it isn't flipped, and that she also appears to be left-handed if it's not flipped. Both of those are possible without a doubt, but it's more likely not
Great Overview👌, I never get tired of watching these videos. There's always something new to learn and gain
Excellent explanation Dr. Nicole Saulnier. Proud to work with such a brilliant team of Research Scientists like you.
This was a great explanation of in memory computing.
Excellent.
Have heard about in memory compute earlier, but came to know the details here.
Such a detailed explanation. Thank You.
From my understanding: You need to design physical chips that perform similarly as brain neurons where each neuron has its computational part and memory colocated?
great explanation❤
Nice description!
the lady is more clear in her explanation than my lecturer.
How registers i.e W weight values can be changed dynamically? Something similar to FPGA?
nah, this can be done using flash memory where electrons/charge is stored in a potential trap, just instead of using discrete values like in 3d-nand, you use continuous charges to represent varying values of G
Nice video
but how are the G Variables with their different values set into the matrix? I guess they need to be modified when training the neural network so they can`t be set by lithography when making the chip. But how is it done afterwards when working with the chip?
this is done using flash memory where electrons/charge is stored in a potential trap, just instead of using discrete values like in 3d-nand, you use continuous charges to represent varying values of G
analog computers are back
Would be great if there is a video that explains stateless vs stateful?
Thank you for the suggestion! We're looking into it. Don't forget to subscribe so you'll know when we publish "Stateless vs Stateful explained."
What do you mean by “memory”, (in beginning of clip) give an example.. RAM?
Yes, it is RAM. During the forward propagation process (and backwards likewise) computations are performed -as usual- by the ALU, so it implies fetch-decode-execute steps that inevitably rely on the buses mentioned in the first diagram. Some limitations can be overcome by GPUs thanks to heavier loads of data and parallel buses between memory and the streamlined multiprocessors. Still way slower than doing everything locally though.
@@thelonespeaker Actually, RAM uses more power than flash, as flash can store weights (G) with no power
I can't understand, what is stored in such memory? Is V or G a variable?
Ok. From the site in the description I understand that G is variable.
How does it work for negative weights?
As far as I understand, using flash memory for this, you can only use positive charge to increase resistance, so you would need to normalize to remove negative values
how far is IBM with in memory computing?
I wish I was smart enough to enter the AI field, this is the future.
it's never too late bud
Using analog logic to compute on memory. Interesting.
wait, is she actually writing in reverse or it's being reversed in real time or something? My apologies if it's something common, there were no digital boards back when I was in school.
See ibm.biz/write-backwards
@IBMTechnology Wow, that's actual clever and simple. Only reinforces my belief of best solutions are often the simplest ones.
How is she writing that way
Just write as usual, then flip the video
🤔 I am thinking about maybe it is possible to replace those resistors by digital potentiometers in rapid prototyping circuits to expirment with them, as a hybrid between a fpga with resistors, or include already the part of the predesigned circuit: with transistors and these to the resistors. The rest how the memory, demux, D/A, A/D, etc circuit would be designed by the engineer or student.
A digital potentiometer has typically 5 to 8 bits controlling 31 to 255 resistors respectively, that is 0.3922% resolution for 8 bits, and analog potentiometers offer 2%. In this way AI systems can be designed for home use or small applications at a very low cost. If they are made of 10 bits would control 1023 resistors that would have a resolution of 0.0978%.
Is she writing backwards?
Probably not, a nice trick is having she write the normal way, but then flip the video horizontally, if you notice it *seems* she's using her left hand, which is statistically un-probable. but if the video is flipped horizontally then she's writing with her right hand.
in this other ibm video this person also "seems" to be using his left hand, but if flipped it would be his right hand. so..
ruclips.net/video/b61DPVFX03I/видео.html
Watched a doctor explain sugars & honey just a hour ago using clear glass{?} & various colors markers writing backwards like this Chancellor Lady,,,man i want to know what is the trick
Customer service chatbots aren't a good example to evoke a positive reaction 😑
Looks like we did indeed move on. Elon Musk and others saying stop lhe lamas.
Did she learn how to write backwards, just for this video or am I missing something?
Very late reply, but the video is probably mirrored left to right. You can see that her ring is on what would be her right hand if it isn't flipped, and that she also appears to be left-handed if it's not flipped.
Both of those are possible without a doubt, but it's more likely not