My favourite part of any good TED talk is when something so mind blowing is presented that the audience doesn't know how to respond. He mentions around 13:20 that the following tech is a few months old and then does magic. The reactions from earlier compared to this segment...
I am literally speechless from the ingenuity of this research. This is absolutely incredible! This is what we call out-of-the-box think. It was right there infront of all of us, always but it took a different perspective and a curious mind to morph it into an entirely new thing we didn't even know was possible. This is research at its finest. What a great presentation too!
Incase this hasn't been thought of already. Point a ridiculously HD, telescopic, long exposure camera (Hubble) out into the universe, use our knowledge of redshift etc to determine distance, then use these algorithms to better estimate Density and movement of planets without the need for worrying about gravity and it's inconsistencies. We could detect the size, shape and velocity of a potentially life threatening asteroid many years in advance just from a few thousandths of a few million pixels. We could interpret the sound of the universe expanding and listen to it as music. Also, on the micro level, we could use microscopes to listen to bacteria, virus's and cells and learn from a completely different sense, just how this world operates at that level. There's so many things to learn from this amazing invention. Maybe with powerful enough cameras that pic up more than just the small bandwidth of the electromagnetic field that we see, maybe x-rays, inferred, UV and powerful enough interpretive algorithms, we can learn the resonating frequencies of certain things, like diseases, bone defects and brain functions... Maybe we'll be able to record our brains and listen back to our very own thoughts in our minds voice. The possibilities are endless.
We already to this but with eletromagnetic fields. I see what you have tought but light is a problem here. It would take a lot of time for this light to come to us and even to record with it
dowddash and the film itself would shift around when it reeled thru the camera causing a false positive for the software to analyze. but it would be worth trying.
pHaTdProductions If it did work that scene from Wayne's World popped into my head, where they're ordering fast food and only partially saying words. =p
I think this is the third TED talk from this group. The first three minutes came from their first talk. The first and second talk included a section on color change amplification. The rest up to the time stamp given was the second talk. Glad I stuck around for the last part which was new and what the title was talking about.
MrC0MPUT3R Some of it, yes. The parts where he shows it to be able to simulate object structure, is the new feature, the other stuff was already mostly shown in a previous presentation.
MrC0MPUT3R What are the chances? I'm a subscriber and I've seen this before too. But, just a few minutes ago, I was thinking of finding it again to forward to someone (to demonstrate that not all TED talks are just one person droning on and on). Well, now I don't have to look for it. :)
Nope! Hold on. This is a different video. I saw two on this subject before. The first was just about magnifying subtle changes in the video (the first part of this video). The second was that plus the ability to use those magnifications to record sound (the second part of this video). Now, this one has added the interactivity work. Their work is obviously advancing rapidly.
Absolutely wonderful! Question: what about filming a plant without any additional stimulations, to observe any sounds the plant makes thru its own vibrations? Maybe its singing it's own song.
In this engaging talk, Abe Davis reveals the new technology that shows the hidden properties behind every object. It's a path-breaking video technology that could potentially alter the audio-visual medium in a big way. He also shows the amazing behind-the-screen experiments that is awe-inspiring. Highly recommended to those who are in the media, TV and Video production.
For those who may not know, A.R.I.I.A (or just "ARIIA") is an Artificial Intelligent (A.I.) computer/robot/mecha that appears in the movie "Eagle Eye" (2008), and in that movie, she was able to recover sound from video by using the vibration sources from a coffee cup - essentially doing what is shown in this video! NOTE: Many note that this is highly exaggerated, because the camera she uses would not be fast enough to sense the vibration frequencies! 🤙
CIA/NSA (and Google and Facebook for that matter) are probably busting down this guys door trying to adapt it for their purposes. I don't think they would do such a thing you say? Our privacy still matters you say? If you believe that way, I find that uproariously funny. So naïve..'
The work is great, but let us not be over enthusiastic. I'm no expert in this field, but I can sense immediately some hidden issues. 1) All the wonders here largely depend on the super-precision of the special cameras. For a regular camera at a reasonable distance, I suspect the noises will overwhelm, and it will become a problem like chaos - undeterministic. 2) Highlighting pulse sensation in this visual way is really a fake exaggeration which does not correspond to reality; it's no more than the authors' personal interpretation. 3) I suspect the sound detection is done through checking the resonance frequencies on the leaves via the pixels or shapes, and is fundamentally prone to massive noises; radar type of detections will do infinitely better. 4) Visual extrapolation of a bush movement does not imply its feasibility. The visual detection of those tiny movements serves as some sort of initial motion vectors which can be utilised as the salt or seed for visual extrapolation. So it's a good work, but it is not really shocking to me.
But then also what if you took it backwards, having the mathematics of how things interact with vibrations and reverse-engineer an image/structure/model. At large-scale, you could get an image pf things that we can't see for whatever reason. Assuming technogy ever got so far as to really record the sounds of space/our satelites went further you could literally construct visual structures of stars and stuff. But if we even use sound to get images, maybe somehow we can use the same sounds to give us readings on structure. What if we could look for habitible planets by knowing how sound reacts with water, and then grabbing the data on what it sounds like on Earth, and use that understanding to be able to distinguish water on other planets. Idk. This is great stuff!
This technology maybe the beginning of being able to see realities we haven't even imagined. For example we know already our senses don't measure up to a lot of other inhabitants of this world. Animals with better sight, smell, hearing, and ability to make sounds. Elephants can make a sound that can be recorded from two miles away.
I would be curious to see 1 or 2 minute exposure of micro-organisms growing in culture. There has already been some work done on yeast moulds that suggests some sort of cognition or coherent movement.
I still don't get how they could extract audio data from a two dimentional video recording... He didn't mention the impact of the audiowave on the whole target spoken to (NO 3D depth info)... tbh I am having doubts about all this...
This kinda reminds me of one of the first ever "telephones" which used light to code information and then it was shined on a piece of paper which vibrated
Nunya Bidness That may be a very real thing. Though I imagine you would need to have a similar event filmed. Or maybe I'm just under-estimating the technique and all that'll be needed is to set a camera outside pointed at a skyscraper long enough to see. Perhaps in time we'll inevitably get there. Not sure if we'll at all need data of stress points of certain materials to fully replicate what'll happen beyond a comically wiggling building.
Using this, can someone.. say, record my house.. and then use this software to see the minuscule vibrationson the walls, and find out when I take a fap break?!!!!!!!
So you could take an old video and interact with the surroundings after processing with software. Awesome. Not worried about the spying implications.. We're already well past level 9000 there with the PA
Wow, this is classic MIT stuff! The kind of thing one imagines when MIT comes to mind :) Though it does raise lot of concerns about privacy, for example getting security credentials from a guy using a keyboard will be so easy ;)
Amazing... Given enough computing power it can also capture sounds of past, I think. Probably the Bhagwat Gita Pravachan by Krishna, Jesus Christ talk of past is not beyond present... That's the way I guess Yogis were all knowing as probably through enlightenment they would have developed keen and powerful perceptions. No I am not joking. __ Shashi Om Namah Shivaya __ Shashi Om Namah Shivaya
The last part is a very practical use and probably the second popular use of this technology, the first being the remote medical use, but the one I am talking will be the most far reaching use of the technology in terms of snooping and corrobative evidence of things happened in past. The most advanced use of this technique, I guess. _ Shashi Om Namah Shivaya
Bro, Looks like there were others before working on recreating audio from images... "Advancements in audio recovery technology have made it possible to hear them again. In May 2011, scientists, for the first time, used a scanning system called IRENE-3D to play back an Edison doll’s voice singing “Twinkle, Twinkle, Little Star” without ever touching its tin record cylinder. IRENE-3D works by optically scanning a phonograph’s surface and digitally replicating the sound it would thus produce." @ blogs.discovermagazine.com/d-brief/2015/05/05/creepy-edison-dolls-speak-again-with-new-technology/#.VUl2DEJIBUQ Cheers!!! __ Shashi ॐ नमः शिवाय Om Namah Shivaya
This will help solve crime in the future, imagine being an investigator and using this technology one day to take video and try and extract sound in a place were a murder occurred.
but you would have to be recording while the murder is happening...and if ur going to do that you might as well just use a microphone...this goes far beyond that as far as its uses...like the dierctor being able to alter a shot after its been filmed...really amazing stuff!
Hello, I am a postgraduate student who needs your data set in my research. Can you tell me the location of your data set, and I would be very grateful to you
+adavidmon exactly what im thinking, there will be no need for model creator, or graphic design. Which cuts down the video game budget, but will also cut their job :(.
I don't understand, they are acting as if this is new...? Military has has devices for years that can monitor sound (voices) from the vibration of windows of the room the people are talking in. (Hence picking up the audio from vibrations.) using video would only enhance that more I guess. But my point is, this "technology" has existed for a while now
My favourite part of any good TED talk is when something so mind blowing is presented that the audience doesn't know how to respond. He mentions around 13:20 that the following tech is a few months old and then does magic. The reactions from earlier compared to this segment...
I am literally speechless from the ingenuity of this research. This is absolutely incredible! This is what we call out-of-the-box think. It was right there infront of all of us, always but it took a different perspective and a curious mind to morph it into an entirely new thing we didn't even know was possible. This is research at its finest. What a great presentation too!
Incase this hasn't been thought of already.
Point a ridiculously HD, telescopic, long exposure camera (Hubble) out into the universe, use our knowledge of redshift etc to determine distance, then use these algorithms to better estimate Density and movement of planets without the need for worrying about gravity and it's inconsistencies. We could detect the size, shape and velocity of a potentially life threatening asteroid many years in advance just from a few thousandths of a few million pixels. We could interpret the sound of the universe expanding and listen to it as music.
Also, on the micro level, we could use microscopes to listen to bacteria, virus's and cells and learn from a completely different sense, just how this world operates at that level.
There's so many things to learn from this amazing invention. Maybe with powerful enough cameras that pic up more than just the small bandwidth of the electromagnetic field that we see, maybe x-rays, inferred, UV and powerful enough interpretive algorithms, we can learn the resonating frequencies of certain things, like diseases, bone defects and brain functions...
Maybe we'll be able to record our brains and listen back to our very own thoughts in our minds voice.
The possibilities are endless.
We already to this but with eletromagnetic fields. I see what you have tought but light is a problem here. It would take a lot of time for this light to come to us and even to record with it
You dont hear the bacteria or virus, your hear the surrounding of it and the source video would be the material where the organism its.
Or just at the human scale; point it at machinery in factories or vehicles to predict points of failure due to tiny vibrations.
@@WilcoVerhoef Well noted! More on that application, admittedly not with the same technology, here: ruclips.net/video/rEoc0YoALt0/видео.html
Best TED talk on tech I've seen in awhile. Great scientific work with enormous potential!! Well done.
I wanna hear sound from a silent film back in the day.
depletable Wouldn't be possible. Somebody said something about there not being enough frames per second to get accurate measurements.
dowddash and the film itself would shift around when it reeled thru the camera causing a false positive for the software to analyze. but it would be worth trying.
pHaTdProductions If it did work that scene from Wayne's World popped into my head, where they're ordering fast food and only partially saying words. =p
Damn. You don't think's a more developed algorithm that would be able to continue improving? I mean dude could use a cell phone camera.
depletable Did you pay attention to the video? If the quality of the video wasn't good enough (number of frames and pixels), then it's not possible
We've seen this video technology used to retrieve sound in another TED talk. But a new (and impressive) application is revealed at 12:16
Maxime Isabelle Was about to say a similar thing.
I think this is the third TED talk from this group. The first three minutes came from their first talk. The first and second talk included a section on color change amplification. The rest up to the time stamp given was the second talk. Glad I stuck around for the last part which was new and what the title was talking about.
Maxime Isabelle Thanks
Yeah, guys just rehashing stuff from other TedTalks.
Somebody had mentioned that this was done in the 2008 movie with Shia Leh Boof, titled, "Eagle Eye".
Only 129,158 views for one year for this amazing official TED video ?
Something's wrong with the youtube users.
Who's here from Veritasium? Super cool stuff
One of the best Ted talks I've watched
Wait... I've seen this talk before...
MrC0MPUT3R Some of it, yes. The parts where he shows it to be able to simulate object structure, is the new feature, the other stuff was already mostly shown in a previous presentation.
Devil's Advocado Ah so I'm not crazy. Too bad... I was looking forward to becoming the next Nostradamus.
MrC0MPUT3R What are the chances? I'm a subscriber and I've seen this before too. But, just a few minutes ago, I was thinking of finding it again to forward to someone (to demonstrate that not all TED talks are just one person droning on and on). Well, now I don't have to look for it. :)
Nope! Hold on. This is a different video. I saw two on this subject before. The first was just about magnifying subtle changes in the video (the first part of this video). The second was that plus the ability to use those magnifications to record sound (the second part of this video). Now, this one has added the interactivity work. Their work is obviously advancing rapidly.
Absolutely wonderful!
Question: what about filming a plant without any additional stimulations, to observe any sounds the plant makes thru its own vibrations? Maybe its singing it's own song.
I wonder what kind of sound it could extract from old old silent videos .... probably not enough frames per second ... but I wonder
I would LOVE to see this applied to some of the Mars Rover video footage!
so if a tree falls in a forest and no one is around to hear it and you film a near by bush with a high speed camera does it make a sound?
Could silent films if the quality was good enough be analysed in such a way so we could hear all the ambient noise? That would be freaky weird.
The interacting technology with just recording is just awesome work. Guess this can be modified and helpful for "space" research as well.
Where to download the Video Microscope Amplifier?
This is an absolutely amazing discovery and is very inspiring! I never even heard about this until today. This was an incredible TED talk!
I'm impressed a third time. This work is like a gift that just keeps on giving.
This is pure genius, this is going to implemented into Virtual Reality tech for sure.
4:23 that MAN who laughed 😂😂😂😂😂💀
4:43 part 2 💀
This is still the best talk on TED.
would this work to study microscopic organisms? or materials on a microscopic level?
In this engaging talk, Abe Davis reveals the new technology that shows the hidden properties behind every object. It's a path-breaking video technology that could potentially alter the audio-visual medium in a big way. He also shows the amazing behind-the-screen experiments that is awe-inspiring. Highly recommended to those who are in the media, TV and Video production.
For those who may not know, A.R.I.I.A (or just "ARIIA") is an Artificial Intelligent (A.I.) computer/robot/mecha that appears in the movie "Eagle Eye" (2008), and in that movie, she was able to recover sound from video by using the vibration sources from a coffee cup - essentially doing what is shown in this video!
NOTE: Many note that this is highly exaggerated, because the camera she uses would not be fast enough to sense the vibration frequencies! 🤙
They've progressed a good bit since I last heard of it, good job.
EDIT: Wow...they really progressed.
SangoProductions21 *The singularity is **-near-** here.*
This is going to allow some very interesting material analysis.
Is there a way I can download the program used to interact with objects? Has anyone released the code?
How do you get data for a fraction of a pixel out of a video frame?
Nashy119 only the illuminate have the answer to that lol
Nashy119 You don't . They used unly changes that caused pixel change.
Truly mind blowing!
Reminded me the scene from movie Eagle Eye from the soundproof room (time 1:27:53).
The wire figure and bush simulations were incredible. I, too, think this technology has a lot of potential.
hmm if I recorded an animated character could I use the program to move the animation without re drawing it?
Truly incredible. Shear brilliance!
YOU ARE INCREDIBLE !!
use it on on ocean sonar..?
Who is here after Terence Howard explanations regards vibrations? WOW God is great!
EAGLE EYE, 2008. I remember that when the computer analyze sounds through vibrating a glass of coffee.
Robert Stovianto First thing that came to my mind! I wonder if he got the idea from it
ideas can come from anywhere, including that movie :)
phenomenal work....
Can we use the software on the Zapruder film?
I see two applications here, forensics and listening to space.
The guy who initially developed the most refined form of this technology wanted to be an Astronomer or something like that.
oh man you guys are so lucky, we were pretty psyched when we watched this video :D
CIA/NSA (and Google and Facebook for that matter) are probably busting down this guys door trying to adapt it for their purposes.
I don't think they would do such a thing you say? Our privacy still matters you say?
If you believe that way, I find that uproariously funny. So naïve..'
When I read the title I had assumed it was going to be a camera that uses sound to find objects that are behind other objects.
Oh, Mr Rubinstein was involved as well. I've cited him quite a bit in the past.
could they use that tech on silent movies to recover sound?
not really, unless it was very high resolution and high fps
As it turns out old film has more resolution than full dh but that part with high speed is still required thou.
Basically it is impossible because the frame rate of the movie is too low to recover sound.
So can you get sound out of animation with no sound. Like sliding a square across a screen.
The work is great, but let us not be over enthusiastic. I'm no expert in this field, but I can sense immediately some hidden issues. 1) All the wonders here largely depend on the super-precision of the special cameras. For a regular camera at a reasonable distance, I suspect the noises will overwhelm, and it will become a problem like chaos - undeterministic. 2) Highlighting pulse sensation in this visual way is really a fake exaggeration which does not correspond to reality; it's no more than the authors' personal interpretation. 3) I suspect the sound detection is done through checking the resonance frequencies on the leaves via the pixels or shapes, and is fundamentally prone to massive noises; radar type of detections will do infinitely better. 4) Visual extrapolation of a bush movement does not imply its feasibility. The visual detection of those tiny movements serves as some sort of initial motion vectors which can be utilised as the salt or seed for visual extrapolation. So it's a good work, but it is not really shocking to me.
Is this a re-upload?
But then also what if you took it backwards, having the mathematics of how things interact with vibrations and reverse-engineer an image/structure/model. At large-scale, you could get an image pf things that we can't see for whatever reason. Assuming technogy ever got so far as to really record the sounds of space/our satelites went further you could literally construct visual structures of stars and stuff. But if we even use sound to get images, maybe somehow we can use the same sounds to give us readings on structure. What if we could look for habitible planets by knowing how sound reacts with water, and then grabbing the data on what it sounds like on Earth, and use that understanding to be able to distinguish water on other planets. Idk. This is great stuff!
This technology maybe the beginning of being able to see realities we haven't even imagined. For example we know already our senses don't measure up to a lot of other inhabitants of this world. Animals with better sight, smell, hearing, and ability to make sounds. Elephants can make a sound that can be recorded from two miles away.
I would be curious to see 1 or 2 minute exposure of micro-organisms growing in culture. There has already been some work done on yeast moulds that suggests some sort of cognition or coherent movement.
OMG, they're latest work with this tech is just blowing my mind. Ohh the possibilities.
Anyone thought of the observers in Fringe using the same technology in the future?
Well done.
I still don't get how they could extract audio data from a two dimentional video recording... He didn't mention the impact of the audiowave on the whole target spoken to (NO 3D depth info)... tbh I am having doubts about all this...
figures you would have doubt and you had the ability to download it for free to validate, guess you are a doubting Jihed......
Amazing talk! this is the reason i subscribed to ted!
This kinda reminds me of one of the first ever "telephones" which used light to code information and then it was shined on a piece of paper which vibrated
I wonder if this technology could be used to predict how certain buildings or land masses would react to earthquakes, before they actually happen?
Nunya Bidness
That may be a very real thing. Though I imagine you would need to have a similar event filmed. Or maybe I'm just under-estimating the technique and all that'll be needed is to set a camera outside pointed at a skyscraper long enough to see.
Perhaps in time we'll inevitably get there. Not sure if we'll at all need data of stress points of certain materials to fully replicate what'll happen beyond a comically wiggling building.
Will we be able to listen to sounds on distant planets with this technology ?
Beyond the boundaries!!!!
Using this, can someone.. say, record my house.. and then use this software to see the minuscule vibrationson the walls, and find out when I take a fap break?!!!!!!!
Devil's Advocado You might want to calm down a bit if you're vibrating the walls.
Johnny Torpedo hahaa
Devil's Advocado Probably, but there are already easier, more accurate ways to do that. He even talks about this in the video.
mollistuff well shiet..
***** Yes, it really can set one back on the... progress.. ;3
gautham mysore.............from mysore? super... make our place proud
My prefered discovery by far ! Thanks !
Fascinating/terrifying.
Wow this is why everyone wants to get into MIT; their ideas are just so out of the box and innovative :D :(
So you could take an old video and interact with the surroundings after processing with software. Awesome. Not worried about the spying implications.. We're already well past level 9000 there with the PA
Hats off to the smart work these guys have done!
I think one day we will be able to talk to nature too. Wow! That'll be awesome! :)
Wow, this is classic MIT stuff! The kind of thing one imagines when MIT comes to mind :) Though it does raise lot of concerns about privacy, for example getting security credentials from a guy using a keyboard will be so easy ;)
Amazing...
Given enough computing power it can also capture sounds of past, I think. Probably the Bhagwat Gita Pravachan by Krishna, Jesus Christ talk of past is not beyond present...
That's the way I guess Yogis were all knowing as probably through enlightenment they would have developed keen and powerful perceptions. No I am not joking.
__
Shashi
Om Namah Shivaya
__
Shashi
Om Namah Shivaya
Shashi S bhaiya, i thought being a civil engineer, the last part would have hit you the most ;)
The last part is a very practical use and probably the second popular use of this technology, the first being the remote medical use, but the one I am talking will be the most far reaching use of the technology in terms of snooping and corrobative evidence of things happened in past. The most advanced use of this technique, I guess.
_
Shashi
Om Namah Shivaya
Bro, Looks like there were others before working on recreating audio from images...
"Advancements in audio recovery technology have made it possible to hear them again. In May 2011, scientists, for the first time, used a scanning system called IRENE-3D to play back an Edison doll’s voice singing “Twinkle, Twinkle, Little Star” without ever touching its tin record cylinder. IRENE-3D works by optically scanning a phonograph’s surface and digitally replicating the sound it would thus produce." @ blogs.discovermagazine.com/d-brief/2015/05/05/creepy-edison-dolls-speak-again-with-new-technology/#.VUl2DEJIBUQ
Cheers!!!
__
Shashi
ॐ नमः शिवाय
Om Namah Shivaya
amazing to learn - I would love to become involved with this and if there is any possibility to help I will
I can imagine some pretty cool AR games and apps using the second technology shown later in the video
can this technology be used to study space?
Can it measure the firmness of the ground and detect underwater streams? Or sinkholes, (I fear sinkholes)-? Or gold, or oil deposits?
This will help solve crime in the future, imagine being an investigator and using this technology one day to take video and try and extract sound in a place were a murder occurred.
but you would have to be recording while the murder is happening...and if ur going to do that you might as well just use a microphone...this goes far beyond that as far as its uses...like the dierctor being able to alter a shot after its been filmed...really amazing stuff!
This has an amazing potential, good science :-)
There is something uncanny about "Merry had a little lamb" recorded through a plant.
Amazing tech.
Endless potentials
Hello, I am a postgraduate student who needs your data set in my research. Can you tell me the location of your data set, and I would be very grateful to you
Best dubstep ever
What kind of laptop is he using?
Surveillance tech? ... yep, it is. I like the animation tech idea.
I`m interested, can be pulled out sound from silent films, as Charlie Chaplin, Garold Lloyd and other?
Indeed good tech for the nsa too
what was that hollywood movie I watched years ago with this idea in it.... They show people in the house and what they were doing at the time.
the ending experiment, Woooooooooooooooooooooooooooooooooow
imagine how realistic cloth and trees motion we can get in videogames with that... .
and then you could use it to clean up spills or swirl the water in your toilet bowl.
This could be used for listening on people without needing a bug or microphone
Still as cool today as it was 9 years ago.
It won't just tell us about our world but others also. By videotaping a distended planet we can detect so much of its being.
the military is going to have a ball with this one
So is this computer vision, machine learning or a wild combination of both?
Random Schmid pretty sure it's computer vision alone.
Cool technology!
Mind = blown
This could be the breakthrough which leads to video games which look exactly like the real world
+adavidmon exactly what im thinking, there will be no need for model creator, or graphic design. Which cuts down the video game budget, but will also cut their job :(.
Can we hear stars?
Magic. Magic. Magic.
sounds and spatial movements are space (telescopes)
wonder how well this could work on objects outside of Earth or even our solar system.
We have everything. -- Q
I don't understand, they are acting as if this is new...?
Military has has devices for years that can monitor sound (voices) from the vibration of windows of the room the people are talking in. (Hence picking up the audio from vibrations.) using video would only enhance that more I guess. But my point is, this "technology" has existed for a while now
Imagine how detectives could use this..