Abe Davis: New video technology that reveals an object's hidden properties
HTML-код
- Опубликовано: 30 сен 2024
- Subtle motion happens around us all the time, including tiny vibrations caused by sound. New technology shows that we can pick up on these vibrations and actually re-create sound and conversations just from a video of a seemingly still object. But now Abe Davis takes it one step further: Watch him demo software that lets anyone interact with these hidden properties, just from a simple video.
TEDTalks is a daily video podcast of the best talks and performances from the TED Conference, where the world's leading thinkers and doers give the talk of their lives in 18 minutes (or less). Look for talks on Technology, Entertainment and Design -- plus science, business, global issues, the arts and much more.
Find closed captions and translated subtitles in many languages at www.ted.com/tal...
Follow TED news on Twitter: / tednews
Like TED on Facebook: / ted
Subscribe to our channel: / tedtalksdirector
I wanna hear sound from a silent film back in the day.
depletable Wouldn't be possible. Somebody said something about there not being enough frames per second to get accurate measurements.
dowddash and the film itself would shift around when it reeled thru the camera causing a false positive for the software to analyze. but it would be worth trying.
pHaTdProductions If it did work that scene from Wayne's World popped into my head, where they're ordering fast food and only partially saying words. =p
Damn. You don't think's a more developed algorithm that would be able to continue improving? I mean dude could use a cell phone camera.
depletable Did you pay attention to the video? If the quality of the video wasn't good enough (number of frames and pixels), then it's not possible
We've seen this video technology used to retrieve sound in another TED talk. But a new (and impressive) application is revealed at 12:16
Maxime Isabelle Was about to say a similar thing.
I think this is the third TED talk from this group. The first three minutes came from their first talk. The first and second talk included a section on color change amplification. The rest up to the time stamp given was the second talk. Glad I stuck around for the last part which was new and what the title was talking about.
Maxime Isabelle Thanks
Yeah, guys just rehashing stuff from other TedTalks.
Somebody had mentioned that this was done in the 2008 movie with Shia Leh Boof, titled, "Eagle Eye".
I would LOVE to see this applied to some of the Mars Rover video footage!
Truly mind blowing!
When I read the title I had assumed it was going to be a camera that uses sound to find objects that are behind other objects.
Mind = blown
Cool technology!
This will help solve crime in the future, imagine being an investigator and using this technology one day to take video and try and extract sound in a place were a murder occurred.
but you would have to be recording while the murder is happening...and if ur going to do that you might as well just use a microphone...this goes far beyond that as far as its uses...like the dierctor being able to alter a shot after its been filmed...really amazing stuff!
May be it would be able to capture the early waves created by an earthquake (like many of the birds and few animals comes to know and migrate away from it) and give enough room to to prepared for it.
reminds me of Aria from the Eagle Eye
Reuploaded
Who's here from Veritasium? Super cool stuff
Wait... I've seen this talk before...
MrC0MPUT3R Some of it, yes. The parts where he shows it to be able to simulate object structure, is the new feature, the other stuff was already mostly shown in a previous presentation.
Devil's Advocado Ah so I'm not crazy. Too bad... I was looking forward to becoming the next Nostradamus.
MrC0MPUT3R What are the chances? I'm a subscriber and I've seen this before too. But, just a few minutes ago, I was thinking of finding it again to forward to someone (to demonstrate that not all TED talks are just one person droning on and on). Well, now I don't have to look for it. :)
Nope! Hold on. This is a different video. I saw two on this subject before. The first was just about magnifying subtle changes in the video (the first part of this video). The second was that plus the ability to use those magnifications to record sound (the second part of this video). Now, this one has added the interactivity work. Their work is obviously advancing rapidly.
Incase this hasn't been thought of already.
Point a ridiculously HD, telescopic, long exposure camera (Hubble) out into the universe, use our knowledge of redshift etc to determine distance, then use these algorithms to better estimate Density and movement of planets without the need for worrying about gravity and it's inconsistencies. We could detect the size, shape and velocity of a potentially life threatening asteroid many years in advance just from a few thousandths of a few million pixels. We could interpret the sound of the universe expanding and listen to it as music.
Also, on the micro level, we could use microscopes to listen to bacteria, virus's and cells and learn from a completely different sense, just how this world operates at that level.
There's so many things to learn from this amazing invention. Maybe with powerful enough cameras that pic up more than just the small bandwidth of the electromagnetic field that we see, maybe x-rays, inferred, UV and powerful enough interpretive algorithms, we can learn the resonating frequencies of certain things, like diseases, bone defects and brain functions...
Maybe we'll be able to record our brains and listen back to our very own thoughts in our minds voice.
The possibilities are endless.
We already to this but with eletromagnetic fields. I see what you have tought but light is a problem here. It would take a lot of time for this light to come to us and even to record with it
You dont hear the bacteria or virus, your hear the surrounding of it and the source video would be the material where the organism its.
Or just at the human scale; point it at machinery in factories or vehicles to predict points of failure due to tiny vibrations.
@@WilcoVerhoef Well noted! More on that application, admittedly not with the same technology, here: ruclips.net/video/rEoc0YoALt0/видео.html
Using this, can someone.. say, record my house.. and then use this software to see the minuscule vibrationson the walls, and find out when I take a fap break?!!!!!!!
Devil's Advocado You might want to calm down a bit if you're vibrating the walls.
Johnny Torpedo hahaa
Devil's Advocado Probably, but there are already easier, more accurate ways to do that. He even talks about this in the video.
mollistuff well shiet..
***** Yes, it really can set one back on the... progress.. ;3
My favourite part of any good TED talk is when something so mind blowing is presented that the audience doesn't know how to respond. He mentions around 13:20 that the following tech is a few months old and then does magic. The reactions from earlier compared to this segment...
CIA/NSA (and Google and Facebook for that matter) are probably busting down this guys door trying to adapt it for their purposes.
I don't think they would do such a thing you say? Our privacy still matters you say?
If you believe that way, I find that uproariously funny. So naïve..'
Wow, this is classic MIT stuff! The kind of thing one imagines when MIT comes to mind :) Though it does raise lot of concerns about privacy, for example getting security credentials from a guy using a keyboard will be so easy ;)
Amazing...
Given enough computing power it can also capture sounds of past, I think. Probably the Bhagwat Gita Pravachan by Krishna, Jesus Christ talk of past is not beyond present...
That's the way I guess Yogis were all knowing as probably through enlightenment they would have developed keen and powerful perceptions. No I am not joking.
__
Shashi
Om Namah Shivaya
__
Shashi
Om Namah Shivaya
Shashi S bhaiya, i thought being a civil engineer, the last part would have hit you the most ;)
The last part is a very practical use and probably the second popular use of this technology, the first being the remote medical use, but the one I am talking will be the most far reaching use of the technology in terms of snooping and corrobative evidence of things happened in past. The most advanced use of this technique, I guess.
_
Shashi
Om Namah Shivaya
Bro, Looks like there were others before working on recreating audio from images...
"Advancements in audio recovery technology have made it possible to hear them again. In May 2011, scientists, for the first time, used a scanning system called IRENE-3D to play back an Edison doll’s voice singing “Twinkle, Twinkle, Little Star” without ever touching its tin record cylinder. IRENE-3D works by optically scanning a phonograph’s surface and digitally replicating the sound it would thus produce." @ blogs.discovermagazine.com/d-brief/2015/05/05/creepy-edison-dolls-speak-again-with-new-technology/#.VUl2DEJIBUQ
Cheers!!!
__
Shashi
ॐ नमः शिवाय
Om Namah Shivaya
Best TED talk on tech I've seen in awhile. Great scientific work with enormous potential!! Well done.
They've progressed a good bit since I last heard of it, good job.
EDIT: Wow...they really progressed.
SangoProductions21 *The singularity is **-near-** here.*
EAGLE EYE, 2008. I remember that when the computer analyze sounds through vibrating a glass of coffee.
Robert Stovianto First thing that came to my mind! I wonder if he got the idea from it
ideas can come from anywhere, including that movie :)
Only 129,158 views for one year for this amazing official TED video ?
Something's wrong with the youtube users.
I wonder what kind of sound it could extract from old old silent videos .... probably not enough frames per second ... but I wonder
so if a tree falls in a forest and no one is around to hear it and you film a near by bush with a high speed camera does it make a sound?
Absolutely wonderful!
Question: what about filming a plant without any additional stimulations, to observe any sounds the plant makes thru its own vibrations? Maybe its singing it's own song.
Where to download the Video Microscope Amplifier?
Could silent films if the quality was good enough be analysed in such a way so we could hear all the ambient noise? That would be freaky weird.
Who is here after Terence Howard explanations regards vibrations? WOW God is great!
I see two applications here, forensics and listening to space.
The guy who initially developed the most refined form of this technology wanted to be an Astronomer or something like that.
In this engaging talk, Abe Davis reveals the new technology that shows the hidden properties behind every object. It's a path-breaking video technology that could potentially alter the audio-visual medium in a big way. He also shows the amazing behind-the-screen experiments that is awe-inspiring. Highly recommended to those who are in the media, TV and Video production.
I am literally speechless from the ingenuity of this research. This is absolutely incredible! This is what we call out-of-the-box think. It was right there infront of all of us, always but it took a different perspective and a curious mind to morph it into an entirely new thing we didn't even know was possible. This is research at its finest. What a great presentation too!
The interacting technology with just recording is just awesome work. Guess this can be modified and helpful for "space" research as well.
Fascinating/terrifying.
One of the best Ted talks I've watched
I wonder if this technology could be used to predict how certain buildings or land masses would react to earthquakes, before they actually happen?
Nunya Bidness
That may be a very real thing. Though I imagine you would need to have a similar event filmed. Or maybe I'm just under-estimating the technique and all that'll be needed is to set a camera outside pointed at a skyscraper long enough to see.
Perhaps in time we'll inevitably get there. Not sure if we'll at all need data of stress points of certain materials to fully replicate what'll happen beyond a comically wiggling building.
Is there a way I can download the program used to interact with objects? Has anyone released the code?
Wow this sounds so 'geekish', I am almost afraid to put this comment on RUclips.
I could see this turning into a substance analyzer to allow remote detection of the materials an object is made of. Using this technique of film analysis along with a bank of libraries of elemental motion studies, you may be able to create one of the first 'tricorders'.
Dewclaws Pretty sure if this guy has figured it out, NASA will already be analysing specs of dust on mars using this tech to figure out all of its properties. Or maybe mapping the entire earth's movements using earthquake shockwaves.
Dewclaws Well, spectroscopy can already identify every single element of an object just by analysing the wavelengths of the electromagnetic signature of its reflected light.
Of course this technology can probably work along side it to provide even more information.
I'll have to go look into spectroscopy to verify this but I thought the only elements available to analysis using spectroscopy are the elements/materials being 'abused' or 'excited' by other energetic particles, like photon's emitted or bounced. So if an objects core is cool and unreceptive, spectroscopy would not be able to view the inner core. Maybe this is spectroscopy done on lower energies. One other question I would like to ask is would this technique be restricted to use in an atmospheres with a known constant(s)?
Would you please be proud of the fact you have synapse that fire in a wonderful unique way and forget about the people who have challenges in making their synapse fire. I know this for you is like me telling you to not think pink elephant, but at least I have set in motion the possibility you might stop giving credence or credibility to those who live in a limited cerebral space and are captives of the culture. Wow I kind of liked my own sentence. Be a friend to yourself.
Will this be used for good, or evil? Or both? Or everything? God, the future man.. The future..
Now it may be possible to hear the conversation in the elevator with Beyoncé, jay z and solange yay!
so sad
For those who may not know, A.R.I.I.A (or just "ARIIA") is an Artificial Intelligent (A.I.) computer/robot/mecha that appears in the movie "Eagle Eye" (2008), and in that movie, she was able to recover sound from video by using the vibration sources from a coffee cup - essentially doing what is shown in this video!
NOTE: Many note that this is highly exaggerated, because the camera she uses would not be fast enough to sense the vibration frequencies! 🤙
would this work to study microscopic organisms? or materials on a microscopic level?
Done . Move on TED understand AND as I have said before THIS stage would not be possible without our like .A.G.B. A.L. L D.V. T.A.E. N. T. Stop research and all stupidity abounds.PEACE... Do not respond if you have a comment that is rude my AUTISM will not tolerate it at this point!!!
This is pure genius, this is going to implemented into Virtual Reality tech for sure.
I don't understand, they are acting as if this is new...?
Military has has devices for years that can monitor sound (voices) from the vibration of windows of the room the people are talking in. (Hence picking up the audio from vibrations.) using video would only enhance that more I guess. But my point is, this "technology" has existed for a while now
I don't like how they first tell the audience what is being said and then they playback what they recorded. This is like playing a file backwards and hearing messages. It's your brain creating patterns and trying to make intelligible speech out of it. It needs to be clearer and much less ambiguity. Otherwise it's like looking up at clouds in the sky to find pictures in the clouds. That's just how your brain works to find meaningful patterns in stimuli.
I'm impressed a third time. This work is like a gift that just keeps on giving.
The song being played while using shazam sounds like ice ice baby by vanilla ice. If anyone else heard it, like this comment so I can see how many heard it, or if I'm crazy just comment
Its funny that for the curtain BC: displacements=0:)) (at least it looks like...)
gautham mysore.............from mysore? super... make our place proud
This is going to allow some very interesting material analysis.
Look up "Super Sexy Science" Grad Student Rap on RUclips. This dude made that song
You would be able to spy on what people were saying from space if you had a zoom lens powerful enough. Great you just increased the power of a possible prison planet greatly. It is cool though. I wonder if this could be used to prove some old conspiracy theories from old video footage.
The fucks I don't give are also invisible. Amazing!
Hello, I am a postgraduate student who needs your data set in my research. Can you tell me the location of your data set, and I would be very grateful to you
This is a terrible idea. I hope it's as just as difficult to implement as it is a terrible idea.I have a feeling it's going to be as simple as scratching an itch though.
The Russians did something quite similar in the 40s. en.wikipedia.org/wiki/Laser_microphone.
the military is going to have a ball with this one
what was that hollywood movie I watched years ago with this idea in it.... They show people in the house and what they were doing at the time.
get a real job that can help the real people.............mate.take away tec and where would u be...?
This could be the breakthrough which leads to video games which look exactly like the real world
+adavidmon exactly what im thinking, there will be no need for model creator, or graphic design. Which cuts down the video game budget, but will also cut their job :(.
mind = blown
4:23 that MAN who laughed 😂😂😂😂😂💀
4:43 part 2 💀
Hats off to the smart work these guys have done!
I think one day we will be able to talk to nature too. Wow! That'll be awesome! :)
use it on on ocean sonar..?
Discovery kim Jess Marquez Mateos Tomás Martínez No way! D:
Anyone thought of the observers in Fringe using the same technology in the future?
So you could take an old video and interact with the surroundings after processing with software. Awesome. Not worried about the spying implications.. We're already well past level 9000 there with the PA
Can we use the software on the Zapruder film?
The work is great, but let us not be over enthusiastic. I'm no expert in this field, but I can sense immediately some hidden issues. 1) All the wonders here largely depend on the super-precision of the special cameras. For a regular camera at a reasonable distance, I suspect the noises will overwhelm, and it will become a problem like chaos - undeterministic. 2) Highlighting pulse sensation in this visual way is really a fake exaggeration which does not correspond to reality; it's no more than the authors' personal interpretation. 3) I suspect the sound detection is done through checking the resonance frequencies on the leaves via the pixels or shapes, and is fundamentally prone to massive noises; radar type of detections will do infinitely better. 4) Visual extrapolation of a bush movement does not imply its feasibility. The visual detection of those tiny movements serves as some sort of initial motion vectors which can be utilised as the salt or seed for visual extrapolation. So it's a good work, but it is not really shocking to me.
This is an absolutely amazing discovery and is very inspiring! I never even heard about this until today. This was an incredible TED talk!
People have been so conditioned by tel-LIE-vision public programming they believe everything somebody with a degree tells them, never questioning the funding motives behind these seminars.
I`m interested, can be pulled out sound from silent films, as Charlie Chaplin, Garold Lloyd and other?
Can it measure the firmness of the ground and detect underwater streams? Or sinkholes, (I fear sinkholes)-? Or gold, or oil deposits?
I would be curious to see 1 or 2 minute exposure of micro-organisms growing in culture. There has already been some work done on yeast moulds that suggests some sort of cognition or coherent movement.
What a waste of time. The first 15 min was the same material from a previous TED talk, way too much background. And the last 5 was showing how we can take video to do the same thing that animators have been doing on computers for a decade. Why is this TED talk worthy again?
Because animators can only do animation with their medium. Does anyone really need to have it explained the upside potential of this tool ? Where do we start ? Oh forget it, if you don't see it then my listing a few of the possibilities would probably be a waste of time.
There is something uncanny about "Merry had a little lamb" recorded through a plant.
Imagine this technology + augmented reality in space. Star wars suddenly becomes a tiny bit more real.
The wire figure and bush simulations were incredible. I, too, think this technology has a lot of potential.
Point it at the sun for a month... or at le earth from L1 er something. Predict weather n stuff..
Please forgive thine funny type n' grammar.
It won't just tell us about our world but others also. By videotaping a distended planet we can detect so much of its being.
OMG, they're latest work with this tech is just blowing my mind. Ohh the possibilities.
Oh, Mr Rubinstein was involved as well. I've cited him quite a bit in the past.
Very cool tech, right up until the police state gets a hold of it. Of course this can be used to keep the gov't in line as well.
Wow this is why everyone wants to get into MIT; their ideas are just so out of the box and innovative :D :(
Почему они хлопают, когда слышат искажённый звук? =)
I'd like to see them interstate this technology into kerliean photography so they can discover the sound and structure of energy
wonder how well this could work on objects outside of Earth or even our solar system.
I can imagine some pretty cool AR games and apps using the second technology shown later in the video
This kinda reminds me of one of the first ever "telephones" which used light to code information and then it was shined on a piece of paper which vibrated
amazing to learn - I would love to become involved with this and if there is any possibility to help I will
Run the picture physics simulation on a human holding his arms out, that'd be interesting
Reminded me the scene from movie Eagle Eye from the soundproof room (time 1:27:53).
That had been a great thing to send out to space when we look at planers and asteroids or maybe to look down on earth to see if we can hear or see coming earthquake oe eruption of a volcano.
So can you get sound out of animation with no sound. Like sliding a square across a screen.
Would be great for Crime Scenes..Listening to the Crimes actually taking place!!!... Aboriginal peoples all know that Rock absorbs and records ALL SOUND!!!
Surveillance tech? ... yep, it is. I like the animation tech idea.
oh man you guys are so lucky, we were pretty psyched when we watched this video :D
I just wanted to ask my favorite plastic bottle a question.
hmm if I recorded an animated character could I use the program to move the animation without re drawing it?
This could be used for listening on people without needing a bug or microphone