man... stumbling on videos/channels like this is why i f**king love youtube so much, even though i barely understand enough of this to get that this is a really cool project
this was an insane homepage pull. I understood 0 of this, but Ben & Anand (hope i got that right o7) y'all explained that very well. mind, boggeled. very cool.
@@SkimoBenyour results appear to be much more detailed than the ones showing movement in a room using a WiFi based system. I'm not familiar with the equations you are using there, so would your resolution and accuracy get better with a higher sample rate or would that just be more noise to sort through?
@@javabeanz8549 Great question ! It's not exactly 'noise' but it's not always critical information. Imagine you have some coordinates: (1,1), (3,3), (5,5). You can remove the middle coordinates (3,3) and it will still form a straight line. Our goal was to make these models run in real time, which is why we downsampled from 1MHz to 100 Hz, you can remove a lot of information as long as your slope maintains a distinguishable shape. That's not to say more samples wouldn't be more accurate, but there's diminishing returns since you have to scale the models parameters with the size of the input tensor.
compared to mmwave, which is generally used in these contexts, this is definitely low frequency. I'm sure in certain contexts 10KHz would be considered high frequency. context is important but yeah, it's still funny to call this low frequency 😅
Compared to 20 to 50GHz it is. 3.4GHz is also right in the middle of what most WiFi devices use. 2.4 and 5 to 6 are common in home and business use, but there's also 3.6GHz wireless networking, less common, but still in use.
I don't think it directly refers to the well defined radio frequency spectrum where even HF ends at 30MHz. It would have been worth noting that this is just relative to the use-case.
You gotta try this with a box around the hand. Correct me if I am wrong, but you are traning the second model off of the radio data captured while looking at the hand gestures. But the information the radio sees is omni directional. So right now there is high correlation of data points from reflection of the surfaces around the wood and not only through. More like seeing around the wood than through. Have you all tested this with the hand movement in a box to block all previous surfaces?
Wall hacks?! In real life? I can't believe I'm witnessing this. This really feels like the future now. Radar at a resolution never seen before. This is just like "Eraser".
Impressive results, especially for what seems to be a teeny tiny dataset!! It makes me wonder what you could do if one of you spent a few hours in front of the sensor to gather a lot more data. You could manually classify the gestures as you went with your other hand. That would give you an impression of what might be possible with a much larger dataset!
If you can get this to continuously learn, that could work as "Spidey sense". Use a multimodal llm to continuously train a sort of hybrid sensor. I remember experiments they did with haptic belts that vibrated true north. The people wearing them got much better with situational awareness, direction, and pathfinding. Instead of pointing North, I want a belt that vibrates in the direction of most concern. This would be super useful on my e-bike, can't be looking behind you all the time. Maybe someday in the future, I can tell if a drunk driver is coming up behind me even in the fog. Also you could get that sweet DARPA money. If you can detect gestures through a wall, you can detect someone holding a weapon.
This would actually be a really interesting HCI study for the tech. I've had the same fear while road biking! Might look into this a bit more actually...
Oh yeah, 5g can already be used to see thru walls and much better than the LoRF-Ha. It is so good, you can measure peoples' heartbeats; it was intended for medical applications, but was very quickly gobbled up by the MIC. There are videos here on YT about how it works.
@@VidarrKerr Yes and no. It's the ability to be contacts aware that's a game changer. I don't want it going off every time a car is behind me, just when a drunk/distracted/whatever driver is behind me. Or if there is an electrical shorts somewhere and you can prevent a fire. Or even just a cheap way to give robots additional perception. Paired with some bone conducting headphones, you can learn the make and model of a car coming up behind you. Or if someone is sneaking up behind you in a trench in Ukraine.
@@jtjames79 I think applying AI to decern road hazards for pedestrians or bikers is finding a solution to a problem that doesn't exisit. Getting alerts via haptic feedback, beeps, lights, and magically finding some AI/radar solution to give you useless data IN THE MOMENT while you're on the road ultimately fails when the human lacks situtional awareness and intiuation. Making a habit of doing a shoulder check and riding your bike without all these gadgets is akin to being aware of your surroundings as a pedistrian by not being distracted by your smartphone.
802.11ac, can use up to 80MHz bandwidth on a single channel. If you set up receivers within range of the WiFi you could get the resolution down to 1.875m. Beam forming from 802.11ax and combine multiple channels to improve the resolution even further. I’m guessing the permeability of various building materials would be something that training data could be adjusted to compensate for in most urban or rural structures. If only there was an openly available dataset like that.
So could we get a "X-Ray" through-walls vision with that system before the end of this decade??? WOW Adam Jensen is coming to reality for sure, _but did he ask for this_ ? :D
@@charliemckay6681 We're currently finalizing a paper that we hope will be accepted for publication in April, so if it works out the code will be on my GitHub for that publication.
It doesn't work like you think. You can't see into a building if it uses steel reinforcement. You can see through plywood or maybe brick, but nothing fancy.
Just spit balling here but first detrrmining distance of object of interest (something hugely importa t in ???) would allow normalizing of that amplitude and therefore have less (low information variance?). In a whole system view this is part of time, spatial and ??? Domain, so the subproblems become more approachable. ❤ Hope this was somewhat understandable working on verbalizjng thoughts 💭 still
Oh by the way, let's say you had multiple phased arrays detecting same object , couldn't you emulate focal length within a single phased array once you established object locstion to a sufficient degree?😊 7:33
Well, tinfoil hats and walls don't do the trick no more. Does anyone know how I can get my hands on a decommissioned stealth jet? I just want to watch videos without getting paranoid of people tracking my hand movements.
man... stumbling on videos/channels like this is why i f**king love youtube so much, even though i barely understand enough of this to get that this is a really cool project
this was an insane homepage pull. I understood 0 of this, but Ben & Anand (hope i got that right o7) y'all explained that very well. mind, boggeled. very cool.
Thank you!!! I'm stoked that you enjoyed it 😊
@@SkimoBenyour results appear to be much more detailed than the ones showing movement in a room using a WiFi based system. I'm not familiar with the equations you are using there, so would your resolution and accuracy get better with a higher sample rate or would that just be more noise to sort through?
@@javabeanz8549 Great question ! It's not exactly 'noise' but it's not always critical information. Imagine you have some coordinates: (1,1), (3,3), (5,5). You can remove the middle coordinates (3,3) and it will still form a straight line. Our goal was to make these models run in real time, which is why we downsampled from 1MHz to 100 Hz, you can remove a lot of information as long as your slope maintains a distinguishable shape. That's not to say more samples wouldn't be more accurate, but there's diminishing returns since you have to scale the models parameters with the size of the input tensor.
How tf could your mind be boggled if you understood "0" of it?
You understood it, stop being fake humble by pretending to be stupid
So, 3.4GHz is low frequency now? 😂
Relative to the size of a finger, yes.
compared to mmwave, which is generally used in these contexts, this is definitely low frequency.
I'm sure in certain contexts 10KHz would be considered high frequency. context is important
but yeah, it's still funny to call this low frequency 😅
Compared to 20 to 50GHz it is. 3.4GHz is also right in the middle of what most WiFi devices use. 2.4 and 5 to 6 are common in home and business use, but there's also 3.6GHz wireless networking, less common, but still in use.
Modern WiFi is ~ 2.4 and 5 , so a consumer level.
I don't think it directly refers to the well defined radio frequency spectrum where even HF ends at 30MHz. It would have been worth noting that this is just relative to the use-case.
Fascinating in presentation and content. The applications and learning value of these presented ideas are vast.
Thank You.
This is sick. Kudos to yall.
This is so freaking cool! You both are brilliant!
You gotta try this with a box around the hand. Correct me if I am wrong, but you are traning the second model off of the radio data captured while looking at the hand gestures. But the information the radio sees is omni directional. So right now there is high correlation of data points from reflection of the surfaces around the wood and not only through. More like seeing around the wood than through. Have you all tested this with the hand movement in a box to block all previous surfaces?
maybe with a lead or graphite backdrop, to absorb the signal?
awesome! I've been thinking about something like this with Wi-Fi for ages now, good to see someone actually pulled it off 👍
Love this format, more!
Rock paper scissors, through walls??? I dunno, first thought. I just wanted to comment for the algo.
Very nice work 👍
Phenomenal work, great job❤🎉
Wall hacks?! In real life? I can't believe I'm witnessing this. This really feels like the future now. Radar at a resolution never seen before. This is just like "Eraser".
Impressive results, especially for what seems to be a teeny tiny dataset!! It makes me wonder what you could do if one of you spent a few hours in front of the sensor to gather a lot more data. You could manually classify the gestures as you went with your other hand. That would give you an impression of what might be possible with a much larger dataset!
If you can get this to continuously learn, that could work as "Spidey sense". Use a multimodal llm to continuously train a sort of hybrid sensor.
I remember experiments they did with haptic belts that vibrated true north. The people wearing them got much better with situational awareness, direction, and pathfinding. Instead of pointing North, I want a belt that vibrates in the direction of most concern. This would be super useful on my e-bike, can't be looking behind you all the time.
Maybe someday in the future, I can tell if a drunk driver is coming up behind me even in the fog.
Also you could get that sweet DARPA money. If you can detect gestures through a wall, you can detect someone holding a weapon.
This would actually be a really interesting HCI study for the tech. I've had the same fear while road biking! Might look into this a bit more actually...
This already exists. It is a motion sensor that vibrates when objects are approaching from behind (or from whichever way you orient it).
Oh yeah, 5g can already be used to see thru walls and much better than the LoRF-Ha. It is so good, you can measure peoples' heartbeats; it was intended for medical applications, but was very quickly gobbled up by the MIC. There are videos here on YT about how it works.
@@VidarrKerr Yes and no. It's the ability to be contacts aware that's a game changer. I don't want it going off every time a car is behind me, just when a drunk/distracted/whatever driver is behind me. Or if there is an electrical shorts somewhere and you can prevent a fire.
Or even just a cheap way to give robots additional perception.
Paired with some bone conducting headphones, you can learn the make and model of a car coming up behind you. Or if someone is sneaking up behind you in a trench in Ukraine.
@@jtjames79 I think applying AI to decern road hazards for pedestrians or bikers is finding a solution to a problem that doesn't exisit. Getting alerts via haptic feedback, beeps, lights, and magically finding some AI/radar solution to give you useless data IN THE MOMENT while you're on the road ultimately fails when the human lacks situtional awareness and intiuation. Making a habit of doing a shoulder check and riding your bike without all these gadgets is akin to being aware of your surroundings as a pedistrian by not being distracted by your smartphone.
Great work, very cool!
Thank you for sharing!
802.11ac, can use up to 80MHz bandwidth on a single channel. If you set up receivers within range of the WiFi you could get the resolution down to 1.875m. Beam forming from 802.11ax and combine multiple channels to improve the resolution even further.
I’m guessing the permeability of various building materials would be something that training data could be adjusted to compensate for in most urban or rural structures. If only there was an openly available dataset like that.
Good work and presentation!~ C:
This is so cool love cool architecture
Have you guys tries mamba layers? Doing some DSP ml stuff and it has been workign quite well. Great work nonetheless!
How did you guys determine the neural network design for the model? The amount of layers, what's in each layer?
@@jamest1240 partially theory, partially intuition, but it always comes down to a bit of trial and error
So how many fingers am I holding up?
Wait, I know that one it's the New York state bird! ;0)
And old school radar with like 1 to 2 GHz This is like normal radar range for like ground radar.
So could we get a "X-Ray" through-walls vision with that system before the end of this decade??? WOW
Adam Jensen is coming to reality for sure, _but did he ask for this_ ? :D
Marvellous! I would like to play with this too- do you have your code anywhere publicly? Thank you
Okay this one is cool
Whats your linkedin?
But can it see tidies?
Depending on the clothing materials, but cloth should be fine.
Very cool
Pretty cool!
I mean WiFi can be used to detect living beings through wall
This how how master thesis should be a bit more
not seeing a github link. is this closed source ?
Good question. I hope not.
@@charliemckay6681 We're currently finalizing a paper that we hope will be accepted for publication in April, so if it works out the code will be on my GitHub for that publication.
God damn AI. Amazing.
RL training this concept?
Be carefull with what you do :)
Are you aware of the positive and negative uses that your technology could have?
Very interesting fellas
choose wisely
you can sell this to cops and the military
or you can sell this to Vegas casinos
why not both
It doesn't work like you think. You can't see into a building if it uses steel reinforcement. You can see through plywood or maybe brick, but nothing fancy.
@@doktork3406 *YET*
They already know about this, read about it more than a year ago
@@paigefoster8396 More like tens years ago, but yeah...
Just spit balling here but first detrrmining distance of object of interest (something hugely importa t in ???) would allow normalizing of that amplitude and therefore have less (low information variance?). In a whole system view this is part of time, spatial and ??? Domain, so the subproblems become more approachable. ❤ Hope this was somewhat understandable working on verbalizjng thoughts 💭 still
Oh by the way, let's say you had multiple phased arrays detecting same object , couldn't you emulate focal length within a single phased array once you established object locstion to a sufficient degree?😊 7:33
What is ???
wifi?
wall hack!
holy shit this is ground-breaking! Imagine the applications for this! Congrats!
Can't wait to put this on my murder robot
The applications: your boss can detect your motion even when there is a door or a wall between you
Can’t see any good applications for this lol
Well, tinfoil hats and walls don't do the trick no more. Does anyone know how I can get my hands on a decommissioned stealth jet? I just want to watch videos without getting paranoid of people tracking my hand movements.
I thought the same thing like anything sub gigahertz is low
🤓😎
Hey how bout y’all just stop?
shut up obama
Why stop?
AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI AI
ARRRGGGGHHH!!!!!!
Yas! AI FTW!
Could you not, please?