Check out Andrew's video here: ruclips.net/video/fDj2kYIF-sE/видео.html and his python code is here: github.com/ajsteele/faceHR The sponsor is KiwiCo: Get your first month free here kiwico.com/stevemould
☝ that card thing you pointed at didn't work ? Nothing came up there. Remember to Reverse your sides when pointing or you have to flip your video in editing
Sometimes I can feel it too. Some pauses are ever so slightly longer, and some are ever so slightly shorter. It happens more often when I'm nervous, anxious, or otherwise distressed in some way.
Super cool video! One note if people are curious, the change in heart rate with breathing is known as respirophasic sinus arrhythmia. It is regulated by baroreceptors, so when you inhale, the negative pressure decreases the signaling from these receptors to the brain, which is similar to a low arterial pressure signal, which activates the sympathetic response and speeds up your heart rate to compensate. When you exhale, your thoracic pressure increases, which increases the baroreceptor signaling and increases the parasympathetic response to decrease your heart rate.
That's super interesting, so if you had to get inflammation say, in the chest wall for exzample would that effect the breathing rate solely through the strength of signal or rather by the pain experienced and so changing your breathing differently? How does your resting heart rate change then when u excersise more? Is it related to your fat content also changing the signal. I'm not a docter, but I find these things super, super intresting and I'm always looking for more knowdagble people reasons:)
In case you are interested, there are other examples of pulse taking using cameras in the world of video games. The Kinect 2.0 was purported to be sensitive enough to read the color variations of the skin to estimate pulse readings, very much like what was shown here. But the Xbox fitness app that used it was discontinued before I'd tested it. Ring Fit Adventure on the Nintendo Switch uses the simple IR camera built into one of the controllers to read pulse rate at the fingertips.
@@GoTeamScotch The botched launch of the Xbox One forcing people to use Kinect probably didn't help. It tarnished the 'Kinect' brand. A shame, as it was a great bit of kit, but unlike PSVR, Microsoft didn't really try to make it stand on its own.
i'd like to see a survival-horror game that somehow used pulse information to find the most optimal way to scare its players (via things like music, etc) and give them nightmares..
A graph of "shrugging emoticon" over "time I guess" is so perfect. Thanks for labeling your axes, and using the correct units Steve. That made me very happy. It may take some time, but someday in the not-too far off future, I imagine every room will have some form of sensors built-in. It's going to take some clever data scientists, like your guest here, to turn the resulting mess of data into useful information. But that information could be used to do things like call the paramedics if someone's heart rhythm goes arrhythmic. That's pretty cool on its own, but it's also one step closer to a Roddenberry-esque post-scarcity eutopia, and I'm all for it. Well, at least perhaps until the advertisers get ahold of the sensor data.
@Pronto Nailed it! (As did the captain.) The rest of this dystopian mess that I encouraged here... Ugh. Please just don't tell my insurance company that I read any of these comments.
This was really interesting, and also explained to me why the pulse ox meter is not accurate in cases of CO poisoning. Red bloodcells with CO bound are still red, that's why people suffering from CO poisoning don't look like they are suffocating, they have perfectly healty looking skin tones.
I thought it was interesting, I had a phone app *YEARS* ago that would detect your heart rate based on video - long before the Apple Watch or Fitbit existed. Comparing to an actual real medical-grade heart rate monitor, it was extremely accurate. I have to imagine it used similar processing to this.
Samsung did a bit of a better job of that with the Note 4/Note Edge, which had an infra-red sensor on the back. It was faff though, just like those apps that use normal camera sensors, so not worth doing.
@@danfr yes but using your phone's camera plus the torch is different than what the guy is describing and this video because there's a lot of difference in accuracy in both scenarios.
@@tams805 no it does work. It's as good as your smartwatch but it won't monitor it continuously obviously unless you want to press your phone against your finger all the time.
Superb video. This heart stuff sounds fascinating. But don't think I'm not silently judging you both for not even attempting to say "photoplethysmography" once during the video.
Next question: How does a smart watch measure blood pressure? I think it also uses light, and no doubt there are a ton of scientific papers out there mentioning things like photoplethysmography but no layman-level explanation which you are brilliant at delivering 😊
Afaik the only smart watches that measure blood pressure use the same technique that a inflatable blood pressure cuff uses. I could be out of the loop though and a watch that can do blood pressure with only light would be awesome to know about!
With difficulty! They basically try to measure the exact shape of that graph of light brightness against time, and depending on whether it’s skewed one way or the other infer the blood pressure using _(waves hands)_ algorithms. They’re not very accurate yet though, and you have to recalibrate them every few weeks with a traditional blood pressure cuff.
I did a whole thesis on this exact topic, turns out you can infer blood pressure by measuring PPG and ECG together, then measuring the time delay between the peaks. That is a crude measurement of the PWV (pulse wave velocity) of your blood going from the heart to the extremity where you're measuring PPG (ECG peaks will always lead PPG peaks). Calibrating for arterial distensibility (strechiness) and temperature and age and baseline and a bunch of other things, and using a complicated model you can MAYBE get something somewhat accurate (I used a medical-grade continuous blood pressure measurement device called the Finapress as a gold standard reference). The ECG/PPG technique was rough though honestly and not something you'd wanna trust if you even so much as breathe heavily.
@@ThePrufessa wasn't my focus at the time but Google scholar shows that some work was done on this! The slope of the PPG graph seems to change before and after alcohol consumption so using that fact, 80%+ accuracy was achieved in computing BAC (blood alcohol content). Interesting stuff indeed.
This is one of those few videos you come across that you feel genuinely deserves a like button. Extremely well done, very informative, learned a lot of new things about biometrics and data analysis
Hey this is exactly what my startup does! The technique is called rPPG, which is remote PPG, or photoplethysmography, which is what the Apple Watch is based on. Super stoked you've decided to showcase this in your video!
One correction for you Steve, for pulse Ox, the sensing still requires the ambient light measurement. The ambient component needs to be removed from the red and IR because it can induce a signal into the red and IR sensors when the environment is bright enough. It's also used as a way to offset the red and IR signals when near saturation and used to detect dark room environments vs bright room environments. Some sensors automatically remove the ambient component and don't report it, but it is still needed for proper measurement.
Hearing that the heart speeds up during a breath taken in explains so much! For years whenever I'd listen to my dog's heart (either dog) I noticed that every time he/she took a breath, there would be about 3 heartbeats quickly in the same space of time they had a single heartbeat when not taking a breath. Cool info!
Six or seven years ago I was attending engineering master thesis dissertations in the Trento University and one of the candidates proposed a similar method to monitor - using a very low resolutions webcam - the heartbeats of bus drivers, as an extra safety measure for public transportation. I was very surprised because it worked!
@@goku445 maybe you could tie in with gradual breaking, throttle limits, if it detected unusual out of limits results, the difference being , it does'nt require anything to be worn, or attached to the driver, just a camera device looking at the driver
Ooooh. I've worked with some biosignal stuff before, and have been curious if this was possible too. Also fun fact, you can get the respiration directly from a PPG signal (instead of the heart rate variability) using a bandpass filter that removes the higher frequencies from the heartbeat. It works surprisingly well, though the sensor has to stay pretty still for it to work.
Fun fact: you can see your heart beats by color changes in your own visual field. Easiest if looking at a white background, takes some attention and the ability to keep your gaze steady for a while.
I have one of those athlete sport watches that can measure HR and sPO2 24/7 and I never knew exactly how they worked before. I had no idea that watching this video would be so informative. Thanks!
The amount of complexity, calculations, adaptation to variations and ingenuity on "crossed scientific branches" in a single device is ... striking, totally awesome and inspiring! I wish to know more, even the mathematics behind it! LOVELY
I had a Fitbit I wore for a few years that was recalled (and they refunded 100% even though it was a few years old) that Fitbit claimed had battery issues where it could get hot and burn the user. Well mine never got hot, and I only wore it during the daytime when I was awake, never at night sleeping. The first year or so, no problems at all, but about 6 months before the recall I developed this lump directly on my arm directly underneath the fitbit that looked like a RF radiation burn. The reason I said that, was because back in the 1980's I worked on 2-way radio transmitters and occasionally would accidently bump up against an antenna that was transmitting and would get burns under my skin. It isn't a burn by surface heat but deep heating due to absorption of RF energy by the living tissues underneath. Since the surface of the skin is relatively dry it isn't harmed but the living tissue under the skin suffers damage causing a deep sore, which takes a while to heal. I had this same strange occurance so I stopped wearing the fitbit and after 3 or 4 months, the lump started oozing pus for a month or so, then finally healed up and went away. There were some articles that claim it was cancer, but I really don't feel it was that, but if my guess is correct, I'm guessing a glitch caused the one of those high energy diodes to get stuck on (which also could explain the battery overheating issue Fitbit mentioned) and radiating the tissue underneath causing a nasty radiation burn. Regardless, I realize nothing is perfect, crap happens, and I feel the benefit of smart watches still outweighs the potential risk of new technology possibly malfunctioning as I think mine may have done (I have no proof that is what truly happened but clearly something was weird was going on) and they will save way more lives as the technology is perfected.
I was using samsung watch 3 for a few months and I used to wear it during the day and also while sleeping. I had the continuous measurement turned on so the green light was on all the time. I started noticing some kind of a spot right underneath the light, it wasn't causing any issues but it was kind of itchy and looked different. I stopped wearing the watch all the time and it went away.
or it might have been an allergy or a reaction to some compound formed in the metal contacts (charging pins)... or just an ant/spider. Sunlight is far stronger than these red/ir/green LEDs, and yes, the sun is shining also in red/ir/green light (and lower and higher). Even talking about sunscreen to avoid sunburns: it only blocks very high frequency light, ultraviolet. How do you we know? because we can see the original skin color beneath the sunscreen (you can see UV cameras youtube videos to see how effective they are at blocking UV). And well those are not high energy diodes. I guess you have in mind the high *power* diodes used for emitting/rectifying RF, but you would know the comparatively high power you were using for transmission vs the dozens of mW that a LED dissipates (and you know power dissipation is not the same as power output).
@Sdlion while anything is possible the problem with your theory is the skin surface was fine, the issue was deeper under the surface of the skin like it had been irradiated.
@@TurpInTexas like getting a splinter? I mean, that sounds as nothing out of the ordinary. Might not be the most common of irritation / infection, but certainly something that we all some times experience in our life, with or without a bio smart watch . And there's no theory in correcting that is not a high power/energy led... you can literally search for the datasheet of the led drivers those devices use and the leds they use. And you worked with RF you should know the power relation and the safety rules regarding power RF devices.
@@sdlion7287 Sun is the best medicine on the planet. They have lied and programmed people to put on sun glasses that stop light from entering the eyes. Also sunblock causes cancer not the Sun. The Sun can cause damage if you allow it to burn your skin. Otherwise it is the best medicine that our Western medical system has lied to everyone about.
I saw a very similar use of color change amplification to measure pulse about 10 years ago, the code was written by some grad students for a computer vision class.
As a person with a complicated cardiovascular condition this had given me some great insight into how some devices work and how the medical system can monitor and maintain my current physiology. I have a condition called HLHS.
I was always wonder why my heart beats slightly faster when I breathe in.. thank you for this and all the other explanations! You both are great together!
When I hold my breath, I always figured my heartrate would go down because it has access to a lot of oxygen. but i couldnt figure out why it would go up instead. but having less space makes a lot of sense!
Here's a thing: My heart rate drops significantly when I hold my breath. This was observed by a doctor many years ago, and he commented that this phenomenon is common in infants. But they "grow" out of it as they reach adolescence apparently. He told me that I have kept the infant behavior (nothing to worry about), but interesting. BTW, I am over 50.
About the color strength, have you ever thought about bayermatrix? Since regular cameras are often tuned with one extra green pixel over red and blue(RGGB). In astrophotography we usually remove green noise due to this. And this is also a reason why we instead use mono cameras to make use of ALL pixel from chip.
Great episode. I recently bought a smart watch that was having trouble reading some of my vitals. Now that I know the science behind the sensors I realize my $18 watch may not be the value I thought it was. Also, that KiwiCo stuff makes me want to be 8 years old again!
I get so excited when I see you have a new video posted. The things you talk about are just so interesting and well explained. Thanks for all the work you put into this.
Most fascinating video I have seen in some time. As someone who has worked in a refinery if loved the scene of the piping and pump. Most likely their is an application for analysis equipment failure. Great job Andrew on the Python Code.
I was told that the haemoglobin absorbs and reflects extremely specific, and importantly different, wavelengths depending on whether or not it is carrying oxygen. And they use the relative levels of those 2 frequencies to calculate the blood oxygen levels. It seems a simpler option than the one explained here. I am completely prepared to be entirely incorrect. Fantastic video and seeing as I just bought a smart watch a couple of weeks ago you have answered a bunch of questions I had. Keep up the amazing work
@@rafeesamith thank you. I think I will rewatch to see if I missed something. I had sort of assumed that the wavelengths were close enough between states that they wouldn't need a visible and IR light source to differentiate between them.
Год назад+4
I love how you meet up with other smart people to do all these interesting stuff i could never even understand
Heart rate variability is related to intrathoracic pressures rather than volumes. Inspiration produces a negative pressure which also draws blood in to the chest. The heart then has to compensate to pump away the extra blood
The reasoning on depth of wavelength is true, this is why you would use IR for deeper tissue imaging. However there is a potential in the case of blood it is actually to do with its auto-fluorescent properties, which for red blood cells are far higher in around the 450-550nm wavelength than others in the visible spectrum, it depends on what the sensor is actually detecting.
I bought a Garmin watch specifically so I could monitor my Blood Ox and Heart Rate. I have Cystic Fibrosis and as I have gotten older. It is super important for me to know what is going on. Especially since my pft score is super low (25%-42% at best). Last few months my heart rate has gotten much higher than it has ever been and my blood ox is slowly going down. Thank you for making this video.
I always assumed the reason your heart pumped harder when you breathe in is that it just got a bunch of oxygen and needs to deliver it. Turns out i was proved wrong. Thank you for the cool information, and awesome video!
They actually got that backwards - your heart pumps harder and *slower* on breathing in because you chest expands, which doesn't just suck in air but it also sucks blood in from the rest of your body into the heart, meaning each heart beat pumps more blood so it can go slower
Just a small comment, the presence or lack of oxygen doesn't make a wavelength less or more (infra)red. That implies a shift in frequency that doesn't happens. Light just gets absorbed by the oxygen, and the sensor detects the light intensity. There is also other reasons why the light pulses at a specific frequency. Sorry if I sound pedantic. It just a small correction that doesn't change the basic ideas of the video. Which is awesome, by the way.
7:09 I did have a good laugh. Well, they always tell you that YOU MUST have labels on your axis. Yours make perfect sense. I am going to borrow this. I have an electrical consumption graph (a non zero derivative) that is "meant" to be viewed over 24 hours with window sizes of 1 hour. So it's units are kwh/h ... but if you zoom in or out it starts reading in units of -\_(:-))_/-
What people really wanted was a watch that could measure blood sugar. The GlucoWatch accomplished this by running a weak electric current through your skin, calibrating against a finger stick, and then using changes in the current to estimate changes to the glucose concentration of the interstitial fluid in real time (a proxy for blood glucose). Too bad it hurt and never really worked well.
That would've been amazing to have back when I was having to finger prick every 4 hours! I've felt low electric currents before, and while a bit uncomfortable, I think I'd rather have dealt with that than finger pricks. But that's just an assumption based off of my time doing finger pricks and never having tried the device mentioned. Cool in theory however!
One of the problems I have with the wristwatch optical sensors vs the electrical chest sensors is that while I'm trail running, unless the watch is uncomfortably tight to my wrist, the amount of signal processing to extract the heartrate from the constant rhythmic jostling of the optics against my wrist is going to yield a pretty rough guess polluted with rapid movement (low frequency noise, coherent to a heartbeat's bandwidth), whereas the chest strap, is relying on a more consistent electrical capacitance undulation despite vibration. Wavelet conformity during periods of high activity is easier to pull from the electrical measurement whereas you can induce perceived arrhythmia within the data of a bouncing optical sensor.
It's best to just continue using an ECG chest strap. Really, it's always best to use one if you care about accuracy. As long as it's kept adequately lubricated. The only time it's really an issue is when swimming, especially pool swimming (well, and obviously high diving).
6:09 - Amazing! Sometimes I lay awake and listen to my own heart beats. I've noticed if I take a deep breath my heart suddenly takes an extra beat. It's reassuring to get confirmed that this is a common phenomenon.
I think I saw something similar a couple of years ago, but it was more nefarious, like using that same detection of the change of colour of the face due to blood pulse for spying.
I have absorbed so much of the information you have distributed through your videos. Such a wide array of knowledge that I will surely pass on to my son.
@@JasonRennieWTF clock chips draw power, and it's literally a tiny led. Idk if you're new to LEDs but they're pretty light on power draw all things considered
@@vappyreon1176 That is not true at all. LEDs are typically dimmed by pulse width modulation - turning them on and off very quickly. And it saves power proportional to the reduced light output.
That's crazy! Because I've seen that on people's faces before, but I didn't know what I was seeing!! This is so cool to know. I'm going to start paying more attention to this.
Great video, although there has been a lot recently about this not working so reliably for people with darker skin. It would be interested to learn more about how it can be just as reliable in such cases.
I often thought it must be motion amplification via phase difference of light wavelengths, but boy this was mind blowing. I really could use this information.
I really really appreciate the thumbnail with the purple light. I thought that’s clickbait and fake. But with the inferred it’s kinda real very nice eye catching thumbnail without being fake. Much appreciated.
I already knew how the sensors worked but your video still made it fascinating. Very well illustrated and some great footage. Too bad we didn't get the IR view motion-amplified as well!
I've done heart and breathing rate measurements differently. If you put a piezo sensor inside for example a pillow and sit on it you can extract both. Just by filtering and amplifying the signal you get a nice clean result if you don't move that much. Really easy, reliable and cheap. :) But it's not a contactless approach like the one in the video.
I wonder if this works just as well with people that have darker skin. There is already a known issue with pulseoxes that are less accurate if you have darker skin.
It's good to remember that these optics-based detection methods are often calibrated based on lighter skin tones- for example, people of color can show oxygen saturation levels that are erroneously high. More work needs to be done to make medical equipment work for everyone!
Just to be clear, none of the gadgets used here (even the Polar H10 - though that is very accurate) are medical equipment. Well, the stethoscope is, I guess.
I was literally just looking at my smart watch last night and asking myself what the light does. My thoughts are starting to sync up with Steve Mould videos!
Wouldn't the infrared go down when the red light goes up if they react oppositely to the amount of oxygen in blood (i.e. there would be an intrinsic "phase" difference in the curves at about 8:20)?
I think it's the amount of blood in the vessels that causes that periodic change, and the change due to oxygen concentration is less obvious until you do the math to extract that data from the ratio.
I like how he went to infants in a hospital as a potential avenue for this kind of tech. I could also see this kind of tech (after some levels of advancement mind you) being used at security check points like airports/customs to see who's heart rate starts to get elevated as they get closer to the check point, or maybe just have random officers walking around to see someone who just got their luggage if their heart rate gets elevated although you have to be very careful about that.
@@dylan-5287 Well what you want and what we get are bound to be two separate things. I don't want to take my shoes off at the security check-in either but thanks to some dude stuffing some explosives in his shoes now we all have to do that. Thank god he didn't try to stuff the explosives in his "prison wallet" I'd never fly again if they put us through that level of scrutiny.
Shared the video with a friend (You're welcome! 😉). He had a good comment. If you folks can do this, then people doing surveillance can do it too. I'm picturing camera's all over an airport just looking for people who's pulse rates seem a bit too high... Or any port of entry, while the Border Agent is asking "Purpose of your visit?" and the AI is staring at your face to see if you get especially nervous.
The main reason why the green LED flashes is to save battery, that significantly increases battery life of the smartwatch. That's also the reason why these LEDs will only flash when the smartwatch senses movement, otherwise it just assumes it's not being used and enters battery saving mode. They can get away with the flashing because our heart rate is super slow when compared to the speeds at which the embedded electronics are capable of running at, so they can sample a few times a second as opposed to constantly reading the heart rate sensor input.
Okey so this is actually kind of funny. I tend to have anxiety attacks from time to time and I’ve learn yo recognize them quite easily, to the point where I calm myself by just realize it’s a panic attack (and I’m not in any actual danger), except for yesterday where I had my biggest anxiety attack I’ve had in my life, where I even started to hyperventilate pretty badly. After a while I managed to calm down and breath semi normally again, and I started thinking that I should buy a pulse oximeter to be able to show my brain that it’s just a anxiety attack. The point is, I started to wonder how they worked and how they could measure the oxygen in the blood, and just as I was wondering that, scrolling though RUclips (to distract myself), this video popped up. I didn’t watching then because I was scared to even think about breathing and oxygen and all of that, but yeah, what a coincidence 😂
Either you accept that technology is good and we keep moving forward, or conclude that it's bad and move back into caves. We can't draw the line at some point and say "this is the optimal level of technology for human kind".
Check out Andrew's video here: ruclips.net/video/fDj2kYIF-sE/видео.html and his python code is here: github.com/ajsteele/faceHR
The sponsor is KiwiCo: Get your first month free here kiwico.com/stevemould
The only code problem I see is that he spelled colors wrong. There is no u in colors. At least he got cvtColor correct. ;-)
@@anthonylosego bludy ell, wot a nonce this bugga is
☝ that card thing you pointed at didn't work ? Nothing came up there. Remember to Reverse your sides when pointing or you have to flip your video in editing
Professor Mold :D
Vertical is intensity and horizontal is time.
The labels on the graph killed me, as someone whose studying to be an engineer I’m definitely gonna steal those for my charts
Future Arup employee
Theranus
@@ImpalerVlad yeah that's why he's an engineer and you're a starbucks employee
@@ImpalerVlad my guy needs to retake english
-\_(シ)_/-
Time, I guess
Thanks for having me! Honestly the most fun I’ve had strapping electrodes to someone and filming it.
The military will definitely use/steal this tech my guy
dude, why do you have less replies than me my guy?
If you've ever listened to a heart for a long period of time, it's kinda scary how irregular they are, but that's how they work
Sometimes I can feel it too. Some pauses are ever so slightly longer, and some are ever so slightly shorter. It happens more often when I'm nervous, anxious, or otherwise distressed in some way.
My heart beats 2-3 times faster when I'm breathing in
Anybody who’s gotten WAY too high can attest to this
Have you ever heard a newborn's heart? And not only their heart, their kind of respiration it's scary
@@noctisumbra4656newborn’s are dark art
Super cool video! One note if people are curious, the change in heart rate with breathing is known as respirophasic sinus arrhythmia. It is regulated by baroreceptors, so when you inhale, the negative pressure decreases the signaling from these receptors to the brain, which is similar to a low arterial pressure signal, which activates the sympathetic response and speeds up your heart rate to compensate. When you exhale, your thoracic pressure increases, which increases the baroreceptor signaling and increases the parasympathetic response to decrease your heart rate.
This is totally what I meant to say
(Anatomy and physiology are amazing! I wish I knew some more. :) )
That's super interesting, so if you had to get inflammation say, in the chest wall for exzample would that effect the breathing rate solely through the strength of signal or rather by the pain experienced and so changing your breathing differently?
How does your resting heart rate change then when u excersise more? Is it related to your fat content also changing the signal.
I'm not a docter, but I find these things super, super intresting and I'm always looking for more knowdagble people reasons:)
In case you are interested, there are other examples of pulse taking using cameras in the world of video games. The Kinect 2.0 was purported to be sensitive enough to read the color variations of the skin to estimate pulse readings, very much like what was shown here. But the Xbox fitness app that used it was discontinued before I'd tested it.
Ring Fit Adventure on the Nintendo Switch uses the simple IR camera built into one of the controllers to read pulse rate at the fingertips.
Phone cameras already over compensate on the green and red to make pictures better 🤔
The kinect 2 was such a cool piece of tech at the time that went woefully underutilized. Too bad it didn't catch on.
@@GoTeamScotch The botched launch of the Xbox One forcing people to use Kinect probably didn't help. It tarnished the 'Kinect' brand.
A shame, as it was a great bit of kit, but unlike PSVR, Microsoft didn't really try to make it stand on its own.
i'd like to see a survival-horror game that somehow used pulse information to find the most optimal way to scare its players (via things like music, etc) and give them nightmares..
If Microsoft marketed Kinect towards students, IOT developers. It’d have sold a lot more.
2:31 “And obviously if the premature baby has a beard like this, you would shave it”
I wheezed at this
Never shave a baby......
I waited for a sponsor segment after that lol
A graph of "shrugging emoticon" over "time I guess" is so perfect. Thanks for labeling your axes, and using the correct units Steve. That made me very happy.
It may take some time, but someday in the not-too far off future, I imagine every room will have some form of sensors built-in. It's going to take some clever data scientists, like your guest here, to turn the resulting mess of data into useful information. But that information could be used to do things like call the paramedics if someone's heart rhythm goes arrhythmic. That's pretty cool on its own, but it's also one step closer to a Roddenberry-esque post-scarcity eutopia, and I'm all for it. Well, at least perhaps until the advertisers get ahold of the sensor data.
I'm sure insurance companies would this data! Having this in their consulting room and they may rethink your low premiums.
It's like Apple's performance graphs
that sounds extremely dystopian
It may take some time, but someday in the not-too far off future, this is how the robots will able to detect the replicants.
@Pronto Nailed it! (As did the captain.)
The rest of this dystopian mess that I encouraged here... Ugh. Please just don't tell my insurance company that I read any of these comments.
This was really interesting, and also explained to me why the pulse ox meter is not accurate in cases of CO poisoning. Red bloodcells with CO bound are still red, that's why people suffering from CO poisoning don't look like they are suffocating, they have perfectly healty looking skin tones.
I thought it was interesting, I had a phone app *YEARS* ago that would detect your heart rate based on video - long before the Apple Watch or Fitbit existed. Comparing to an actual real medical-grade heart rate monitor, it was extremely accurate. I have to imagine it used similar processing to this.
You can still do that today in Google Fit even without buying a smartwatch.
Samsung did a bit of a better job of that with the Note 4/Note Edge, which had an infra-red sensor on the back.
It was faff though, just like those apps that use normal camera sensors, so not worth doing.
Legend has it that ancient man developed the closely guarded secret technique of feeling one's wrist
@@danfr yes but using your phone's camera plus the torch is different than what the guy is describing and this video because there's a lot of difference in accuracy in both scenarios.
@@tams805 no it does work. It's as good as your smartwatch but it won't monitor it continuously obviously unless you want to press your phone against your finger all the time.
Superb video. This heart stuff sounds fascinating. But don't think I'm not silently judging you both for not even attempting to say "photoplethysmography" once during the video.
Andrew actually said it many times. I cut it out for time/because I was jealous that I couldn't say it.
Next question: How does a smart watch measure blood pressure? I think it also uses light, and no doubt there are a ton of scientific papers out there mentioning things like photoplethysmography but no layman-level explanation which you are brilliant at delivering 😊
Afaik the only smart watches that measure blood pressure use the same technique that a inflatable blood pressure cuff uses. I could be out of the loop though and a watch that can do blood pressure with only light would be awesome to know about!
With difficulty! They basically try to measure the exact shape of that graph of light brightness against time, and depending on whether it’s skewed one way or the other infer the blood pressure using _(waves hands)_ algorithms. They’re not very accurate yet though, and you have to recalibrate them every few weeks with a traditional blood pressure cuff.
I did a whole thesis on this exact topic, turns out you can infer blood pressure by measuring PPG and ECG together, then measuring the time delay between the peaks. That is a crude measurement of the PWV (pulse wave velocity) of your blood going from the heart to the extremity where you're measuring PPG (ECG peaks will always lead PPG peaks). Calibrating for arterial distensibility (strechiness) and temperature and age and baseline and a bunch of other things, and using a complicated model you can MAYBE get something somewhat accurate (I used a medical-grade continuous blood pressure measurement device called the Finapress as a gold standard reference). The ECG/PPG technique was rough though honestly and not something you'd wanna trust if you even so much as breathe heavily.
@@Masalmeh321 can these lights detect how much alcohol is in the blood?
@@ThePrufessa wasn't my focus at the time but Google scholar shows that some work was done on this! The slope of the PPG graph seems to change before and after alcohol consumption so using that fact, 80%+ accuracy was achieved in computing BAC (blood alcohol content). Interesting stuff indeed.
This is one of those few videos you come across that you feel genuinely deserves a like button. Extremely well done, very informative, learned a lot of new things about biometrics and data analysis
Hey this is exactly what my startup does!
The technique is called rPPG, which is remote PPG, or photoplethysmography, which is what the Apple Watch is based on. Super stoked you've decided to showcase this in your video!
@steve_mould. Feck off, scammer.
What is the name of your startup, and when will your software be available? :D
One correction for you Steve, for pulse Ox, the sensing still requires the ambient light measurement. The ambient component needs to be removed from the red and IR because it can induce a signal into the red and IR sensors when the environment is bright enough. It's also used as a way to offset the red and IR signals when near saturation and used to detect dark room environments vs bright room environments. Some sensors automatically remove the ambient component and don't report it, but it is still needed for proper measurement.
Hearing that the heart speeds up during a breath taken in explains so much! For years whenever I'd listen to my dog's heart (either dog) I noticed that every time he/she took a breath, there would be about 3 heartbeats quickly in the same space of time they had a single heartbeat when not taking a breath. Cool info!
Easy to see if you hold your breath.
Six or seven years ago I was attending engineering master thesis dissertations in the Trento University and one of the candidates proposed a similar method to monitor - using a very low resolutions webcam - the heartbeats of bus drivers, as an extra safety measure for public transportation. I was very surprised because it worked!
"the heartbeats of bus drivers"
How would you use such data?
@@goku445 maybe you could tie in with gradual breaking, throttle limits, if it detected unusual out of limits results, the difference being , it does'nt require anything to be worn, or attached to the driver, just a camera device looking at the driver
Ooooh. I've worked with some biosignal stuff before, and have been curious if this was possible too. Also fun fact, you can get the respiration directly from a PPG signal (instead of the heart rate variability) using a bandpass filter that removes the higher frequencies from the heartbeat. It works surprisingly well, though the sensor has to stay pretty still for it to work.
1:08 Red lights and invisible (IR) lights are used in pulse oximeters to measure blood oxygen.
Your video amplification presentation was fascinating but this is something else.
Fun fact: you can see your heart beats by color changes in your own visual field. Easiest if looking at a white background, takes some attention and the ability to keep your gaze steady for a while.
You can tell Andrew is a data guy from the way he talks about a simple graph of numbers.
IT WAS PRETTY DAMMIT
@@DrAndrewSteele lol
I have one of those athlete sport watches that can measure HR and sPO2 24/7 and I never knew exactly how they worked before. I had no idea that watching this video would be so informative. Thanks!
The amount of complexity, calculations, adaptation to variations and ingenuity on "crossed scientific branches" in a single device is ... striking, totally awesome and inspiring! I wish to know more, even the mathematics behind it!
LOVELY
I had a Fitbit I wore for a few years that was recalled (and they refunded 100% even though it was a few years old) that Fitbit claimed had battery issues where it could get hot and burn the user. Well mine never got hot, and I only wore it during the daytime when I was awake, never at night sleeping. The first year or so, no problems at all, but about 6 months before the recall I developed this lump directly on my arm directly underneath the fitbit that looked like a RF radiation burn. The reason I said that, was because back in the 1980's I worked on 2-way radio transmitters and occasionally would accidently bump up against an antenna that was transmitting and would get burns under my skin. It isn't a burn by surface heat but deep heating due to absorption of RF energy by the living tissues underneath. Since the surface of the skin is relatively dry it isn't harmed but the living tissue under the skin suffers damage causing a deep sore, which takes a while to heal. I had this same strange occurance so I stopped wearing the fitbit and after 3 or 4 months, the lump started oozing pus for a month or so, then finally healed up and went away. There were some articles that claim it was cancer, but I really don't feel it was that, but if my guess is correct, I'm guessing a glitch caused the one of those high energy diodes to get stuck on (which also could explain the battery overheating issue Fitbit mentioned) and radiating the tissue underneath causing a nasty radiation burn. Regardless, I realize nothing is perfect, crap happens, and I feel the benefit of smart watches still outweighs the potential risk of new technology possibly malfunctioning as I think mine may have done (I have no proof that is what truly happened but clearly something was weird was going on) and they will save way more lives as the technology is perfected.
I was using samsung watch 3 for a few months and I used to wear it during the day and also while sleeping. I had the continuous measurement turned on so the green light was on all the time.
I started noticing some kind of a spot right underneath the light, it wasn't causing any issues but it was kind of itchy and looked different. I stopped wearing the watch all the time and it went away.
or it might have been an allergy or a reaction to some compound formed in the metal contacts (charging pins)... or just an ant/spider.
Sunlight is far stronger than these red/ir/green LEDs, and yes, the sun is shining also in red/ir/green light (and lower and higher). Even talking about sunscreen to avoid sunburns: it only blocks very high frequency light, ultraviolet. How do you we know? because we can see the original skin color beneath the sunscreen (you can see UV cameras youtube videos to see how effective they are at blocking UV).
And well those are not high energy diodes. I guess you have in mind the high *power* diodes used for emitting/rectifying RF, but you would know the comparatively high power you were using for transmission vs the dozens of mW that a LED dissipates (and you know power dissipation is not the same as power output).
@Sdlion while anything is possible the problem with your theory is the skin surface was fine, the issue was deeper under the surface of the skin like it had been irradiated.
@@TurpInTexas like getting a splinter?
I mean, that sounds as nothing out of the ordinary. Might not be the most common of irritation / infection, but certainly something that we all some times experience in our life, with or without a bio smart watch .
And there's no theory in correcting that is not a high power/energy led... you can literally search for the datasheet of the led drivers those devices use and the leds they use. And you worked with RF you should know the power relation and the safety rules regarding power RF devices.
@@sdlion7287 Sun is the best medicine on the planet. They have lied and programmed people to put on sun glasses that stop light from entering the eyes. Also sunblock causes cancer not the Sun. The Sun can cause damage if you allow it to burn your skin. Otherwise it is the best medicine that our Western medical system has lied to everyone about.
9:37 Thanks, this will haunt my worst nightmares now
If it illuminated the blood vessels, i would have just Thanos snap myself then and there.
Someone needs to make this a creepy pasta
The banter at the end of this was really enjoyable
I saw a very similar use of color change amplification to measure pulse about 10 years ago, the code was written by some grad students for a computer vision class.
As a person with a complicated cardiovascular condition this had given me some great insight into how some devices work and how the medical system can monitor and maintain my current physiology. I have a condition called HLHS.
I was always wonder why my heart beats slightly faster when I breathe in.. thank you for this and all the other explanations! You both are great together!
This channel definitely deserves more recognition
When I hold my breath, I always figured my heartrate would go down because it has access to a lot of oxygen. but i couldnt figure out why it would go up instead. but having less space makes a lot of sense!
Here's a thing:
My heart rate drops significantly when I hold my breath.
This was observed by a doctor many years ago, and he commented that this phenomenon is common in infants. But they "grow" out of it as they reach adolescence apparently.
He told me that I have kept the infant behavior (nothing to worry about), but interesting. BTW, I am over 50.
@@AnotherPointOfView944 Weird!
@@AnotherPointOfView944
I think that's probably from taking a deep enough breath to stimulate the Vagus nerve.
@@AnotherPointOfView944 That's so cool! You must be really good at holding your breath underwater.
I think the heartrate will go up with CO2 in your blood. When you hold your breath, the CO2 concentration will obviously go up.
About the color strength, have you ever thought about bayermatrix? Since regular cameras are often tuned with one extra green pixel over red and blue(RGGB). In astrophotography we usually remove green noise due to this. And this is also a reason why we instead use mono cameras to make use of ALL pixel from chip.
Great episode. I recently bought a smart watch that was having trouble reading some of my vitals. Now that I know the science behind the sensors I realize my $18 watch may not be the value I thought it was. Also, that KiwiCo stuff makes me want to be 8 years old again!
I get so excited when I see you have a new video posted. The things you talk about are just so interesting and well explained. Thanks for all the work you put into this.
Most fascinating video I have seen in some time. As someone who has worked in a refinery if loved the scene of the piping and pump. Most likely their is an application for analysis equipment failure. Great job Andrew on the Python Code.
Thanks for letting us know a little of what technology we have on our wrists
That's way too much data to trust a connected device.
I was told that the haemoglobin absorbs and reflects extremely specific, and importantly different, wavelengths depending on whether or not it is carrying oxygen. And they use the relative levels of those 2 frequencies to calculate the blood oxygen levels. It seems a simpler option than the one explained here.
I am completely prepared to be entirely incorrect. Fantastic video and seeing as I just bought a smart watch a couple of weeks ago you have answered a bunch of questions I had. Keep up the amazing work
in essense yes that's exactly what's described in the video, red and infrared have two distinctly different wavelengths (IR being longer than red)
@@rafeesamith thank you. I think I will rewatch to see if I missed something. I had sort of assumed that the wavelengths were close enough between states that they wouldn't need a visible and IR light source to differentiate between them.
I love how you meet up with other smart people to do all these interesting stuff i could never even understand
Human ingenuity is so fascinating. Another great video, steve!
Heart rate variability is related to intrathoracic pressures rather than volumes. Inspiration produces a negative pressure which also draws blood in to the chest. The heart then has to compensate to pump away the extra blood
The reasoning on depth of wavelength is true, this is why you would use IR for deeper tissue imaging. However there is a potential in the case of blood it is actually to do with its auto-fluorescent properties, which for red blood cells are far higher in around the 450-550nm wavelength than others in the visible spectrum, it depends on what the sensor is actually detecting.
I really loved the Axis labels here. One is definitely 'absorbance at a given frequency of light' the other is 'time I guess'. :D
Those may have been the funniest outtakes I've seen in a long time, thank you for including them!
Finally, thank you. I sort of had an idea how it works but you really clarified it
I bought a Garmin watch specifically so I could monitor my Blood Ox and Heart Rate. I have Cystic Fibrosis and as I have gotten older. It is super important for me to know what is going on. Especially since my pft score is super low (25%-42% at best). Last few months my heart rate has gotten much higher than it has ever been and my blood ox is slowly going down.
Thank you for making this video.
Let's get this working on videos of politicians lying when asked certain questions. No seriously. Let's.
2:44 love the voyager plaque on the back of the laptop
I always assumed the reason your heart pumped harder when you breathe in is that it just got a bunch of oxygen and needs to deliver it. Turns out i was proved wrong. Thank you for the cool information, and awesome video!
They actually got that backwards - your heart pumps harder and *slower* on breathing in because you chest expands, which doesn't just suck in air but it also sucks blood in from the rest of your body into the heart, meaning each heart beat pumps more blood so it can go slower
Just a small comment, the presence or lack of oxygen doesn't make a wavelength less or more (infra)red. That implies a shift in frequency that doesn't happens. Light just gets absorbed by the oxygen, and the sensor detects the light intensity.
There is also other reasons why the light pulses at a specific frequency.
Sorry if I sound pedantic. It just a small correction that doesn't change the basic ideas of the video. Which is awesome, by the way.
7:09 I did have a good laugh. Well, they always tell you that YOU MUST have labels on your axis. Yours make perfect sense. I am going to borrow this. I have an electrical consumption graph (a non zero derivative) that is "meant" to be viewed over 24 hours with window sizes of 1 hour. So it's units are kwh/h ... but if you zoom in or out it starts reading in units of -\_(:-))_/-
Really awesome, This should be followed up by a research paper
What people really wanted was a watch that could measure blood sugar. The GlucoWatch accomplished this by running a weak electric current through your skin, calibrating against a finger stick, and then using changes in the current to estimate changes to the glucose concentration of the interstitial fluid in real time (a proxy for blood glucose). Too bad it hurt and never really worked well.
That would've been amazing to have back when I was having to finger prick every 4 hours! I've felt low electric currents before, and while a bit uncomfortable, I think I'd rather have dealt with that than finger pricks. But that's just an assumption based off of my time doing finger pricks and never having tried the device mentioned. Cool in theory however!
It made me so happy to see R being used for cool novel ideas like this. Great video!
0:32 I've seen some very..... Interesting parties start this way.
I love your videos. You break things down so well. Any other person's video would be twice as long with unnecessary fluff
7:04 I love the axes. 🤣
That picture where the Infrared shines through your body is AMAZING!!!
One of the problems I have with the wristwatch optical sensors vs the electrical chest sensors is that while I'm trail running, unless the watch is uncomfortably tight to my wrist, the amount of signal processing to extract the heartrate from the constant rhythmic jostling of the optics against my wrist is going to yield a pretty rough guess polluted with rapid movement (low frequency noise, coherent to a heartbeat's bandwidth), whereas the chest strap, is relying on a more consistent electrical capacitance undulation despite vibration. Wavelet conformity during periods of high activity is easier to pull from the electrical measurement whereas you can induce perceived arrhythmia within the data of a bouncing optical sensor.
It's best to just continue using an ECG chest strap. Really, it's always best to use one if you care about accuracy. As long as it's kept adequately lubricated. The only time it's really an issue is when swimming, especially pool swimming (well, and obviously high diving).
@@tams805 oya hard to get a pulse when the strap is around ur ankles
6:09 - Amazing! Sometimes I lay awake and listen to my own heart beats. I've noticed if I take a deep breath my heart suddenly takes an extra beat. It's reassuring to get confirmed that this is a common phenomenon.
I think I saw something similar a couple of years ago, but it was more nefarious, like using that same detection of the change of colour of the face due to blood pulse for spying.
I have absorbed so much of the information you have distributed through your videos. Such a wide array of knowledge that I will surely pass on to my son.
I think the flash green light could also be energy saving, as you just need a few samples per second the deduce the hear rate.
Turning something on and off rapidly doesn't save much energy, and in some cases uses more. Especially with a tiny led
@@vappyreon1176 that’s not correct
@@JasonRennieWTF clock chips draw power, and it's literally a tiny led. Idk if you're new to LEDs but they're pretty light on power draw all things considered
@@vappyreon1176 That is not true at all. LEDs are typically dimmed by pulse width modulation - turning them on and off very quickly. And it saves power proportional to the reduced light output.
That's crazy! Because I've seen that on people's faces before, but I didn't know what I was seeing!! This is so cool to know. I'm going to start paying more attention to this.
Great video, although there has been a lot recently about this not working so reliably for people with darker skin. It would be interested to learn more about how it can be just as reliable in such cases.
I often thought it must be motion amplification via phase difference of light wavelengths, but boy this was mind blowing. I really could use this information.
I really appreciate how you break your videos down. You make science approachable, fun and cool. Thanks!
I really really appreciate the thumbnail with the purple light. I thought that’s clickbait and fake. But with the inferred it’s kinda real very nice eye catching thumbnail without being fake. Much appreciated.
The professor has done it again! Another great video by Steve Mould, Professor of RUclips!
4:11 and it goes wu dum, wu dum.
8:00 Therefore, the two lines/curves (reflected red light and reflected infrared light) should be out of phase (in anti-phase), not in-phase.
I already knew how the sensors worked but your video still made it fascinating. Very well illustrated and some great footage. Too bad we didn't get the IR view motion-amplified as well!
I've done heart and breathing rate measurements differently. If you put a piezo sensor inside for example a pillow and sit on it you can extract both. Just by filtering and amplifying the signal you get a nice clean result if you don't move that much. Really easy, reliable and cheap. :)
But it's not a contactless approach like the one in the video.
Hi @Steve Mould, please can you put warnings up before flashing lights (e.g. 5:35) in future? Thanks!
I wonder if this works just as well with people that have darker skin. There is already a known issue with pulseoxes that are less accurate if you have darker skin.
Came here to say the same exact thing surprised hardly anyone else noticed no mention of that....
just gonna agknowledge the fact that you're at 1.99M subscribers, godspeed steve mould
8:36 Will the ups and downs of the infra red line be reversed as compared to the red line?
This video deserves tens of million views. I think title and thumbnail is bit low key
It's good to remember that these optics-based detection methods are often calibrated based on lighter skin tones- for example, people of color can show oxygen saturation levels that are erroneously high. More work needs to be done to make medical equipment work for everyone!
Just to be clear, none of the gadgets used here (even the Polar H10 - though that is very accurate) are medical equipment.
Well, the stethoscope is, I guess.
My physics teacher is yelling at you for the axis labels! XD nice!
I was literally just looking at my smart watch last night and asking myself what the light does. My thoughts are starting to sync up with Steve Mould videos!
Wouldn't the infrared go down when the red light goes up if they react oppositely to the amount of oxygen in blood (i.e. there would be an intrinsic "phase" difference in the curves at about 8:20)?
I thought the infrared would be antiphase too
I think it's the amount of blood in the vessels that causes that periodic change, and the change due to oxygen concentration is less obvious until you do the math to extract that data from the ratio.
Steve is the most serious and goofy person I know 🤣
I'm glad that I discovered this channel from Smarter Everyday
I like how he went to infants in a hospital as a potential avenue for this kind of tech. I could also see this kind of tech (after some levels of advancement mind you) being used at security check points like airports/customs to see who's heart rate starts to get elevated as they get closer to the check point, or maybe just have random officers walking around to see someone who just got their luggage if their heart rate gets elevated although you have to be very careful about that.
The last thing we need is more security theater at airports.
@@dylan-5287 Well what you want and what we get are bound to be two separate things. I don't want to take my shoes off at the security check-in either but thanks to some dude stuffing some explosives in his shoes now we all have to do that. Thank god he didn't try to stuff the explosives in his "prison wallet" I'd never fly again if they put us through that level of scrutiny.
This is an unexpectedly quick answer to a question I have had for a really short time and I'm amazed with how this all works
Today we learnt that steve actually shaves his chest.
I had no idea. That is insanely clever.
Shared the video with a friend (You're welcome! 😉). He had a good comment. If you folks can do this, then people doing surveillance can do it too. I'm picturing camera's all over an airport just looking for people who's pulse rates seem a bit too high... Or any port of entry, while the Border Agent is asking "Purpose of your visit?" and the AI is staring at your face to see if you get especially nervous.
Wow! 2 million. Congratulations. It doesn't seem very long since you hit 1 million.
The main reason why the green LED flashes is to save battery, that significantly increases battery life of the smartwatch. That's also the reason why these LEDs will only flash when the smartwatch senses movement, otherwise it just assumes it's not being used and enters battery saving mode. They can get away with the flashing because our heart rate is super slow when compared to the speeds at which the embedded electronics are capable of running at, so they can sample a few times a second as opposed to constantly reading the heart rate sensor input.
Andrews Channel is criminally underrated
Okey so this is actually kind of funny. I tend to have anxiety attacks from time to time and I’ve learn yo recognize them quite easily, to the point where I calm myself by just realize it’s a panic attack (and I’m not in any actual danger), except for yesterday where I had my biggest anxiety attack I’ve had in my life, where I even started to hyperventilate pretty badly. After a while I managed to calm down and breath semi normally again, and I started thinking that I should buy a pulse oximeter to be able to show my brain that it’s just a anxiety attack. The point is, I started to wonder how they worked and how they could measure the oxygen in the blood, and just as I was wondering that, scrolling though RUclips (to distract myself), this video popped up. I didn’t watching then because I was scared to even think about breathing and oxygen and all of that, but yeah, what a coincidence 😂
Is it okay to say that your videos gives me goosebumps every time.
film everyone and findout who are aliens
Wow that IR camera + torch view is coool!
"Reflected back"? Why not just "reflected"? Doesn't that already imply a "back"?
@@coinonaribbon: No, because "reflected" already _means_ "...back." Even if the angle doesn't bring the signal right back to the source.
One of the best video I watched on YT !!! Thanks Steve & Andrew ❤
Bio status: mined
Great work guys. Give authorities even more tools to monitor us. Now they can even know our heartbeat just by filming us.
Either you accept that technology is good and we keep moving forward, or conclude that it's bad and move back into caves. We can't draw the line at some point and say "this is the optimal level of technology for human kind".
7:10 this is the best graph I've ever seen 😅😂