This microscope uses touch
HTML-код
- Опубликовано: 17 май 2024
- Visit janestreet.com/amp to find out more about their Academy of Math and Programing
Gelsight is a microscope the presses gel into the object of study.
Thanks to The Sempre Group for showing me their machines! Find out more about them here:
thesempregroup.com
And here's a direct link to GelSight (for non-UK people this is a more useful link):
www.gelsight.com
You can buy my books here:
stevemould.com/books
You can support me on Patreon and get access to the exclusive Discord:
/ stevemould
just like these amazing people:
Brendan Williams
Edward Unthank
Frank Hereford
Jeremy Buchanan
Tj Steyn
Jeremy Cole
Matthew Cocke
Pavel Dubov
Alan Wilderland
Alnitak
DamTheGreat
David Whitesell
Grant Hay
Heather Liu
John Doe
John Zelinka
Lukas Biewald
Marshall Fitzpatrick
Matt
Nathan Blubaugh
Paul Warelis
Tim S Root
Alnitak
Damien Szerszinski
Lizzy and Jack
Twitter: / moulds
Instagram: / stevemouldscience
Facebook: / stevemouldscience
Buy nerdy maths things: mathsgear.co.uk 
Наука
Would *you* call it a microscope?
The sponsor is Jane Street. Find out about their Academy of Math and Programing here: janestreet.com/amp
have used janestreet before
Well, I wouldn't classify it as an imaging microscope. However, given that it's capable of allowing users to view and take very minute measurements, I think I'll categorise it as a profiling microscope.
I guess that still makes it a microscope 😁🤷🏾♂️🤷🏾♂️
Micro thigmoscope
Hey, at 3:05, I'd recommend you edit that part out, putting out your fingerprint like that might not be safe.
Edit: Yeah, we don't know who it's from or which finger it might be, so it can still be safe. Just wanted to let you know though.
Well microscopes touch objects with light or electrons so why not?
I'd love to know more about the gel they are using, It seems absolutely incredible that a gel would conform to such small details so perfectly and yet have no memory in that it returns to it's original shape.
I'll bet you there's some nice patents on the gel.
Either the video codec has an echo or the gel does have some memory.
You can see on the lego that it has some issues with cornering. Around the base of the lego nubs it has some issues. Stuff like that may get worked out though in the future.
@@PS-bc5qt I think that tracks with what the guy was saying about the limitations - it won't perfectly show sharp angles but for the application it's not a huge issue as you're not looking at that directly
My guess is nanoparticles, and the size of the nanoparticles determines the resolution.
I could probably watch about 12 hours of Steve pressing various objects into the gel and just describing what they are before moving onto the next one.
Right? I wonder what various kinds of feathers would look like - flight feathers vs down, etc
It's actually nuts that you're getting such a deep depth of field on such small things, and you can even move them around a few millimeters and still retain focus. And you can capture it all with video and then share it with other people. Amazing.
That's a really good point. My experience with regular microscopes is that the focus is touchy and the depth of field is very shallow, unlike here where the whole object is in focus.
The color are wack ....BUT the 3d efect are amazing
An alien wrote this comment
The wierd neon-colored image you can see used to create 3D model is called a normal map and it's meant to hold information about height of the object using different colours for different light angles. It is used widely in video games to create detail in low poly objects by changing how light reflects off of the surface.
I find it fascinating they used normal map and not a regular height map but given the technology of 6 light sources it makes all the sense
Height maps contain less information than normal maps. Each pixel's color channel in a normal map contains information on the normal (orientation) of the surface, whereas a height map only contains direct vertical information.
@@BiffGheek Paradoxically heightmaps can be said to have "more" information - you cannot recover height from normal, but you can generate normals from height - a 90 degree drop would not appear in a normal map.
@@BiffGheek It's possible to obtain the normal vector from a heightmap. It was a common trick in computer graphics to use the depth buffer for things like ambient occlusion, an algorithm which needed normal vectors but didn't need to be 100% accurate.
@@0xD1CEall I know about Ambient Occlusion is how it lags my computer more thany other graphics setting
As a 3D artist myself this was one of the most fascinating things I have ever seen in a long while. If produced massively across the globe. It could be really helpful in 3D Film Industry.
Could you elaborate on why you think this microscope could be helpful to 3D artists?
@@Spudcosmiccc maybe creating normal or roughness materials? its a bit too small though I think
Considering the human brain's ability to extract a sense of 3D from watching 2D movies, it might explain the abysmal failure of of 3D movies and TV sets every time there's been a push in that direction. If this can help 3D movies improve on a human's innate ability (and be anything more than a gimmick) I'm all for it.
this is litterally comercially available since 2015 ruclips.net/video/VN9ICs02vcY/видео.html
with full on 3d measurements since 2019
@@AlanBerger1337 It's $500, which is actually pretty reasonable (by that I mean somebody who has a use for something like this can actually get one)
im a 3d artist and i want one of these to make instant normalmaps from objects ,these look great
Yes! Inagine having super accurate normal maps in games. It would look amazing.
You can make normal maps of objects by taking 4 pictures with light from up, down, left and right and then composing them manually. Like this thing but of course without the whole "immune to transparency/reflection" thing. The composing would be inverting right and blending with left 50%, put in red channel, invert down and blend with up 50% and put in green. For blue, Nvidia's old normal map plugin for Photoshop could get close with its "normalize" feature from the red and green channel.
I know. The moment I saw that coin being lit from different sides, I immediately knew it was compiling a normalmap. I'd love this ability too, especially because I like making normalmaps of coins.
it instantly reminded me of normal maps as well xD
there's already plenty of software now that can use multiple images to "scan" and object and make a 3d model of it. i'd be surprised if there wasn't similar software that makes normal maps as well
This has WILD implications as a mobile highly accurate surface measurer. Damn, I want one just to have. I bet they're expensive. Industry really needs to have this. Feels revolutionary.
We could actually use that almost every day at work to measure engravings on the toolings for chocolate moulds. Would safe us a lot of time.
Super interesting 🧐👌🏼
what
This would be awesome to generate height maps for making smart materials in substance.
Just noticed the height map at 2:41 and wanted to plug it into substance.
Mistook height map and normal map as the same thing by mistake here. Thankfully people seem to get what I meant
This is _somewhat_ doable. I make a basic version of it in my tutorial series on convolution in Unreal Engine. The tutorials are quite dry but the example project is free to download.
It's not very straight forward to use as a tool though.
this is how normal maps are created, the RGB values on a normal map correspond to "lighting" the object from 3 different direction (on top, sideways, and from the bottom)
I wonder if they could make one that's 4x4m in size 🤔
@@Noclip420 The point of a microscope dissappears slowly xD
i got the same idea...
so i think that could really do a thing or two
That zigzag motion of the print head is "boustrophedonic". It is a literal reference to the motion of an ox ploughing a field, and you still usually see it in patents to describe print heads, scanners, etc.
I'm here to learn words like boustrophedonic
This concludes my night, I've learnt enough. Thank you for that nugget of information.
I'm here to learn where you learned this
Space filling curves
I learned what that word meant in the context of writing systems.
This is insane. We are getting to watch the first steps of an entire new tool that will change entire fields of study. It is amazing. I'm probably way more hyped about this then I should
Not at all! We're curious beings and seeing new innovations inspires our mindset!
The gel pad is a hexagon, and that fact just makes my brain light up in excitement and happiness all on its own. Hexagons are the best!
Steve: this needs a sound effect
Me: schlorp
Steve: schlorp
I feel it shoud soun like a ruber balloon crackling while it deflates beside an object
Being animals, some noises are built into the experience.
Its incredible how well that gel conforms to objects
I think it is the 80% of that technology.
I bet the gell have "working life time" ..... The gell Will be the thing Will replace regulary ..... 🤔🤔🤔
@@memesfromdeepspace1075 He actually did say that in the video, on the part with the stickiness. He said that it becomes sticky and needs replacing :)
what a *fascinating* tool. I personally don't have a lot of use for it, but I can just imagine machinists being absolutely delighted at the details and measurements you can get from it.
Never knew such thing existed.
But this is really a valuable tool for a lot of people who do precision work.
So thank you for brining it to my attention it deserves that.
I was expecting a video on an Atomic Force Microscope, but this is far more fascinating! What a novel way to collect 3d data!
You beat me, I am also expect an AFM.
yes exactly!
I also thought it was AFM.
I was thinking STM, but AFM makes more sense.
Yeah same here. Well played on his part.
uploaded 57 seconds ago, gotta binge this up
Ok, Steve, I'm so glad that youtube thought to put your video in my path. That's an odd microscope indeed, and the gel is strangely sensitive. Removing color does indeed leave us with form, texture, and shadow. Photometric Stereo? Ok then. I wonder what insects would look like with this system? Does that gel wear out? So it has a quality control inspection use, also interesting. Jane Street Academy... sounds excellent. Thank you for all of this information. Well done. But then you already know that.
So cool. Its like making a normal map of a object 😍
@05:38 I see a mountain in both cases. The shading in the detail around the perimeter comes from bulging shapes, not depressions. Also, if cast shadows are present in such cases, they clarify the situation.
Uh, no, I clearly see a crater
I agree. I think I've seen something like this before that actually flips between being a depression and a bulge, but in this specific photo, it's both a bulge for me. I can't make my brain think that it's a crater.
@@hamuelagulto796 same
Maybe it's because you already saw it as a mountain first?
Same here.
This would be great for artefact studies in archaeology
I was just thinking about that! I believe it would be amazing for understanding markings, but, if the resolution is enough, even some features of the fabrics of materials
So much easier to look for tooling marks and scratches to see how things were made
I'm doubtful about the willingness of the archaeologists to press the artifacts into a gel. Things could easily break from the force.
@ well I can imagine a lot of situation in which force it's not a problem. There are some artifacts, like terra sigillata (I don't remember now the english name, sorry) which may not suffer. Or stoneware
@ I was mainly thinking about markings on rustued metal, but on the other hand if you take (for example) a coin, you just scan both sides and you don't have to handle the coin nearly as much, which would make it far less likely to crumble.
As a mechanic this would be really useful to check for wear on parts quickly because most parts that we check for normal wear that occurs during the engine's lifetime is layered and sometimes you may be searching for a deeper scratch in the middle of thousands and it would eliminate the need to send parts for measurements in some cases.
I work with 3D making games for a living. Seeing those 3D models being created was so cool. Essentially using the same information that's used to generate and render normal and displacement maps.
With the crater illusion, I saw it as a mountain the whole time, even after seeing the shadows falling on it like a crater - the look of the inside of the crater looks so plateu like. Same with the indented text, in fact I think the top light source makes it stronger, and I did eventually recognise the crater, but I had to look closely at the shadows, and looking at the light side makes it look like a mountain no matter what.
I otoh saw the concave text immediately (after rotation), yet can't see the crater at all. It's always a mountain | plateau for me.
Same for me.
same
I had to work hard to see it as a crater, including switching to only one eye or looking away for a while. Even then, it would pop back up to mountain pretty quickly. I think it's because I saw it as mountain before rotation.
Yeah- I was able to resolve the crater in the second position and not the first, but even after resolving a crater, it was still more natural to see a mountain in both. And that's ignoring that my brain thought it was something small in the gel at the first glance.
Honestly, I'm most impressed by the sensitivity of that gel pad, that is incredible detail! I would have assumed it just mushed
wait I thought there was a camera on the inside looking at the back of the pad.?
@@wishywashy1153 There is a camera behind it. They're saying that it's amazing that you can squish something into this gel and see such incredible detail on the other side.
@@unwaveringflame2482 Thanks but you just restated what I and he said.
@@wishywashy1153 You didn’t state anything, you asked a question to which “Unwavering Flame” gave you an answer.
5:00 Thanks for making the left/right arrangement compatible with the cross-eye technique! 😸👍
I work with lots of 3D LIDAR scans and one huge issue for us is the reflection off of reflective surfaces such as mirrors. Interseting how this gets rid of that, at least on a micro scale.
Just tape a gel cube to the sensor and drive into things!
Would training an algorithm be easier using this technique as a source of truth?
Small images, and maybe large images later, could be developed accurately with this, but would that information from this technique help you train a model faster? And further, would information about small things like the quarter or matchhead be useful for larger scale applications of LIDAR such as autonomous vehicles?
Mirrors look really interesting in a 3D picture. They're kind of like a hole.
This would be an incredible medium to make some kind of animated movie, even just a short little thing. What i'm imagining is in the same vein as the "A Boy And His Atom" animation made with individual atoms.
This is nowhere, anywhere, even remotely close to having the resolution to see atoms.
@@JustOneAsbesto I was not suggesting that it did, thanks though.
‘A Boy and His Micron’ maybe?
@@nickcoleman2765 yo thats perfect!
@@JustOneAsbesto I mean isn't technically everything that it sees made of atoms?
Mate, I love the content of your videos, and I've bought all of your posters which so simply put things into clear perspective, keep up the amazing work❤
Sometimes I get so excited to have finished my course and get to work with these amazing machines!
This seems similar to the atomic force microscope I was taught about, like the ones with the tips which touch the surface of the object!
4:37 That's called monocular depth estimation and it very much already exists. There are quite a few open-source models that can estimate depth fairly well from a single flat image, and if you have multiple images from different angles there are neural networks that can build up an accurate model of the entire scene, sufficient to move a virtual "camera" around and through the scene and create new views in the process.
But how reliable and accurate are they? Probably not good enough for the purposes these camera gel deforming sensing things are bought for?
I saw this (specifically the tiny GelSight mini) recently and thought "what would anyone use this for?". Now that I actually see the accuracy and resolution it has it makes sense
I think my reaction to your example at 6:00 brings up an interesting point. See, I DIDNT experience the crater illusion when you used the touch microscope, and I think it's because humans are VERY good at context clues and learning; Because you'd shown me other visuals of the touch microscope, I was able to tell from the patterning around the letters that a flat, intended surface was pressed against the microscope, and that the letters were smooth because nothing was touching the gel surface. If they really had been protruding from the block, it would be the letters that had texturing, not the background.
Was really wondering about how easily it was contaminated and how frequently you would need to replace the gel, how expensive is the replacement and how easy is the process? It looked like everything you had on there had some dust or hairs, it seemed like it wasn't like you were constantly scanning dirty pieces but instead that the microscope itself had issue. Incredibly cool technology, I even want one but the dirt issue seems hard to deal with.
Also if the subject's too fragile, the scope itself might impact and disrupt it. This'd be useless for microscopic stuff
5:45 I can’t really see the crater. I understand why the human mind SHOULD think that it’s a crater, but I can’t see the crater. Sometimes I get the slightest illusion it might be dipping down, but it doesn’t hold.
Same
I think it's because the perimeter of the crater is all lit
Nearly every mundane object under that type of imaging looks quite mesmerizing! I actually thought your stubble was one of the more interesting ones. It really shows just how cleanly the blades cut the hairs at the ends. The draping problem was the first thing that came to mind as someone who has done a lot of vacuum forming. But that’s very interesting that it’s not much of an issue as long as they can get the depth value. And as long as it’s not deeper than it is wide.
This video was absolutely fascinating. Superb. Really nicely pitched script.
What if you press two of these against each other? Will the universe achieve singularity?
8:55 I was not ready for this...
That’s amazing. Fantastic device
now we have your fingerprint and can unlock your phone
😂
I had a stroke trying to read this
4:23
"Your brain doesn't think 'Oh, This region is dark because it's facing away from the light source.' You know that it's:"
PAINT
"The eyeball!"
FUCK
Mould is on another level with his science videos. He always finds the most interesting and obscure stuff.
This is pretty amazing! Nice find!
I imagine this could be incredibly useful for making normal maps for added detail in 3d modeling
Every one of your videos is a treasure. Thank you
8:54
Love Steve for moments like these
This multi-light setup that generates the depth and normal maps is the same method that is used to generate many textures and material used in games and film. You can do it yourself with a camera, a light and either free software or Adobe Substance. You can also remove the reflection from the images using cross polarisation by putting a polarising gel on your light source and in a different orientation on your camera.
The amount of detail was impressive.
wow, what an awesome tool :)
thanks Semper Group for letting Steve play with it, maybe send him more stuff to test eh 👍
I work at a jet engine company and we use these all the time! So cool to see!
to what end? wear inspection?
Your prints are pretty good :D
Nice layers no gap, no loose belts, nice
Really nive device. Astonishing how many details it still retains 8:43
In cartography it's called photocylinometry (aka: shape-from-shading)...it's how Magellan made 3D maps of Venus's surface with a synthetic aperture radar (SAR). It's been mostly replaced by lidar and interferometric SAR, which measure the 3D surface directly.
Really cool that it showed the depth of your mouse cursor in so many shots!
I would love a larger scale of this, it would be amazing for modelling small electronics where the manufacturer doesn't provide 3d files
I would sit on it
Oh I see the crater now. Focusing on the area outside the shape helped!
Focusing on the exterior near the dark side made my brain switch from mountain to crater, and Vice verse on the light side’s exterior
Omg this is a genius way to finish a step of creating ultra realistic 3D models and CGI renders, that’s very epic
0:02 ah yes! The polo mint!
6:03 "Yeah, it does look like it's sticking out!"
6:12 "Yeah, it does look like it's sticking out!"
lol, same!😁
It's kind of the same for me. Except I see it as indented for a few seconds and then it starts to look like it's sticking out again.
@@That_Awesome_Guy1 "Just when I thought I was in, they pull me back out"
It's a watch!
I've had to replace those little buggers one times too many.
I was actually shown this in school, it was so cool.
8:20 you should do all the numbers on the front and back next time too that'd be fun
Funni
"No one would see this face as concave" 6:27
Steve how dare you put this curse on me.
You just made a bowl out of a human
Okay at 6:13 im seeing it as opposite of what you said and its confusing me as to which is real now
Coolest technology I've seen in years. Makes you think about what else might be out there and kept from the public.
No money in keeping it from the public.
5:50 I still see a mountain :()
Same,
Me too
love watching these videos. thanks.
this would be amazing for scanning art pieces
Woah, now that's some impressive photogrametry
8:36 YO HOLD UP
Why is literally no one else talking about this !?😂
Tes5icle
This is honestly amazing! You're so lucky to be able to play with. Is there any information on whether this is going to be sold commercially? I'd love to get my hands on this.
In the intro he shows a picture of the company 'The Sempre Group' and their website. The device is there, no price listed but "contact us" so probably very expensive.
@@JeffBilkins that’s silly, it doesn’t look that hard to make
@@tigergold5990 It depend...
What's the resolution of the camera ?
What's the focal point of the camera ?
What's the material of the pad ?
How many time did the software got developped ?
Without thoses anwsers, you cannot estimate the price.
It's like saying that VR headset do not look so expensive... Yeah but it's a small 2k screen, a very narrow focal point, and a very difficulte to make tracking device.
I'm pretty sure that given the software and the regular change of the pad, it's not sold but got offer at a service.
@@tigergold5990 Probably isn't expensive to make but they will want to recoup all their research and software development.
TigerGold59, I definitely think you should make one! Maybe sell it at a lower price then this one.
Nailed it! I recognized the battery from the mystery device after a bit of pondering, and then just guessed the most common application of that kind of battery.
I replaced a watch battery a month ago and remembered that. Lol
I'm a structures technician on aircraft and we don't "take a cast" of a scratch. We use a depth gage, the depth gage has a pin with a fine point you press down on the surface and it will tell you the depth of the scratch compared to the surface around it. Depending on the depth, it could be considered negligible damage, repairable damage, or out of limits.
Complete side note: The free software “Materialize” does an okay job at taking a single color image and extracting depth, reflectivity, and other properties. It was used to remaster the Uncharted game series.
The Crater illusion doesn't work for me. Neither in the crater image nor with the letters. Both times it just looks like it's sticking out. Never do I perceive an indentation
After it is rotated, the light is actually coming from the top-left. You can clearly see shadows being cast into the crater from the lip of the crater in the top-left.
@@JustOneAsbesto my brain almost always interprets it as an extrusion, even when skipping straight to the crater image, except only once I managed to see it as a crater. I have much better luck with the preview thumbnail though.
That concave face is the most horrifying thing I've seen in a long time. Thanks Steve for putting me onto the phenomenon of seeing faces as being concave when lit from below
This is like real life normal mapping. Super cool
5:38 I can’t see a crater…
0:25 It's definitely a watch. And 2:42, this image looks so similar to a "normal map" used as a bump texture in 3D software.
Imagine if objects scanned this way were used as bump maps in games. It would look amazing.
Very interesting Steve. Thanks
Hey man I’m an aircraft tech and just wanted to give some input on other ways we can measure damage nondestructively.
We have ultrasonic testers that work much like ultrasound.
We also have eddy current which is extremely interesting. We have blanks of different materials; aluminum, titanium, magnesium, steel, and others. We use a calibrated head on the tool that uses electromagnetic eddy currents that are then disrupted by the damage.
Using those methods we can actually measure and find cracks and scratches which may not even be visible!
I'd love to see a record under this
The fabric you showed was not woven, but in fact knit. Woven fabrics have parallel lines.
**yes** thank you.
My past 5+ years of EM experience using various electron detectors approves this video on a fundamental level.
Metrology has to be some of the most fascinating disciplines
I thought of this like 10 years ago (minus the gel pad), to generate normal maps from real surfaces. Of course I didn't have the techical know-how, the scale would have been much larger and it didn't seam feasible to paint everything I wanted to capture.
When you have a "stupid", you sometimes are up to something good.
yup, that's pretty much how normal maps are generated, at least in the traditional sense of how we assign the RGB values in a normal map
this is litterally comercially available since 2015 ruclips.net/video/VN9ICs02vcY/видео.html
with full on 3d measurements since 2019
Thats amazing. It's like a digital terrain model, but for tiny things
Absolutely fascinating
I have absolutely no practical use for a device like this, and yet I absolutely must have it!!!
Weird, at 5:23 I got a sudden urge to throw my computer off a high counter or something. This is such a cool device! I was thinking from the thumbnail and title that we’d be looking at an AFM or something. What a fun surprise!
Yeah, *somebody* has a bead-chain fixation!
As a 3D modeller, we have a render pass called default, which is just a grey material applied to the scene or object, so others artists can analyze, appreciate... the shape, i would love one of these microscopes, because is what we do in our job, until we texture, and often we can't separate the color from the shape from images, even if you look up close, its hard to wrap your head around the shape, and the shape only, this is amazing.
I expect that got different names depending on what app you're using
@@TiagoTiagoT yes it is, sometimes is hard to express some 3D terms, and AOV's sounded difficult to explains. I have that same problem when talking about smooth, in 3ds is called turbosmooth/opensubdiv, on blender there is shade smooth and one other, in zbrush is subd...
Where do 3D modelers usually ger their textures from? is there like a marketplace for texture maps?
@@alejandroespinosa9030 There are many, including some free ones, and people can also create their own textures (from photos or images created in drawing apps and stuff, and other sources of images). And in recent times, there's even AI apps you can use to generate textures. And with many 3D apps you can also create textures procedurally and then bake the result into image files. And if you're talking about stuff used on characters and other complex models, those textures with a bunch of disjointed patches or that look like the character was skinned and turned into a rug, in many cases you get the images via one or more of the methods above, and manually place them on the models, either by polygon (which can be done manually or automatically, with varying levels of success, and so it's often automatic with manual touch-ups) or drawing on the models, and then in the 3D app you bake that combination into a final image (or more than one image) that has the different parts of the model mapped to specific parts of the image(s) with the combined textures.
I saw another one that vibrates a microscopic filament over a surface to create a microscopic 3D model of it. Really cool!
Dear Steve, I work with a microscope which is called a scanning ion conductance microscope (SICM). We can measure objects on the sub um scale. Importantly, its non contact and doesn't need sample labelling. It works particularly well with cardiomyocytes. Let me know if you want to have a go! Ben
Does the gel leave a residue behind to the point to where you have to refill the gel over time?
I have to say Steve, you come up with pretty amazing & interesting stuff in your videos !
5:05 i crosseyed to see this in 3D, I recommend everyone to learn this technique, its so fun, however hard in the beginning
Love the double seaslug clip you can use the inverted "magic eye" technique to view a 3D video