There was a French canadian show where they parody this. From security footage, they zoomed in a client's sweat drop, and now had a 360 view of the store, and zoomed toward the door where the suspect was exiting, then zoomed in his hand, he was holding a credit card, then zoomed in to read the guy's name. But that CSI eye reflection tops it all because it was intended to be serious.
@@jsl151850b No, the Red Dwarf scene was "uncrop, forehead, through a window at other end of street, a doorknob, a faucet, out the window again, a car door, a building's glass door. Video ID 2aINa6tg3fo
@@lightcycle87 I think it's when Bart was trying to find out what made skinner such a boring square, so it's the episode where Bart finds a prankster who put worms into a swimming pool. I could Google the episode name, but I'd guess its called 'pranks and all' or something like that.
I have had better results with Topaz Video Enhance AI, because that does not only use the one pixel, but takes into account how it changes over time. It's a bit like image stacking.
There was a case I remember reading about where the investigator had a blurry image of a license plate but he also had precise locations of the lights and the camera (it was at night) and geometry for the car and street. So he recreated the scene on a computer and rendered it many, many times. Each time he used a different license plate number trying to match the pixels in the blurry original with the pixels in the blurry render. He ended up with a few matching license plate numbers (fewer than 10 I think), which was enough to find the car and solve the case. IIRC this was the first time this technique was used. Seems like you guys are well qualified to do something similar.
this is the same reason why blurring or mosaic-ing text is not enough to obfuscate it since you can just iterate through blurred text and see what matches
@@rp9674 I think the principle is that it was able to narrow down a list of potential suspects who could then be investigated and one of the suspects panned out. So the license plate enhancement couldn't be used to convict the person as it was inconclusive, but the evidence from the investigate that followed could. Now it would be in error to assume that one of that group of suspects must be guilty and that the most plausible suspect was in fact guilty because the pool of created suspects might not include the guilty party. But it would not be in error to investigate and see if you could find the guilty person among them with more conclusive evidence. Like maybe you eliminate the vast majority who had great alibis, then focus on the remaining few.
A tv show that really took the whole enhance trope to the extreme was one episode of Red Dwarf, where they were zooming and enhaceing through reflections in reflections several times over and each time was just as clear and crisp as the first.
Y'all should do "Sound Designers React!" Like: How they made the T-Rex or Godzilla roar; How they make body sounds for creatures that don't even exist; Even the subtle ways they have to add basic clothing, movement, and walking sounds into animated films. You know those punching sounds in old action movies? WHAT IS THAT?
We need that series considering the fact that I've just came to know that the dolphin sounds use to censor slurs was actually the sound of a kookaburra in faster speed.
Oh I would love if they did that. I remember there was a tutorial they did once on punching sound effects, but I'd love to see a bunch of audio stuff broken down.
I love how they always have the tech furiously click-clacking away at the keyboard for these enhance sequences, as if every calculation by the computer required a key press
@@Shade01982 I would argue that using this tired old cliche scene (wait what’s that? Zoom and enhance!) to advance the plot already makes for a boring show
The most impressive thing isn't the enhancement, but the fact the investigator needs to tell the technician to enhance. I mean, if the algorithm always work why even present the blurred image? The image/video app should just enhance by default when zooming in
What if the reason they don't enhance immediately is because the computing power is so much, every time you press enhance it costs thousand of dollars lol I can imagine if such a thing is even possible it would require using some super computers to completely recreate things
A goat must be blood sacrificed every-time they enhance, so they try not to do it too much (blood sacrifice is the only logical answer for there enhance "algorithm")
So you’re just ignoring when they actually enhance? When they were looking at license plates, that’s the technology needed. It is just based on numbers because we don’t have the processing power to be able to enhance a hat.
Several years ago in the UK a driver was charged for speeding caught by a camera. They used computer algorithm to read the plate and that was submitted as evidence. The defence asked for a copy of the original picture where the plate was completely unreadable. The defence argued the evidence was generated by a computer algorithm and who can say how accurate it was. The defence won.
"But . . . they did it on CSI!" I think prosecutors are hoping for jurors who don't know a thing about image enhancement. I'm glad the defense saw through the state's BS.
I used to work in this area, so here's some insight: The rule of thumb is that any image clarification needs to be performed in a way that can be reproduced by a third party using the same tools. and by "reproduced" I mean each step in the process needs to produce identical md5 or sha hash values. Some people may be using AI, but I wouldn't touch that stuff.
Is this so that it is verifiable in court? Really brings up an interesting topic in the near future as more and more black box algorithms are used to run things.
@@philipfahy9658 Yeah, basically. Black boxes can be a nightmare. As a technician, I documented my process as best I could. If opposing council has an issue with my results from a black box, they may just have to bring it up with whoever created it.
@@peterehowell Yeah and it gets worrying when you realize how poorly most of our politicians understand the internet. I can only imagine what level of familiarity or lack thereof different public institutions have with this technology.
If Kyle Rittenhouse trial is representative of your average trial then the CSI one is closer to reality where the prosecution just phtoshops shit in to fake evidence.
@@XAJUSS And yet it was still allowed in. Later we had the HD recording leaked that showed that the "reconstruction" is bullshit, they cropped, reduced the resolution and sharpened it up to make sure you cant tell whats going on. But yeah, the prosecution was awful, and showed how corrupt the legal system is. My favorite part was when the prosecutor pointed a gun at the jury with finger on the trigger while talking about gun safety, peak comedy ...
Enhance! Great video. I’m actually blown away by how much you’re all able to do. I remember using genuine fractals ten years ago, it’s come a long ways with AI.
It really has! the licence plate demo straight up blew my mind with how the AI took basically nothing, just incomprehensible smudges, and figured out what it said.
@@justleo5997 The license plate demo didn't use AI (5:18 - 5:35), Niko explicitly points out how that was just mathematical algorithms. If the application did use AI there would be no way to guarantee what was real information and what was generated information. Given a solid enough model and the correct type of AI algo it could be sort of possible to do this with a high enough confidence to have this be admissible evidence, but I wouldn't trust generative AI to be able to do that, maybe a neural net that is well tuned but in general nothing is better than just solid math. Not everything is AI, software and math have been doing these things incredibly well for a long time.... I guess the Mark Ruffalo thing was impressive as long as you discount that the team was primed with information about the image, and then refined the generation with the additional information. This would mean that if used in a real court, not only would a eye-witness testimony (which at this point DNA evidence and project innocence have proven eye witness testimony is nearly never reliable) be a part of this, but the results would be dependent on the type of model, the input parameters, and the model training to perform the enhancement of the "Evidence". AI is a great tool for some use cases, but its not a panacea, and its certainly not something that should be trusted with matters of fact.
The mark ruffalo thing could benefit from some temporal solver. You know, combining data from 30 frames instead of one. Is there any AI-solvers that could handle that?
Wouldn't that just reinforce the blocky, pixelated features? Give the AI 30 frames of black holes for eyes and over exposed face and it will just assume that's what it's supposed to look like and not correct it.
@@radioethiopiate It's how jpg handles brightness, combining some greyscale preset images (quantization table, technically) to generate a more detailed composition, the only issue for this usage case is nailing the alignment, but I've manually done this in the past and athought still very soft, figures can become way more recognizable.
I saw some tiktok about something like this where a guy took a video of a spinning record and then took the average of all the frames in photoshop or smth and it was suddenly really readable
As a graphics artist at a TV station, I want to say THANK YOU for doing this video! Cops bring video to us all the time, and my producers are always saying, "just zoom in and enhance this"
One could argue that the CSi "Enhance" feature exists because the team has huge arrays of sensors lodged in every crevice of everywhere in the world, and the Enhance feature simply loads in data from those sensors and displays it in a way that is consistent with the viewing angle of the footage, or in some cases, completely inconsistent with the viewing angle for a better view.
@@SOLIDSNAKE. No, it really doesn't. It kinda would make sense in futuristic scenes like Star Trek or something to that effect, but a scene that takes place in current day, no. Even assuming all those devices are even aimed at what you want to see, the majority of those are not going to be of any better quality than what you already have. The resolution is not going to get any better. You know just have two blurry images instead of one.
not sure if that'd explain why the image of "Nuevitas" was flipped vertically clearly, they thought - well, words appear backwards in a mirror, so we'll flip the image horizontally, then rotate it to where we need. That'll be fine, right?
The only "Zoom and Enhance" that I ever let slide was in the Loki show. The pocket computers of the TVA had a pretty low resolution display, but had very high resolution surveillance footage. In that instance it made sense that you could scale a part of the image up without losing detail.
There was a TV show called I think Continuum, where the lady was from 200 years in the future and her nanosuit thing accessed the security cameras of every business downtown and created a 3d reconstruction of a vehicle moving from their footage and in cases like that, yeah just go with it. For the TVA, they can do whatever they want, they have Time Magic
I'd like to imagine the CSI crew just has access to an advanced neural net with centuries of training, just vibing underneath their building. Catching criminals is the only way to satisfy its bloodlust
I watched Enemy of the State a couple weeks ago and that scene caught my eye, something they did actually added to the legitimacy, which was they called out that they weren't able to look "inside" the bag, and what they found "could just be a shadow" or "the bag shifted". I liked that they tried to ground it in reality somewhat by giving it limitations.
The car numberplate goes way beyond sharpening, it's done with deconvolution. It's been in the Wolfram Language (Mathematica) for a long time. It's not as easy as you'll see it described as because slight errors in the lens shape, or rounding in the calculation etc… blow up into a huge mess really easily.
@@plzletmebefrankthey called it both sharpening and motion blur, you can use deconvolution to remove both motion and disc blur. The ripples around the edge are an indicator.
@@alan2here Fair. They are probably just using terms people like me can understand though. It's definitely more complicated than they make it out to be, but the software hides a lot of that complexity within itself.
That's something I hadn't thought about! If you're talking about trying to flatten an angled image (as per the second license plate example in this vid), does the type of lens used in the original camera make a difference? Like, do you need to know what kind it is so you can calculate how much it may be distorting the image? Or can the software just work out all of that?
Well, google maps and other civilian services provide satellite photos that are clear enough you can sometimes spot individuals on them. Stuff not for public access like government and military sats should be of even higher fidelity, so it's not particularly impossible.
while not satellites..... there are actual military drones these days that can do that (UAV miltary drones that fly around cloud height, some use up to 500 separate sensors similar to a smartphone and than record and send that feed to a technician in real time). the key difference with high tech miltray drones is they are equivalent to 2500 combined megapixles.......for perspective 4k video is 8 megapixels
@@DarthBiomech Google map doesn't use satellite data for the most detailed zoom levels. They use photos from planes overflying areas, which is also why the maximum detail level can vary from area to area, since it depends on whether they have airplane photos from each area.
@@Carewolf Commercial satellite resolution used to be limited to 50cm/pixel by U.S. law until 2014 - then they allowed up to 30cm/px native resolution, however some companies started offering even better 'synthetic' resolution - basically A.I. upscaled. Best I've seen is 15cm/px by Maxar. Spy satellites are said to have resolution up to 5cm/px which is still not enough to read license plates or recognize faces. It's much easier, cheaper and far less regulated to achieve higher resolutions using planes or drones.
We saw this unfold with the Kenosha self defense case. Prosecution was trying to submit a video as evidence that had been enhanced by software to make an argument. Luckily that argument wasn’t even clear with the enhancement, but it could be potentially dangerous using enhanced video as admissible evidence in court.
In order to zoom in on tiny objects on security footage like they show in the movies, there camera resolution would have to be extremely high to pick out those details.
Mine is the lack of marksmanship by law enforcement officers that have supposedly spent years training with firearms missing shots that I know I could make cold even with my limited ammo supply and range time.
@@phosphatepod Very true, I'll download some more RAM, switch my editor to green highlights, and deploy some swanky UI loading bars with flashing red warning labels. FBI will stand no chance.
RLM recently brought this up if I recall correctly. It is fun to imagine how Klingon culture deals with scientists and nerds. They've got starships and all, but...how?
@@FangsFirst they probably have a planet in the empire that is tech support with a conquered people on it. "Welcome to Zainar Technologies how can I help", "Tell me how to make the view screen work WORM", "So I understand you have a problem with the view screen, is this a Brell or a K'tinga?", "THIS IS A VORCHA P'TAK", "Oooh sorry, the Vorcha uses an M117 Digiplex Viewscreen, you want a different support line. Please demonstrate your honor by holding." *music starts playing*. Klingon Captain: "I'm not dishonorable, I will wait through your music. *taps foot*"
To be fair, it wasn't the Klingons that pulled it off. Soran was the one smart enough to bug Geordi's visor. The Klingons just had to watch the footage.
Appreciate the emphasis that AI upscaling invents details from nothing, it doesn't reconstruct real ones. Too often people have the misconception that the upscaled image is supposed to be accurate in any way, shape or form. Imagine the damage if a tech-illiterate judge accepted something like this as evidence
I mean, your brain does the same thing for what you're actually looking at in real life, so this is really just semantics. The AI definitely does *not* invent details from nothing. It reconstructs (yes, reconstructs) detail from context. The real question is how accurate the AI is at reconstructing this detail. That VFX artists don't fully understand upscaling AI is not surprising.
@@skipfred the difference between inventing and reconstructing (the way you're using the word) in this context is just semantics. You can't use AI to prove or even indicate what was there irl
@@soundninja99 The way I'm using them is exactly how OP used them, so yes it is semantics, that was my point, but it's not a semantic argument because we're agreeing on OP's original usage. Saying that you can't use AI to prove or indicate what was there is like saying that you can't use blood splatters to prove or indicate that a stabbing occurred. Yes, you very obviously can because of *context*. The question is whether or not the investigator (or the AI) is capable and reliable, not whether it's technically feasible.
Thoughts while watching the video: The potion seller inverse distortion reminded me of a case where someone had censored their face using the "swirl" filter in some software and it was easily reversed by applying the swirl in the opposite direction. Second: You can do some limited superresolution if you consider the movement of the camera or the subject in front of the camera because if the subject moves by a sub-pixel amount, it can theoretically give you more information than a perfectly static image. I remember using filters like that like over a decade ago. The results aren't mindblowing, but they're pretty good. I think the software was Anti Lamenessing Engine, after a short websearch. Third: I remember seeing people dunking on some magic upscaling algorithm after using a low resolution photo of Barack Obama as the input and the algorithm outputted some white guy.
Yeah, I'm pretty sure you could get a much better image of Mark Ruffalo if you took the raw footage, then motion tracked the face and stabilised it accounting for distortion, then superimposed all of the frames and averaged them out. If you think about it it's the same idea as the motion blur number plate example Niko was showing, except spread out frame by frame in the video. If you took all the frames and composed them together you can get more data out of it.
I was thinking exactly the same thing. I was half expecting the swirl face case to come up in the video. It was actually a nice case of karma. The guy in question was a prolific producer or trader of child pornography and his online avatar image was his face made unrecognisable - he assumed - by that swirl algorithm. The police 'unswirled' the image and then found and arrested him. His name is Christopher Paul Neil and he even has a wikipedia page if you're interested in the story.
@@bucklogos That's the type of thing intelligence agencies were doing years ago with specialized software that could provide greater resolution than what the camera alone could provide. One guy who had done work on satellites, though limited in what he could say, pointed out that while a camera in orbit taking a picture had a certain minimal resolution, taking _two_ images in rapid succession could, hypothetically and with the right processing, be functionally the equivalent of taking a single image with the optics of the camera being the size of how far the satellite had traveled between images, thus vastly greater resolution.
@@bucklogos I agree, stacking multiple frames from video should give additional information (after correction for changes in angle). Astronomy software such as Registax does something similar
Would love for you to break down scenes where characters 'hack into the mainframe', or take over systems. Usually its verbal garbage with visual garbage so would be awesome to see them break it down!
NCIS has a pretty good one where 2 characters to counter-hack the mainframe by pressing random keys on the same keyboard. Like they each take one side of the keyboard, and then another character walks in and unplugs the computer. It goes from complete fantasy to the easiest and most realistic solution in a single scene and its fucking hilarious.
I know this isn’t a “react” video, but I do like how Wren is genuinely reacting to CGI stuff. That being said, what doesn’t Wren react to? 😂 love you guys
The even more ridiculous tech CSI had was when they would magically have recreations of the crime scenes in 3D with animation to go with it. I think with Lidar scans you could get close to mapping the room but all the animation stuff that happened seemingly automatically was ridiculous. As an animator myself I always laughed at that stuff.
@@xWood4000 actually in batman arkham origins they do make a holoprojeciton of the crime scene and you have to connect the evidence to fill in missing data, is pretty cool
One of my favorite "zoom and enhance" scenes was from Gotham. Gordon was looking at a tiny TV wired to a crappy security camera. The suspect was wearing his work shirt, but the logo was blurry. So Gordon grabs his partner's glasses, and holds them up like a magnifying glass. Bloom! Crystal clear!
I remember seeing papers or technical demos on "enhancing" / de-blurring video frames by looking at the frames as a set, rather than individually. That can produce even more detail. Essentially, the lens and camera is constant so small movements result in features being pixelated in a variety of ways, which in turn provides sub-pixel level information.
More detail could definitely be drawn from the footage this way if we had the ideal algorithm. The Google Pixel’s camera does this to enhance its clarity. It will take a quick set of photos and combines the info captured in them to allow an incredibly impressive improvement to its digital zoom. Our eyes actually do the same thing. If look closely a focused eye doesn’t stay still, it has these microjitters essentially using a “set of frames” from slightly different offsets to get as much information as possible from it sensor cells.
"Is there police departments using this kind of tech" yes, a fair amount I reckon and fairly advanced too. Some students from the computer graphics major at my uni worked on a project requested by the police to generate 3D heads from mugshots to facilitate identification on videosurveillance footage. Basically they would position the 3D head at the same angle that the camera is watching the real person for comparison and that would help them decide wether or not it is the person in the video they are looking for.
Anybody that used VR goggles to try to read small text might understand this: If you look at the text with no head movement it's a blurry, unreadable mess. If you move your head in a circle, the letters become visible BECAUSE you're actually moving the physical pixels in the headset around "fixed" digital image where what they're averaging is changing letting your brain have more info to interpret. That's also why a good trick when watching videos in VR headsets is to set the video to somewhat larger size than your field of view (FOV), such as have the video's vertical size match your vertical FOV. This makes you want to look left and right more to see the video giving more virtual information than the raw pixel count would suggest.
That first paragraph described what temporal upscaling is essentially. Just done within your brain. It's the most high level straight upscaling method we have, reconstruction is the hard part to tune for a clear image though.
A real world analog might be riding in a car past a slat fence and being able to see what's going on on the other side almost as I'd there was nothing in the way. But if you stop suddenly you can only see through the slits. The human brain is great at averaging information over time I to a synthetic image.
I saw a really interesting technique for enhancing a while ago - the issue, as they say, is that you can't enhance something where the data just doesn't exist, where it's just a couple of pixels. But the technique I saw used video and processed multiple frames at the same time. The area of interest may only be a couple of pixels wide, but each frame captures those pixels slightly differently. By combining the data from many frames, it's possible to find artefacts far smaller than the static pixels that contain them.
Yup, the whole time I was thinking of this. I don't remember where I saw it but it is definitely possible to accurately calculate sub-pixel details by using the data from multiple frames, if the subject shifts even a fraction of a pixel all the pixels will shift slightly in brightness so the more frames the more detail can be extracted
I actually tried this on the footage in the video this afternoon. Unfortunately (not helped by youtube compression) the results after stabilizing and stacking weren't very good. But perhaps with access to the original raw footage it would be better.
I was thinking this same thing the whole time. If you have, say, 3 seconds where a face is visible (and moving) then you have potentially 60-180 different versions of that object. Then instead of training the AI with 'real world' data, you select the clearest frame and train the AI with the other 179 frames. I'm sure it's much more complicated than that but you're no longer using invented data, so more admissable perhaps.
Here's an idea: recreate "zooming into someone's eyeball to reveal the suspect in the reflection", but using 3d software and 3d scans of your heads. Position the head models so that when the 3d camera uses some ridiculously high focal length, you can actually see the reflection off the eyeball.
Unless I missed it in the video, I only heard one type of upscaling mentioned. There is upscaling that looks at only a single frame, but there is another type of upscaling that looks at the frame(s) before and after. By looking at additional frames, this provides significantly more information for upscaling.
That was the case for me too actually. We have decided to digitize and enhance old family photos, and when scrolling through and zooming in, I have discovered a bearded person face in the air between my family members. Originally it was blurred (out of focus) background made of shadows of a tree leaves. I was scared to shit, lol.
One technique they didn't talk about was image stacking, where you take multiple pictures and average the pixels values; this dramatically lowers grain, and when paired with upscaling this can get some pretty good results. You can't do this with just 1 picture but you can with video because each frame is like a single image which can go into collecting the data to create a more detailed final image.
I always heard about the "Enhance!" bit being ridiculous, but I never realized just HOW ridiculous it actually was til you showed these examples. It's worse than the parodies of it!
My favourite enhance scene is in season 1 of Barry. The detective asks the tech guys to enhance footage taken by a lipstick camera, they give each a look, and in the next scene as she's questioning the acting students she holds up the blurriest picture, everyone squints and start debating what it's supposed to look like.
You guys should've looked at an early classic "Enhance" scene from the original Blade Runner (1982). As I recall, Decker scans a polaroid, zooms in on a tiny, out-of-focus bathroom mirror in the background, fixes the focus, adds detail, and then rotates to see a label on a medicine bottle that WOULD be reflected in the mirror if the photo were taken from a different angle.
I would argue Blade Runner gets a pass because it's not a Polaroid, it's supposed to be a science-fiction-y holographic photograph with unspecified 3D-type features built into it, which the machine he inserts it into can manipulate. In the movie, he adjusts the perspective of the camera such that he sees around the frame of a doorway to reveal what's behind it, and I suppose in any case that's pretty implausible. But at least they make it clear this is all some technology that's foreign to us, something in the future that doesn't currently exist. CSI doesn't have this excuse.
The victim was standing in front of the mirror. We can't see the perp, but we can partially see the rest of the room in the image. What's that? Enhance the victim's eye in the mirror. It's a fishbowl. Apply inverse focal length and water light refraction. Zoom in on the fish's eye. It's our guy! It was the butler all along!
Another thing you could do with a video is use a super resolution algorithm, that utilizes more than one frame. That way really get an image with higher resolution than the input video.
@@blindleader42 As I understood it they just watched an example video of the software, but didn't use it themselves. Using multi frame super resolution on the CCTV video should yield much clearer results, if you get enough good frames.
There was an episode of CSI: Cyber where they tracked a guy down by finding his phone. Apparently when he touched the screen, it left a "digital fingerprint" made out of code that they could find, reconstruct, and identify the person by. Like the phone screen can randomly copy fingerprints onto the memory by any touch. Then the technique was never used again in the show. Couldn't keep watching after that.
It sounds like a rather extreme extrapolation of keystroke recognition, the existing software that can capture and individualise people's typing patterns?? 😆
It's worth mentioning that one of the first instances of "zoom and enhance" was from Blade Runner (1982), where it made sense as far-off futuristic sci-fi technology (even though Blade Runner was set in the then far future of 2019).
they are just extrapolating existing film photography, like for example they are already familiar with micro-film where they could zoom in and see a document otherwise unintelligible to an unaided eye.
this was an awesome video that was actually really interesting. it also felt like you guys said, 'hey we need to do a video. lets do that simple idea we had' and sat in some chairs and killed it.
Y'all should do a review of Robin Williams in Law and Order SVU. He brings up how the AI is guessing details in an enhanced image and gets it removed from court. I think it's in season 9.
I’d have loved to see multi-shot techniques applied to the problem. Use a couple of consecutive frames where the subject is still and light-path jitter will allow you to interpolate more pixels.
I feel like the Ai face generation upscaling thing they were using would have worked a whole lot better if the mark Ruffalo mask was in colour. if it looks for recognisable pixel clusters surely the colour of the pixel is also important
One thing they could have done with the security camera footage, which would be more complicated but seems like just the thing the Clonix software might also be capable of, is combine several frames to create a slightly more detailed version of the face. Essentially, this is comparable to what they did to get those images from the black holes with the Event Horizon project.
Please Do Muppets From Space! Would love to see you break down those effects. Especially the Noah scene, the Gonzo getting a message from Space scene, the invisibility spray scene, and the spaceship scene at the end! Thanks!
The "Enhance" from Blade Runner maybe COULD work. The reason is that its future-past cyberpunky aesthetic seems to still use some analogue technology, which could include film cameras. Because digital cameras use a limited pixel resolution, "enhancing" requires a lot of extrapolating gaps in information. However, when a picture is captured on film, it is captured with a chemical process which goes down to a molecular level. So as long as the lighting is sufficient, etc, it is actually possible to see extremely tiny details through close analysis.
Film grain is small but never on molecular lvl, hence all the ISO standards. Same problem, we cannot control particle size to that degree be it pnp-based sensors or silver precipitate particles. I think record with (gold) particles atm is nano scale or something
The other thing to note about AI upscaling is that it does not recover lost information not captured in the original image, only gives a best guess of the missing detail based on its image training.
Real talk, the first time I watched Enemy of the State, I actually assumed it was a "mistaken identity" movie because the agency had _framed_ Smith with that obviously fabricated footage. Even tween me was like, "Wait you can't see the reverse of this, they're obviously making it up," so I was waiting the whole runtime for the reveal of why they _really_ wanted to catch him. It was very confusing.
Possibly my favorite video you guys have done. This has long been my biggest pet peeve with CSI in particular, but I remember shouting angrily when they pulled it in X-Files.
I love the way that Red Dwarf did it: scan a photograph, uncrop the image, zoom in on the detail that was cropped out of the original image, zoom in on a piece of jewelry, enhance the reflection to get an image of the street, zoom in on a lamp post, look for piss droplets from a dog, zoom in, enhance the reflection off the piss, zoom in on a window, enhance reflection again, flip image horizontally, zoom and rotate, boom: the suspect's business address, which was in the phone book the whole time.
"You see that reflection?" "Yeah, sure do." "Zoom and enhance." *Swivels chair* "Sir, i can do exactly one of those things and it's not the useful one."
One thing to note is that the AI enhancement will actually have an advantage on the celebrity face as well, because as public figures celebrities are almost certainly in every face database used in AI research and development.
For astro photography I"ve used an program which can compose multiple lowres images to a higher resolution one and made more detial visible . Maybe something like that can be also used.
What's funny about the AI upscaling example, is that you could probably force it to give you "a reflection of the murderer" from a few pixels in someone's eyeball. You would get an image of a person from your training data set (or an amalgamation of people from it), not who you're looking for 😂
In college, in my Image Processing class, the Professor also worked with the forensics team of our local police. I'm so glad that Clonix made an appearance hahah I don't know about US law, but where I'm from, Niko is 100% correct about the usage of these tools in court. EVERY SINGLE TRANSFORMATION to the image is recorded and the final result can only be used as evidence if there are absolutely no info added. If the transformations oblige to this restrictions, you can apply the reverse of all transformations to the final image and have the exact same image as the initial one, before any processing. Because of this, no AI assisted enhancement are allowed to be used if you want to find evidence. No matter how small or imperceptible its effect is. So yeah, those image processing thingies that Clonix did are all bijecting functions (reversible math transformations). Similarly to Math's Topology.
Thinking out of the box here, but if a record was kept of a million monkeys doing a million single-step transforms (to borrow an ancient analogy,) and one of them manipulated an image into an acceptable state to use as evidence in a prosecution - one that could be reversed step-by-step into the original image - even though that resultant image had nothing to do with the actual perpetrator or crime (let's say manipulating the off-angle license plate of a waiting getaway vehicle into the license plate of the bank manager), how is this acceptable? Recording steps shouldn't be the defining point for acceptability; a minimal number of steps to get from unusable to 'believable' acceptable evidence should be of more concern.
@@mamapapaxp When I use the word "transformation", I'm strictly using mathematical terminology. Manipulation is different to transformation. You cannot transform an image into something that it isn't. Every transformation is applied to the entire image or it will cease to be homomorphic/affine (I don't recall the correct terminology, as there has been almost 8 years since I took this class and 6 since I did anything this Math heavy). So, transforming a "kinda 1" into a "7" will distort all other numbers, so it's a useless transformation. If you correctly apply them, math guarantees that if the end result is the license plate of the bank manager, then it WAS the bank manager's car's license plate. Now it's a matter of figuring out if the car was stolen, had the plate cloned, or if the manager was acting with them. Also, what allows a transformed image to be used in court as evidence is the fact that it doesn't add info to the original. Recording every single step made and each step being a transformation function are just to prove that what you did oblige to the restriction. The evidence can still be contested, in which case, the transformations will be evaluated to see if those are valid, or if there could be any ambiguity. As discussed on my previous paragraph, there are a lot of restrictions to which functions are allowed. They need to be transformation functions (using strictly math terminology here again), so bijection is just one of it's characteristics. A look at Wikipedia's article about Transformation Functions and their subtypes can help with that.
I actually think that the fact that it's video gives you way more data. You could definitely write software to pull extra details out of the video. In astrophotography you do something similar with lucky imaging to take pictures of planets. You basically take high framerate video and stack it. I'm convinced you could do it with facial recognition in low res video
I would like to add that there is one more layer that could theoretically be used to enhance security footage: other frames. In theory, you can mathematically determine, using how the pixels brighten and darken, more details about what's happening "between" pixels. Idk if there are any tools out there using that method currently.
This principle is used in reverse for video compression, The use interframes and see how pixel change between frames and only store the changes Which come to thing of it may interfere with attempts to go backwards to get more information, as pretty much all (compressed) video uses this kind of compression
The end conversation reminded me of a tv show. There was some old CSI SVU episode staring Robin Williams where Williams argued that the enhanced image guesses what the zoomed image looked like and therefore can’t be trusted to be accurate.
Red Dwarf, the newer 2009 mini-series, had a “Super Enhance” that not only extends the footage beyond the borders, enhance a reflection, enhance the zoomed in reflection off a waterdrop, just to somehow view a business card from the other side.
You could use image stacking to enhance the face. Just find 10 consecutive frames when the subject is not moving and then stacking all of it with median to reduce the noise. Median essentially averages out each pixel position value between shot, thus effectively reducing random noise.
you should have used a temporal solution. Two minute papers has a few videos about it. Some algorithms that can use the data from several frames can get crazy detail back, it's amazing
Speaking of movie tech suggestions, I've always been curious about holograms in movies, is that sort of thing possible in the way it's shown on screen?
Maybe a 2D hologram. That’s been done a bunch or a 3D model projected into 4 screens positioned into a pillar so you could walk around it and view all angles.
As some one who works with "enhancing photos a lot" getting more detail from one is hard if it can be done, you can make it look 10% better and that's usually it but the biggest thing you can enhance are the blacks in the shawdows and pulling out details in darker areas you can go from only seeing black to close to if not the same quality as the rest of the image.
It's amazing the difference just tweaking contrast & shadow can make in terms of perceived clarity of an image? (I basically have to apply that to most of my outdoor shots as I find the auto on both my phone camera & digital SLR are consistently a bit lacking in gamma gradient.) Creating greater detail from interpolating pixels in a low-res pic is a whole other level of algorithmic magic though!
During one Imaging Science workshop while I was getting my first forensics degree we were tasked with enhancing a photo to get a licence plate readable. It was a great deal of messing with sliders and asking my friend who was next to me: "Hey, does it look better like this, or" [moves slider slightly] "like this?" "Uhh... first one maybe?" "Thought so too. But they're both awful." "Yup."
Sounds like those differential lens tests the optometrist always does when one is getting fitted for a new pair of glasses! 😂 "Is it better like this....? Or like this?"
I'm a graphic artist and I can't begin to explain how frustrating it is to have the sales personnel at my job hand me a thumbnail and tell me to "just enhance the image" so it looks good on a 15 foot banner
That happened to me. They asked me to enhance a client’s low res logo. I eventually got it done and they were happy. They asked me how I enhanced it at the end, I told them I just redrew it in illustrator
as someone who has been asked that before (not an illustrator but, for other things IT related) my response was it'll only work if you can give me the project file. you're out a lot of time or are boned otherwise.
I saw a very powerful camera one time that you can take photos from at a great distance and when you zoom into one, itll show every tiny detail on a thing like it were taken from right up close
For your security footage you could've *actually* improved the clarity slightly. You know how temporal anti-aliasing works? You could apply a similar algorithm to subsequent frames of the "criminal's" face
Yes, frame stacking to achieve higher resolution. They should have used Topaz labs' video enhance AI before screen grabbing to Gigapixel for even better results.
Super fun video idea! Here are some VFX/move trope ideas that can be examined/debunked with science: Explosions (like grenades) knocking people into the air, cars exploding by shooting the gas tank, poorly comped muzzle flashes, over the top punch sound effects, sound in space, lens flare abuse.
It almost makes me mad they guessed it right when that undercuts their whole point that the methodology is unreliable, but Corridor Crew is just that good.
First let me just say that the "enhance" tech in most police shows, CSI and similar shows are so obviously fake it's laughable. But it's still used to get a "good" story. Sometimes it works and sometimes it's just cringe worthy. Now some things that wasn't talked about in this episode are color space and temporal enhancement. In a lot of pictures there can be details lost in dark areas or in single color areas that can be made visible by manipulating gamma. It's even used to embed water marks in pictures that are close to impossible to see unless you use a program to look for single step color changes. If you look at a digital photo there can be some details in a dark area that can be made visible this way. But you never get the full color result shown in these shows. Temporal enhancement is pretty interesting. It uses several consecutive pictures from a video to enhance a still image. This works very well with low light photography where you end up with a very grainy picture as there are simply to few photons available. By combining several pictures it's possible to recreate the information missing in one picture by combining it with the information in the other pictures. Similar things can be achieved if you have a video sequence of something small that takes up to few pixels of video resolution as long as the camera or the target is moving at least slowly. It's still no CSI "enhance" but can improve the image quality a bit more than only using a single frame source. And it never introduces any information that isn't available in the source. Now as for untangling distorted images I remember some years back when the police was looking for some person who had uploaded a lot of pictures on the net, but always had obscured the face using a distortion filter in a image manipulation program, I think it was Photoshop, but that's not really important. What turned out to be important was that they figured out what program was used, probably by looking at the metadata in the pictures. Then they were able to apply the same filter but using negative values. The filter was some kind of skew filter applied a swirl effect to a circular area. Reversing the "swirl" they were able to get a picture good enough to identify the person. Now if a hard blur, posterization or simply just a solid color fill had been used this would not have been possible.
Thanks for actually bringing up video/image forensics software. Certainly deconvolution as used in astronomy might be a worthy discussion topic. If you know what you *should* be seeing you can improve quality. Also stuff like drizzle or tap super resolution, highlight recovery etc. Also thank you for noticing that "AI" methods are just informed by their training data and will make stuff up. Steve mould had a video with video forensics software go highlight small subpixel changes, it could read your pulse from a video. Veritasium showed research to film a plastic bag and reconstruct sound. Video has Soo much data that is around - we just need to use it. Not all transformations are inversible. It's not always a bijection. Injective functions exist. Entropy isn't reversible. There is 3 directions a ball could have rolled from to end in one spot. Even if you know the exact state of every subatomic particle(impossible by quantum mechanics) you can predict the future but not calculate the past.
Astro photography is often done by taking video footage of the target, and then using Image Stacking software to combine the information of hundreds, or even thousands, of images. The programs can identify 'noise' from the sensor and eliminate it, while preserving recurring data. There are some incredible photos from backyard astronomers who have used these methods. To try it out I took a few seconds of film of the floorboard in our dining room, in poor light, and then ran them through a free ware astro photography stacking program. The individual frames were pretty ordinary, but the stacked image had a lot more detail in it (that I could compare by looking at the floor :) ). Cheers
Hai guys, excellent topic , next time include digital forensics also like , recovering damaged data , creating whole image from pieces of data , location etc... Thanks 👍
I like to imagine that there is one day out of the year where Jake films an entire years worth of sponsor segments all in one go, one after another. It's like Christmas but it sucks.
![[DRAGON BALL LEGENDS REVEALS ＆ STUFF] LEGENDS FESTIVAL 2024 Special Edition Part 1](http://i.ytimg.com/vi/mV5bGIswTtQ/mqdefault.jpg)
There was a French canadian show where they parody this. From security footage, they zoomed in a client's sweat drop, and now had a 360 view of the store, and zoomed toward the door where the suspect was exiting, then zoomed in his hand, he was holding a credit card, then zoomed in to read the guy's name. But that CSI eye reflection tops it all because it was intended to be serious.
Are you sure you're not thinking of Red Dwarf? (Which was British)
@@jsl151850b No, the Red Dwarf scene was "uncrop, forehead, through a window at other end of street, a doorknob, a faucet, out the window again, a car door, a building's glass door.
Video ID 2aINa6tg3fo
what's the name? I'd love to see that lol
I’m dead bro lolololol
What's the name of the show?
I love the simpons joke where Barts on a computer, says 'zoom in and enhance' and Lisa just shrugs and pushes barts head closer to the screen.
Which episode was that from?
@@lightcycle87 I think it's when Bart was trying to find out what made skinner such a boring square, so it's the episode where Bart finds a prankster who put worms into a swimming pool.
I could Google the episode name, but I'd guess its called 'pranks and all' or something like that.
@@lightcycle87 Bit late but this episode that @iseriver3982 described is called "Pranks and green", so not far off from their guess
I have had better results with Topaz Video Enhance AI, because that does not only use the one pixel, but takes into account how it changes over time. It's a bit like image stacking.
That's what I was thinking. If you have one-second 24 fps video, 1 frame is only 1/24th of your available data. Use it all!
It doesn't help that they were also doing this with a 2D 'mask' instead of an actual 3D face.
I was like screaming use the plate number enhancement tool, it tracks the plane and stacks with opacity bringing the details up
This would be a very cool followup!
yes please that sounds like it has a lot of potential!
The trick is that the cameras just take 8K photos and they only have 1 team member that knows where the Zoom button is.
JUST 8k? XD
@Steven Shorts 🅥 no
8K is only 33 megapixels lol
@@Velgeiss bsd
800k maybe.
There was a case I remember reading about where the investigator had a blurry image of a license plate but he also had precise locations of the lights and the camera (it was at night) and geometry for the car and street. So he recreated the scene on a computer and rendered it many, many times. Each time he used a different license plate number trying to match the pixels in the blurry original with the pixels in the blurry render. He ended up with a few matching license plate numbers (fewer than 10 I think), which was enough to find the car and solve the case. IIRC this was the first time this technique was used.
Seems like you guys are well qualified to do something similar.
this is the same reason why blurring or mosaic-ing text is not enough to obfuscate it since you can just iterate through blurred text and see what matches
Mans was on the grindset
Man the police really go to the nines to catch those speeders.
That sounds like some reverse logic and should not be allowed in court
@@rp9674 I think the principle is that it was able to narrow down a list of potential suspects who could then be investigated and one of the suspects panned out. So the license plate enhancement couldn't be used to convict the person as it was inconclusive, but the evidence from the investigate that followed could.
Now it would be in error to assume that one of that group of suspects must be guilty and that the most plausible suspect was in fact guilty because the pool of created suspects might not include the guilty party.
But it would not be in error to investigate and see if you could find the guilty person among them with more conclusive evidence. Like maybe you eliminate the vast majority who had great alibis, then focus on the remaining few.
A tv show that really took the whole enhance trope to the extreme was one episode of Red Dwarf, where they were zooming and enhaceing through reflections in reflections several times over and each time was just as clear and crisp as the first.
Un-crop!
😂😂😂🚑
Red Dwarf was a comedy, so that just shows how long that's been a joke...
Unless that joke was in the revival series?
Red Dwarf is literally self-aware satire.
Crap like CSI obviously isn't.
You guys should make a CSI video where the characters try to enhance images with the most ridiculous results
I second this!
i would love that!
also you have a lovely hat.
This is a totally awesome idea. I want to see it happen now!
Love this idea!
yess
Y'all should do "Sound Designers React!"
Like: How they made the T-Rex or Godzilla roar; How they make body sounds for creatures that don't even exist; Even the subtle ways they have to add basic clothing, movement, and walking sounds into animated films.
You know those punching sounds in old action movies? WHAT IS THAT?
great! as a recording engineer i would love to see this!
Foley artists. There are a few RUclips vids about what they do.
We need that series considering the fact that I've just came to know that the dolphin sounds use to censor slurs was actually the sound of a kookaburra in faster speed.
Like the Squidward Walking sound
Oh I would love if they did that. I remember there was a tutorial they did once on punching sound effects, but I'd love to see a bunch of audio stuff broken down.
I love how they always have the tech furiously click-clacking away at the keyboard for these enhance sequences, as if every calculation by the computer required a key press
Because simply clicking one '+' to zoom in would make a boring show :P
@@Shade01982 I would argue that using this tired old cliche scene (wait what’s that? Zoom and enhance!) to advance the plot already makes for a boring show
@@matthewwright2524 True...
That's how brilliant the technician is. He's programming ai in real time
The most impressive thing isn't the enhancement, but the fact the investigator needs to tell the technician to enhance. I mean, if the algorithm always work why even present the blurred image? The image/video app should just enhance by default when zooming in
What if the reason they don't enhance immediately is because the computing power is so much, every time you press enhance it costs thousand of dollars lol
I can imagine if such a thing is even possible it would require using some super computers to completely recreate things
The enhance feature only works in the presence of a trained technician.
A goat must be blood sacrificed every-time they enhance, so they try not to do it too much (blood sacrifice is the only logical answer for there enhance "algorithm")
@@durza7173 a trained technician guided by an authority figure, of course
They are actors, they love to have lines to speak however silly they might be.
The technology in CSI is simply advanced alien technology that humanity hasn’t even come close to replicating yet.
Maybe we already have and they're just holding back on the public?
@@ROBOHOLIC1 if so, we wouldn't have so much unsolved murderers and criminals.
Fourth
.
So you’re just ignoring when they actually enhance?
When they were looking at license plates, that’s the technology needed. It is just based on numbers because we don’t have the processing power to be able to enhance a hat.
You did a good job demonstrating how predictive AI should not be court admissible. It is the equivalent to an artist's rendition from a blurry image.
Several years ago in the UK a driver was charged for speeding caught by a camera. They used computer algorithm to read the plate and that was submitted as evidence. The defence asked for a copy of the original picture where the plate was completely unreadable. The defence argued the evidence was generated by a computer algorithm and who can say how accurate it was.
The defence won.
The only way you could use an enhanced image like that would be to find a vehicle (or person) and then find additional REAL evidence to use in court.
@@angelbear_og indeed and with speeding, usually the photo is the only evidence.
was it an AI algorithm adding detail, or simple math translations?
"But . . . they did it on CSI!"
I think prosecutors are hoping for jurors who don't know a thing about image enhancement. I'm glad the defense saw through the state's BS.
Ah, the idiocy of archaic courts.
I used to work in this area, so here's some insight: The rule of thumb is that any image clarification needs to be performed in a way that can be reproduced by a third party using the same tools. and by "reproduced" I mean each step in the process needs to produce identical md5 or sha hash values. Some people may be using AI, but I wouldn't touch that stuff.
Is this so that it is verifiable in court? Really brings up an interesting topic in the near future as more and more black box algorithms are used to run things.
@@philipfahy9658 Yeah, basically. Black boxes can be a nightmare. As a technician, I documented my process as best I could. If opposing council has an issue with my results from a black box, they may just have to bring it up with whoever created it.
@@peterehowell Yeah and it gets worrying when you realize how poorly most of our politicians understand the internet. I can only imagine what level of familiarity or lack thereof different public institutions have with this technology.
If Kyle Rittenhouse trial is representative of your average trial then the CSI one is closer to reality where the prosecution just phtoshops shit in to fake evidence.
@@XAJUSS
And yet it was still allowed in.
Later we had the HD recording leaked that showed that the "reconstruction" is bullshit, they cropped, reduced the resolution and sharpened it up to make sure you cant tell whats going on.
But yeah, the prosecution was awful, and showed how corrupt the legal system is. My favorite part was when the prosecutor pointed a gun at the jury with finger on the trigger while talking about gun safety, peak comedy ...
Enhance! Great video. I’m actually blown away by how much you’re all able to do. I remember using genuine fractals ten years ago, it’s come a long ways with AI.
It really has! the licence plate demo straight up blew my mind with how the AI took basically nothing, just incomprehensible smudges, and figured out what it said.
Out of darts comments : 7 likes
How
@@justleo5997 The license plate demo didn't use AI (5:18 - 5:35), Niko explicitly points out how that was just mathematical algorithms. If the application did use AI there would be no way to guarantee what was real information and what was generated information. Given a solid enough model and the correct type of AI algo it could be sort of possible to do this with a high enough confidence to have this be admissible evidence, but I wouldn't trust generative AI to be able to do that, maybe a neural net that is well tuned but in general nothing is better than just solid math.
Not everything is AI, software and math have been doing these things incredibly well for a long time....
I guess the Mark Ruffalo thing was impressive as long as you discount that the team was primed with information about the image, and then refined the generation with the additional information. This would mean that if used in a real court, not only would a eye-witness testimony (which at this point DNA evidence and project innocence have proven eye witness testimony is nearly never reliable) be a part of this, but the results would be dependent on the type of model, the input parameters, and the model training to perform the enhancement of the "Evidence". AI is a great tool for some use cases, but its not a panacea, and its certainly not something that should be trusted with matters of fact.
The mark ruffalo thing could benefit from some temporal solver. You know, combining data from 30 frames instead of one. Is there any AI-solvers that could handle that?
I was thinking the same thing. That would provide a lot more "detail" (or at least "data") to work with.
@@justaviking yeah! Astronomy programs can stack frames like that, so that’s an obvious way to be more ”state of art” if implemented well
Wouldn't that just reinforce the blocky, pixelated features? Give the AI 30 frames of black holes for eyes and over exposed face and it will just assume that's what it's supposed to look like and not correct it.
@@radioethiopiate It's how jpg handles brightness, combining some greyscale preset images (quantization table, technically) to generate a more detailed composition, the only issue for this usage case is nailing the alignment, but I've manually done this in the past and athought still very soft, figures can become way more recognizable.
I saw some tiktok about something like this where a guy took a video of a spinning record and then took the average of all the frames in photoshop or smth and it was suddenly really readable
As a graphics artist at a TV station, I want to say THANK YOU for doing this video! Cops bring video to us all the time, and my producers are always saying, "just zoom in and enhance this"
One could argue that the CSi "Enhance" feature exists because the team has huge arrays of sensors lodged in every crevice of everywhere in the world, and the Enhance feature simply loads in data from those sensors and displays it in a way that is consistent with the viewing angle of the footage, or in some cases, completely inconsistent with the viewing angle for a better view.
so, google street view
Exactly it feels plausible in a "big brother" surveillance state
@@SOLIDSNAKE. No, it really doesn't. It kinda would make sense in futuristic scenes like Star Trek or something to that effect, but a scene that takes place in current day, no. Even assuming all those devices are even aimed at what you want to see, the majority of those are not going to be of any better quality than what you already have. The resolution is not going to get any better. You know just have two blurry images instead of one.
not sure if that'd explain why the image of "Nuevitas" was flipped vertically
clearly, they thought - well, words appear backwards in a mirror, so we'll flip the image horizontally,
then rotate it to where we need. That'll be fine, right?
The only "Zoom and Enhance" that I ever let slide was in the Loki show. The pocket computers of the TVA had a pretty low resolution display, but had very high resolution surveillance footage. In that instance it made sense that you could scale a part of the image up without losing detail.
I was willing to allow _Blade Runner_ as well (which appears in here).
Fancy super future technology where replicants exist? Sure, why not.
@@FangsFirst That’s a very fair point!
There was a TV show called I think Continuum, where the lady was from 200 years in the future and her nanosuit thing accessed the security cameras of every business downtown and created a 3d reconstruction of a vehicle moving from their footage and in cases like that, yeah just go with it.
For the TVA, they can do whatever they want, they have Time Magic
some pornbot stole your comment, so i subscribed to you
The original comment
I'd like to imagine the CSI crew just has access to an advanced neural net with centuries of training, just vibing underneath their building. Catching criminals is the only way to satisfy its bloodlust
Maybe CSI is in the same universe as Person of Interest.
A little bit like SIBYL in Psycho-Pass? xD
@@taylankammer
Yeah...
Honestly, that's a Netflix anime adaptation I'd actually watch. CSI but it's in the same universe as Psycho Pass
Even such net wouldn't be able to recreate whole picture from one pixel.
I watched Enemy of the State a couple weeks ago and that scene caught my eye, something they did actually added to the legitimacy, which was they called out that they weren't able to look "inside" the bag, and what they found "could just be a shadow" or "the bag shifted". I liked that they tried to ground it in reality somewhat by giving it limitations.
The car numberplate goes way beyond sharpening, it's done with deconvolution. It's been in the Wolfram Language (Mathematica) for a long time. It's not as easy as you'll see it described as because slight errors in the lens shape, or rounding in the calculation etc… blow up into a huge mess really easily.
They didn't call it sharpening? They called it deblurring. And they were referring to motion blur.
@@plzletmebefrankthey called it both sharpening and motion blur, you can use deconvolution to remove both motion and disc blur. The ripples around the edge are an indicator.
@@alan2here Fair. They are probably just using terms people like me can understand though.
It's definitely more complicated than they make it out to be, but the software hides a lot of that complexity within itself.
That's something I hadn't thought about! If you're talking about trying to flatten an angled image (as per the second license plate example in this vid), does the type of lens used in the original camera make a difference? Like, do you need to know what kind it is so you can calculate how much it may be distorting the image? Or can the software just work out all of that?
A suggestion for another vfx relate movie trope:
How satellites are able to see someone suspiciously drinking coffee from space.
Well, google maps and other civilian services provide satellite photos that are clear enough you can sometimes spot individuals on them. Stuff not for public access like government and military sats should be of even higher fidelity, so it's not particularly impossible.
while not satellites..... there are actual military drones these days that can do that (UAV miltary drones that fly around cloud height, some use up to 500 separate sensors similar to a smartphone and than record and send that feed to a technician in real time). the key difference with high tech miltray drones is they are equivalent to 2500 combined megapixles.......for perspective 4k video is 8 megapixels
@@scottmantooth8785 maybe because it doesn't exist? Even it existed it should've died from old age
@@DarthBiomech Google map doesn't use satellite data for the most detailed zoom levels. They use photos from planes overflying areas, which is also why the maximum detail level can vary from area to area, since it depends on whether they have airplane photos from each area.
@@Carewolf Commercial satellite resolution used to be limited to 50cm/pixel by U.S. law until 2014 - then they allowed up to 30cm/px native resolution, however some companies started offering even better 'synthetic' resolution - basically A.I. upscaled. Best I've seen is 15cm/px by Maxar.
Spy satellites are said to have resolution up to 5cm/px which is still not enough to read license plates or recognize faces.
It's much easier, cheaper and far less regulated to achieve higher resolutions using planes or drones.
We saw this unfold with the Kenosha self defense case. Prosecution was trying to submit a video as evidence that had been enhanced by software to make an argument. Luckily that argument wasn’t even clear with the enhancement, but it could be potentially dangerous using enhanced video as admissible evidence in court.
The "enhance" effect is one of my BIGGEST pet peeves in movies/TV shows and it makes me so happy you guys finally debunked it.
In order to zoom in on tiny objects on security footage like they show in the movies, there camera resolution would have to be extremely high to pick out those details.
Mine is the lack of marksmanship by law enforcement officers that have supposedly spent years training with firearms missing shots that I know I could make cold even with my limited ammo supply and range time.
As a programmer, the 1000 word-per-minute typing in a lime green command console to hack into the FBI in 10 seconds tips it for me.
@@TheMightyHams sounds like you're just not good enough at your job
@@phosphatepod Very true, I'll download some more RAM, switch my editor to green highlights, and deploy some swanky UI loading bars with flashing red warning labels. FBI will stand no chance.
The Klingon “Enhance” blew up the Enterprise D in “Star Trek: Generations” and I was like “If Klingons built computers; they’d be like Ataris”
And the pieces of the enterprise fell back down to earth in the form of “Chocolate rain”
You may have stayed dry but the others felt the pain
They literally build interstellar spaceships. I think they can go a bit better than Atari level computers.
RLM recently brought this up if I recall correctly. It is fun to imagine how Klingon culture deals with scientists and nerds.
They've got starships and all, but...how?
@@FangsFirst they probably have a planet in the empire that is tech support with a conquered people on it. "Welcome to Zainar Technologies how can I help", "Tell me how to make the view screen work WORM", "So I understand you have a problem with the view screen, is this a Brell or a K'tinga?", "THIS IS A VORCHA P'TAK", "Oooh sorry, the Vorcha uses an M117 Digiplex Viewscreen, you want a different support line. Please demonstrate your honor by holding." *music starts playing*. Klingon Captain: "I'm not dishonorable, I will wait through your music. *taps foot*"
To be fair, it wasn't the Klingons that pulled it off. Soran was the one smart enough to bug Geordi's visor. The Klingons just had to watch the footage.
Appreciate the emphasis that AI upscaling invents details from nothing, it doesn't reconstruct real ones. Too often people have the misconception that the upscaled image is supposed to be accurate in any way, shape or form. Imagine the damage if a tech-illiterate judge accepted something like this as evidence
It apparently happened in the Kyle Ryttenhouse trial
I mean, your brain does the same thing for what you're actually looking at in real life, so this is really just semantics. The AI definitely does *not* invent details from nothing. It reconstructs (yes, reconstructs) detail from context. The real question is how accurate the AI is at reconstructing this detail. That VFX artists don't fully understand upscaling AI is not surprising.
@@skipfred the difference between inventing and reconstructing (the way you're using the word) in this context is just semantics. You can't use AI to prove or even indicate what was there irl
@@soundninja99 The way I'm using them is exactly how OP used them, so yes it is semantics, that was my point, but it's not a semantic argument because we're agreeing on OP's original usage. Saying that you can't use AI to prove or indicate what was there is like saying that you can't use blood splatters to prove or indicate that a stabbing occurred. Yes, you very obviously can because of *context*. The question is whether or not the investigator (or the AI) is capable and reliable, not whether it's technically feasible.
@@skipfred fine, it can be used to indicate what was there to the same degree as some investiigator staring at it and guessing what was there
Thoughts while watching the video: The potion seller inverse distortion reminded me of a case where someone had censored their face using the "swirl" filter in some software and it was easily reversed by applying the swirl in the opposite direction.
Second: You can do some limited superresolution if you consider the movement of the camera or the subject in front of the camera because if the subject moves by a sub-pixel amount, it can theoretically give you more information than a perfectly static image. I remember using filters like that like over a decade ago. The results aren't mindblowing, but they're pretty good. I think the software was Anti Lamenessing Engine, after a short websearch.
Third: I remember seeing people dunking on some magic upscaling algorithm after using a low resolution photo of Barack Obama as the input and the algorithm outputted some white guy.
Yeah, I'm pretty sure you could get a much better image of Mark Ruffalo if you took the raw footage, then motion tracked the face and stabilised it accounting for distortion, then superimposed all of the frames and averaged them out. If you think about it it's the same idea as the motion blur number plate example Niko was showing, except spread out frame by frame in the video. If you took all the frames and composed them together you can get more data out of it.
I was thinking exactly the same thing. I was half expecting the swirl face case to come up in the video. It was actually a nice case of karma. The guy in question was a prolific producer or trader of child pornography and his online avatar image was his face made unrecognisable - he assumed - by that swirl algorithm. The police 'unswirled' the image and then found and arrested him. His name is Christopher Paul Neil and he even has a wikipedia page if you're interested in the story.
@@bucklogos That's the type of thing intelligence agencies were doing years ago with specialized software that could provide greater resolution than what the camera alone could provide. One guy who had done work on satellites, though limited in what he could say, pointed out that while a camera in orbit taking a picture had a certain minimal resolution, taking _two_ images in rapid succession could, hypothetically and with the right processing, be functionally the equivalent of taking a single image with the optics of the camera being the size of how far the satellite had traveled between images, thus vastly greater resolution.
@@bucklogos I agree, stacking multiple frames from video should give additional information (after correction for changes in angle). Astronomy software such as Registax does something similar
The
Would love for you to break down scenes where characters 'hack into the mainframe', or take over systems. Usually its verbal garbage with visual garbage so would be awesome to see them break it down!
A gigabite of RAM should do the trick
Matrix Reloaded is one of the few movies to depict hacking accurately.
NCIS has a pretty good one where 2 characters to counter-hack the mainframe by pressing random keys on the same keyboard. Like they each take one side of the keyboard, and then another character walks in and unplugs the computer. It goes from complete fantasy to the easiest and most realistic solution in a single scene and its fucking hilarious.
i bet this would be a great tool for enhancing older aerial photography. I work in archaeology and really want to try this for locating old buildings
Everybody needs a wren in their life. He is always so interested and energetic about everything. I love it.
you can be your own Wren
@@holliswilliams8426 not like wren. Wren is a special breed 😇
For ten minutes, then you just want to lock him in the fridge for some peace and quiet.
?acting
The human Labrador Retriever
I know this isn’t a “react” video, but I do like how Wren is genuinely reacting to CGI stuff. That being said, what doesn’t Wren react to? 😂 love you guys
@Don't read profile photo I read your mind, you sick puppy!!!!
Unfortunately the OneWheel scandal...
The even more ridiculous tech CSI had was when they would magically have recreations of the crime scenes in 3D with animation to go with it. I think with Lidar scans you could get close to mapping the room but all the animation stuff that happened seemingly automatically was ridiculous. As an animator myself I always laughed at that stuff.
Would you be able to do photogrammetry with multiple security cameras? I know that there's some pretty good software for it nowadays
@@xWood4000 Yes, in fact you can. You get a blurry mess as an animation but you can. Look for radiohead-house of cards for an example.
Closest tech to this now is NeRF i guess. They generate 3d model and textures from one photo
We laugh at it, until we make it real :)
@@xWood4000 actually in batman arkham origins they do make a holoprojeciton of the crime scene and you have to connect the evidence to fill in missing data, is pretty cool
One of my favorite "zoom and enhance" scenes was from Gotham. Gordon was looking at a tiny TV wired to a crappy security camera. The suspect was wearing his work shirt, but the logo was blurry. So Gordon grabs his partner's glasses, and holds them up like a magnifying glass. Bloom! Crystal clear!
You serious?
lmfao
@@kappadarwin9476 yeah, it was great! That's why Gordon is the best! He's a magician
Gotta get my hands on those new pixel multiplying glasses, in stores now!
I remember seeing papers or technical demos on "enhancing" / de-blurring video frames by looking at the frames as a set, rather than individually. That can produce even more detail. Essentially, the lens and camera is constant so small movements result in features being pixelated in a variety of ways, which in turn provides sub-pixel level information.
More detail could definitely be drawn from the footage this way if we had the ideal algorithm.
The Google Pixel’s camera does this to enhance its clarity. It will take a quick set of photos and combines the info captured in them to allow an incredibly impressive improvement to its digital zoom. Our eyes actually do the same thing. If look closely a focused eye doesn’t stay still, it has these microjitters essentially using a “set of frames” from slightly different offsets to get as much information as possible from it sensor cells.
I believe this is how someone was doing some...DS9 remasters, I think it was? They looked pretty dang good.
Amateur Astronomic Photographers use this method to get ultra sharp photos of the Moon and planets.
Dozens of photos superimposed.
"Is there police departments using this kind of tech" yes, a fair amount I reckon and fairly advanced too. Some students from the computer graphics major at my uni worked on a project requested by the police to generate 3D heads from mugshots to facilitate identification on videosurveillance footage. Basically they would position the 3D head at the same angle that the camera is watching the real person for comparison and that would help them decide wether or not it is the person in the video they are looking for.
Anybody that used VR goggles to try to read small text might understand this: If you look at the text with no head movement it's a blurry, unreadable mess. If you move your head in a circle, the letters become visible BECAUSE you're actually moving the physical pixels in the headset around "fixed" digital image where what they're averaging is changing letting your brain have more info to interpret.
That's also why a good trick when watching videos in VR headsets is to set the video to somewhat larger size than your field of view (FOV), such as have the video's vertical size match your vertical FOV. This makes you want to look left and right more to see the video giving more virtual information than the raw pixel count would suggest.
That first paragraph described what temporal upscaling is essentially. Just done within your brain. It's the most high level straight upscaling method we have, reconstruction is the hard part to tune for a clear image though.
A real world analog might be riding in a car past a slat fence and being able to see what's going on on the other side almost as I'd there was nothing in the way. But if you stop suddenly you can only see through the slits. The human brain is great at averaging information over time I to a synthetic image.
I saw a really interesting technique for enhancing a while ago - the issue, as they say, is that you can't enhance something where the data just doesn't exist, where it's just a couple of pixels.
But the technique I saw used video and processed multiple frames at the same time. The area of interest may only be a couple of pixels wide, but each frame captures those pixels slightly differently. By combining the data from many frames, it's possible to find artefacts far smaller than the static pixels that contain them.
Yup, the whole time I was thinking of this. I don't remember where I saw it but it is definitely possible to accurately calculate sub-pixel details by using the data from multiple frames, if the subject shifts even a fraction of a pixel all the pixels will shift slightly in brightness so the more frames the more detail can be extracted
@@LennertK This is how temporal anti aliasing also works (TAA).
I actually tried this on the footage in the video this afternoon. Unfortunately (not helped by youtube compression) the results after stabilizing and stacking weren't very good. But perhaps with access to the original raw footage it would be better.
I was thinking this same thing the whole time. If you have, say, 3 seconds where a face is visible (and moving) then you have potentially 60-180 different versions of that object. Then instead of training the AI with 'real world' data, you select the clearest frame and train the AI with the other 179 frames. I'm sure it's much more complicated than that but you're no longer using invented data, so more admissable perhaps.
Here's an idea: recreate "zooming into someone's eyeball to reveal the suspect in the reflection", but using 3d software and 3d scans of your heads. Position the head models so that when the 3d camera uses some ridiculously high focal length, you can actually see the reflection off the eyeball.
You can do that with good DSLR, no need for 3d modeling.
Unless I missed it in the video, I only heard one type of upscaling mentioned. There is upscaling that looks at only a single frame, but there is another type of upscaling that looks at the frame(s) before and after. By looking at additional frames, this provides significantly more information for upscaling.
That was the case for me too actually. We have decided to digitize and enhance old family photos, and when scrolling through and zooming in, I have discovered a bearded person face in the air between my family members. Originally it was blurred (out of focus) background made of shadows of a tree leaves. I was scared to shit, lol.
That's hilarious
One technique they didn't talk about was image stacking, where you take multiple pictures and average the pixels values; this dramatically lowers grain, and when paired with upscaling this can get some pretty good results. You can't do this with just 1 picture but you can with video because each frame is like a single image which can go into collecting the data to create a more detailed final image.
I always heard about the "Enhance!" bit being ridiculous, but I never realized just HOW ridiculous it actually was til you showed these examples. It's worse than the parodies of it!
My favourite enhance scene is in season 1 of Barry. The detective asks the tech guys to enhance footage taken by a lipstick camera, they give each a look, and in the next scene as she's questioning the acting students she holds up the blurriest picture, everyone squints and start debating what it's supposed to look like.
You guys should've looked at an early classic "Enhance" scene from the original Blade Runner (1982). As I recall, Decker scans a polaroid, zooms in on a tiny, out-of-focus bathroom mirror in the background, fixes the focus, adds detail, and then rotates to see a label on a medicine bottle that WOULD be reflected in the mirror if the photo were taken from a different angle.
I would argue Blade Runner gets a pass because it's not a Polaroid, it's supposed to be a science-fiction-y holographic photograph with unspecified 3D-type features built into it, which the machine he inserts it into can manipulate. In the movie, he adjusts the perspective of the camera such that he sees around the frame of a doorway to reveal what's behind it, and I suppose in any case that's pretty implausible. But at least they make it clear this is all some technology that's foreign to us, something in the future that doesn't currently exist. CSI doesn't have this excuse.
@@mikekohary1075 That's actually pretty good.
@@mikekohary1075 Seems like present day lightfield photo tech could be a very early precursor to the kind of photography they have in Blade Runner
You guys should make the most ridiculous unrealistic "enhance" scene you can as a sketch
The victim was standing in front of the mirror. We can't see the perp, but we can partially see the rest of the room in the image. What's that? Enhance the victim's eye in the mirror. It's a fishbowl. Apply inverse focal length and water light refraction. Zoom in on the fish's eye. It's our guy! It was the butler all along!
@@mac1991seth lol, exactly 😂
There was a scene like that in an episode of Red Dwarf.
@@mrcydonia "Uncrop!"
Well they kind of already did it was in blade runner the original one
Another thing you could do with a video is use a super resolution algorithm, that utilizes more than one frame. That way really get an image with higher resolution than the input video.
That's exactly what they did for the license plate enhancement shown in this video.
@@blindleader42 As I understood it they just watched an example video of the software, but didn't use it themselves. Using multi frame super resolution on the CCTV video should yield much clearer results, if you get enough good frames.
There was an episode of CSI: Cyber where they tracked a guy down by finding his phone. Apparently when he touched the screen, it left a "digital fingerprint" made out of code that they could find, reconstruct, and identify the person by. Like the phone screen can randomly copy fingerprints onto the memory by any touch. Then the technique was never used again in the show. Couldn't keep watching after that.
It sounds like a rather extreme extrapolation of keystroke recognition, the existing software that can capture and individualise people's typing patterns?? 😆
My phone screen does have a fingerprint reader built into it, so this is not entirely ridiculous. A lot of phones do.
It's worth mentioning that one of the first instances of "zoom and enhance" was from Blade Runner (1982), where it made sense as far-off futuristic sci-fi technology (even though Blade Runner was set in the then far future of 2019).
Thank you. Now l know l am not the only one.
they are just extrapolating existing film photography, like for example they are already familiar with micro-film where they could zoom in and see a document otherwise unintelligible to an unaided eye.
"Enhance" and "Compensate", two of the most magical words in fiction.
Coincidentally, also the most common words on gas station boner supplement packaging
this was an awesome video that was actually really interesting. it also felt like you guys said, 'hey we need to do a video. lets do that simple idea we had' and sat in some chairs and killed it.
Y'all should do a review of Robin Williams in Law and Order SVU. He brings up how the AI is guessing details in an enhanced image and gets it removed from court. I think it's in season 9.
The "enhance" camera effect in CSi has better quality than the security cameras at the bank.
I’d have loved to see multi-shot techniques applied to the problem. Use a couple of consecutive frames where the subject is still and light-path jitter will allow you to interpolate more pixels.
I feel like the Ai face generation upscaling thing they were using would have worked a whole lot better if the mark Ruffalo mask was in colour. if it looks for recognisable pixel clusters surely the colour of the pixel is also important
One thing they could have done with the security camera footage, which would be more complicated but seems like just the thing the Clonix software might also be capable of, is combine several frames to create a slightly more detailed version of the face.
Essentially, this is comparable to what they did to get those images from the black holes with the Event Horizon project.
i thought the same
Please Do Muppets From Space! Would love to see you break down those effects. Especially the Noah scene, the Gonzo getting a message from Space scene, the invisibility spray scene, and the spaceship scene at the end! Thanks!
The "Enhance" from Blade Runner maybe COULD work. The reason is that its future-past cyberpunky aesthetic seems to still use some analogue technology, which could include film cameras. Because digital cameras use a limited pixel resolution, "enhancing" requires a lot of extrapolating gaps in information. However, when a picture is captured on film, it is captured with a chemical process which goes down to a molecular level. So as long as the lighting is sufficient, etc, it is actually possible to see extremely tiny details through close analysis.
doesn't he get the image of the woman from extrapolating a mirror I.age of parts if the room you cannot see? :)
Analog photography is limited by film grain
Film grain is small but never on molecular lvl, hence all the ISO standards. Same problem, we cannot control particle size to that degree be it pnp-based sensors or silver precipitate particles. I think record with (gold) particles atm is nano scale or something
Nerd
@@noahduncan5524 good thing nerd is a compliment now
17:37 the way Wren said that is go great😂
16:19 I legit just did the voice right before that Team America clip came up. Good job editor
The other thing to note about AI upscaling is that it does not recover lost information not captured in the original image, only gives a best guess of the missing detail based on its image training.
Real talk, the first time I watched Enemy of the State, I actually assumed it was a "mistaken identity" movie because the agency had _framed_ Smith with that obviously fabricated footage. Even tween me was like, "Wait you can't see the reverse of this, they're obviously making it up," so I was waiting the whole runtime for the reveal of why they _really_ wanted to catch him. It was very confusing.
That’s hilarious.
How many movies were ruined by one too-illogical VFX shot? We’ll never know…
The Red Dwarf take on the old "enhance" was great. it's called "Red Dwarf Blade Runner Esper Sequence" on youtube.
I mean, the zoom and enhance is such a movie trope that even movies like "Blade Runner" way back in the 80's used it. In a very cool way btw.
3:35 Look Wren, Black Magic is a great camera but I don't think even it could capture that detail.
Possibly my favorite video you guys have done. This has long been my biggest pet peeve with CSI in particular, but I remember shouting angrily when they pulled it in X-Files.
I love the way that Red Dwarf did it: scan a photograph, uncrop the image, zoom in on the detail that was cropped out of the original image, zoom in on a piece of jewelry, enhance the reflection to get an image of the street, zoom in on a lamp post, look for piss droplets from a dog, zoom in, enhance the reflection off the piss, zoom in on a window, enhance reflection again, flip image horizontally, zoom and rotate, boom: the suspect's business address, which was in the phone book the whole time.
I have watched RD many times, but don't remember this. Which episode was it in?
@@lotuselansteve Red Dwarf: Back to Earth Episode 2
"You see that reflection?"
"Yeah, sure do."
"Zoom and enhance."
*Swivels chair*
"Sir, i can do exactly one of those things and it's not the useful one."
the "enhance" meme began with Bladerunner... and that was on a frickin' POLAROID PHOTO!!! Cheers!
0:41 To be fair, the Enterprise probably has sensors that register at the Planck scale.
One thing to note is that the AI enhancement will actually have an advantage on the celebrity face as well, because as public figures celebrities are almost certainly in every face database used in AI research and development.
For astro photography I"ve used an program which can compose multiple lowres images to a higher resolution one and made more detial visible . Maybe something like that can be also used.
What's funny about the AI upscaling example, is that you could probably force it to give you "a reflection of the murderer" from a few pixels in someone's eyeball. You would get an image of a person from your training data set (or an amalgamation of people from it), not who you're looking for 😂
In college, in my Image Processing class, the Professor also worked with the forensics team of our local police. I'm so glad that Clonix made an appearance hahah
I don't know about US law, but where I'm from, Niko is 100% correct about the usage of these tools in court. EVERY SINGLE TRANSFORMATION to the image is recorded and the final result can only be used as evidence if there are absolutely no info added. If the transformations oblige to this restrictions, you can apply the reverse of all transformations to the final image and have the exact same image as the initial one, before any processing. Because of this, no AI assisted enhancement are allowed to be used if you want to find evidence. No matter how small or imperceptible its effect is. So yeah, those image processing thingies that Clonix did are all bijecting functions (reversible math transformations). Similarly to Math's Topology.
Thinking out of the box here, but if a record was kept of a million monkeys doing a million single-step transforms (to borrow an ancient analogy,) and one of them manipulated an image into an acceptable state to use as evidence in a prosecution - one that could be reversed step-by-step into the original image - even though that resultant image had nothing to do with the actual perpetrator or crime (let's say manipulating the off-angle license plate of a waiting getaway vehicle into the license plate of the bank manager), how is this acceptable? Recording steps shouldn't be the defining point for acceptability; a minimal number of steps to get from unusable to 'believable' acceptable evidence should be of more concern.
@@mamapapaxp When I use the word "transformation", I'm strictly using mathematical terminology. Manipulation is different to transformation. You cannot transform an image into something that it isn't. Every transformation is applied to the entire image or it will cease to be homomorphic/affine (I don't recall the correct terminology, as there has been almost 8 years since I took this class and 6 since I did anything this Math heavy). So, transforming a "kinda 1" into a "7" will distort all other numbers, so it's a useless transformation. If you correctly apply them, math guarantees that if the end result is the license plate of the bank manager, then it WAS the bank manager's car's license plate. Now it's a matter of figuring out if the car was stolen, had the plate cloned, or if the manager was acting with them.
Also, what allows a transformed image to be used in court as evidence is the fact that it doesn't add info to the original. Recording every single step made and each step being a transformation function are just to prove that what you did oblige to the restriction. The evidence can still be contested, in which case, the transformations will be evaluated to see if those are valid, or if there could be any ambiguity. As discussed on my previous paragraph, there are a lot of restrictions to which functions are allowed. They need to be transformation functions (using strictly math terminology here again), so bijection is just one of it's characteristics. A look at Wikipedia's article about Transformation Functions and their subtypes can help with that.
Wren: *does a single thing in after effects*
Wren: "I think I've pushed after effects to it's limits"
that's actullay very accurate
I actually think that the fact that it's video gives you way more data. You could definitely write software to pull extra details out of the video. In astrophotography you do something similar with lucky imaging to take pictures of planets. You basically take high framerate video and stack it. I'm convinced you could do it with facial recognition in low res video
I would like to add that there is one more layer that could theoretically be used to enhance security footage: other frames. In theory, you can mathematically determine, using how the pixels brighten and darken, more details about what's happening "between" pixels. Idk if there are any tools out there using that method currently.
temporal deconvolution is difficult but possible
This would then work something similar to the way DLSS image reconstruction works... cool.
Ive always thought this would be the future, successive frame enhancement via AI, motion tracking pixels, deconvolution, feature detection
The comment i was looking for before wirting it myself :)
This principle is used in reverse for video compression,
The use interframes and see how pixel change between frames and only store the changes
Which come to thing of it may interfere with attempts to go backwards to get more information, as pretty much all (compressed) video uses this kind of compression
The end conversation reminded me of a tv show. There was some old CSI SVU episode staring Robin Williams where Williams argued that the enhanced image guesses what the zoomed image looked like and therefore can’t be trusted to be accurate.
Red Dwarf, the newer 2009 mini-series, had a “Super Enhance” that not only extends the footage beyond the borders, enhance a reflection, enhance the zoomed in reflection off a waterdrop, just to somehow view a business card from the other side.
You mean the three part mini series from 2009?
Kryten also then pointed out that they could have just looked him up in the phone book
@@FFKonoko yep
You could use image stacking to enhance the face. Just find 10 consecutive frames when the subject is not moving and then stacking all of it with median to reduce the noise. Median essentially averages out each pixel position value between shot, thus effectively reducing random noise.
you should have used a temporal solution. Two minute papers has a few videos about it. Some algorithms that can use the data from several frames can get crazy detail back, it's amazing
Speaking of movie tech suggestions, I've always been curious about holograms in movies, is that sort of thing possible in the way it's shown on screen?
Maybe a 2D hologram. That’s been done a bunch or a 3D model projected into 4 screens positioned into a pillar so you could walk around it and view all angles.
As some one who works with "enhancing photos a lot" getting more detail from one is hard if it can be done, you can make it look 10% better and that's usually it but the biggest thing you can enhance are the blacks in the shawdows and pulling out details in darker areas you can go from only seeing black to close to if not the same quality as the rest of the image.
It's amazing the difference just tweaking contrast & shadow can make in terms of perceived clarity of an image? (I basically have to apply that to most of my outdoor shots as I find the auto on both my phone camera & digital SLR are consistently a bit lacking in gamma gradient.) Creating greater detail from interpolating pixels in a low-res pic is a whole other level of algorithmic magic though!
During one Imaging Science workshop while I was getting my first forensics degree we were tasked with enhancing a photo to get a licence plate readable. It was a great deal of messing with sliders and asking my friend who was next to me: "Hey, does it look better like this, or" [moves slider slightly] "like this?"
"Uhh... first one maybe?"
"Thought so too. But they're both awful."
"Yup."
Sounds like those differential lens tests the optometrist always does when one is getting fitted for a new pair of glasses! 😂 "Is it better like this....? Or like this?"
"There's high resolution hidden in that pixely blur!"
I'm a graphic artist and I can't begin to explain how frustrating it is to have the sales personnel at my job hand me a thumbnail and tell me to "just enhance the image" so it looks good on a 15 foot banner
That happened to me. They asked me to enhance a client’s low res logo. I eventually got it done and they were happy. They asked me how I enhanced it at the end, I told them I just redrew it in illustrator
Been there too. 🤓
as someone who has been asked that before (not an illustrator but, for other things IT related) my response was it'll only work if you can give me the project file. you're out a lot of time or are boned otherwise.
As working in the same kind of profession, I feel your pain... A LOT!
Gather around, children - I'll tell you the story of the one time I had a client who actually still possessed the vector file for their company logo
I saw a very powerful camera one time that you can take photos from at a great distance and when you zoom into one, itll show every tiny detail on a thing like it were taken from right up close
For your security footage you could've *actually* improved the clarity slightly. You know how temporal anti-aliasing works? You could apply a similar algorithm to subsequent frames of the "criminal's" face
Did you actually watch the video?
@@zaphodb9213 Did you actually fall on your head as a child?
@@insu_na bro, what a cool comeback, so epic
Yes, frame stacking to achieve higher resolution. They should have used Topaz labs' video enhance AI before screen grabbing to Gigapixel for even better results.
@@thejadedjester4935 I made a legitimate point in my comment. Just because you 2 seem to have your heads up your asses doesn't change that.
1:51 "It's informed by knowledge"
Hmm yes, the floor is made of floor
Super fun video idea! Here are some VFX/move trope ideas that can be examined/debunked with science: Explosions (like grenades) knocking people into the air, cars exploding by shooting the gas tank, poorly comped muzzle flashes, over the top punch sound effects, sound in space, lens flare abuse.
LOL such a goated video idea, this was so entertaining but that potion seller clip was not expected ahahaha
.
you!
welp, time to watch it again, I guess!
Remember when Binger tried to pass a fake image off in court. We remember. The worst part was that he had the close up image and hid it from trial.
Good to see some content from that case here. Truly one of the trials of the century.
Poor little Binger.
I was thinking of this too. Excellent argument from the defense how they explained that the "enhanced" image contained literally made-up data
I am SO glad you guys made a video about this, it was one of my pet peeves in spy/police shows
It almost makes me mad they guessed it right when that undercuts their whole point that the methodology is unreliable, but Corridor Crew is just that good.
First let me just say that the "enhance" tech in most police shows, CSI and similar shows are so obviously fake it's laughable. But it's still used to get a "good" story. Sometimes it works and sometimes it's just cringe worthy.
Now some things that wasn't talked about in this episode are color space and temporal enhancement. In a lot of pictures there can be details lost in dark areas or in single color areas that can be made visible by manipulating gamma. It's even used to embed water marks in pictures that are close to impossible to see unless you use a program to look for single step color changes. If you look at a digital photo there can be some details in a dark area that can be made visible this way. But you never get the full color result shown in these shows.
Temporal enhancement is pretty interesting. It uses several consecutive pictures from a video to enhance a still image. This works very well with low light photography where you end up with a very grainy picture as there are simply to few photons available. By combining several pictures it's possible to recreate the information missing in one picture by combining it with the information in the other pictures. Similar things can be achieved if you have a video sequence of something small that takes up to few pixels of video resolution as long as the camera or the target is moving at least slowly. It's still no CSI "enhance" but can improve the image quality a bit more than only using a single frame source. And it never introduces any information that isn't available in the source.
Now as for untangling distorted images I remember some years back when the police was looking for some person who had uploaded a lot of pictures on the net, but always had obscured the face using a distortion filter in a image manipulation program, I think it was Photoshop, but that's not really important. What turned out to be important was that they figured out what program was used, probably by looking at the metadata in the pictures. Then they were able to apply the same filter but using negative values. The filter was some kind of skew filter applied a swirl effect to a circular area. Reversing the "swirl" they were able to get a picture good enough to identify the person. Now if a hard blur, posterization or simply just a solid color fill had been used this would not have been possible.
Thanks for actually bringing up video/image forensics software. Certainly deconvolution as used in astronomy might be a worthy discussion topic. If you know what you *should* be seeing you can improve quality. Also stuff like drizzle or tap super resolution, highlight recovery etc.
Also thank you for noticing that "AI" methods are just informed by their training data and will make stuff up.
Steve mould had a video with video forensics software go highlight small subpixel changes, it could read your pulse from a video. Veritasium showed research to film a plastic bag and reconstruct sound.
Video has Soo much data that is around - we just need to use it.
Not all transformations are inversible. It's not always a bijection. Injective functions exist. Entropy isn't reversible. There is 3 directions a ball could have rolled from to end in one spot. Even if you know the exact state of every subatomic particle(impossible by quantum mechanics) you can predict the future but not calculate the past.
Wren telling Niko he solved the case and Niko staying perfectly calm and going "you did? Wow." Was so damn funny.
For the security camera footage test, would it be possible to assimilate data together from multiple frames rather than just picking one frame?
Yes. I've had to do it. It can get annoying and time-consuming, but it's worth it. The more data you can provide to a solver, the better.
Astro photography is often done by taking video footage of the target, and then using Image Stacking software to combine the information of hundreds, or even thousands, of images. The programs can identify 'noise' from the sensor and eliminate it, while preserving recurring data. There are some incredible photos from backyard astronomers who have used these methods. To try it out I took a few seconds of film of the floorboard in our dining room, in poor light, and then ran them through a free ware astro photography stacking program. The individual frames were pretty ordinary, but the stacked image had a lot more detail in it (that I could compare by looking at the floor :) ). Cheers
Hai guys, excellent topic , next time include digital forensics also like , recovering damaged data , creating whole image from pieces of data , location etc... Thanks 👍
I like to imagine that there is one day out of the year where Jake films an entire years worth of sponsor segments all in one go, one after another. It's like Christmas but it sucks.