Видео

Thank you-glad it is useful! And yes: I use Preview here because opening the L & R images in 2 separate panels is very convenient, but you can replicate this in Photopea etc.

Thanks for the appreciation! In my tests, the best AI depth estimator is still MiDaS, but the new version 3.1, released after this video. There is a very new one called Marigold (huggingface.co/spaces/toshas/marigold), but in my first tests, it's not as good as MiDaS v3.1.

@@WorldofDepth Thank you for the response, I have done quite a bit of research since that comment so I have actually heard about Marigold. Have you tried out Depth Anything? It's even newer than Marigold and is in my testing much better than MiDaS. It has a HF space if you want to try it out.

i have a problem, when i run the last proccess in the 3d Photo inpainting notebook, it says "TypeError: load() missing 1 required positional argument: 'Loader'" i already uploaded the depth and the base image

As you can see, this video workshop was part of NSA Virtual 3D-Con 2021, and other videos of mine were made for similar conferences and some smaller regional events. I presented again at the 2022 3D-Con last month, but unfortunately, the conference was not recorded. If you want to see the latest 3D I'm producing, follow @WorldOfDepth on Instagram, and/or check WorldOfDepth.com (though I need to update that more!).

I use FFMPEG myself, a free command-line program. The relevant filter is ‘stereo3d,’ which has all the options you mentioned: trac.ffmpeg.org/wiki/Stereoscopic. I'm sure there are other programs that will also do it, but I don't know which offhand, and this is the only I can personally recommend.

​@@WorldofDepth thank you man i appreciate the response its very helpful! one last question for you if you dont mind, is t her a tutorial on how to use and incorporate this method ?

@@WorldofDepth because i use 3d combine to do automatic depthmaps and convert into anaglyph 3d just fine but it comes out full color when i would prefer dubois so isther a way to just run the video to get dubois color hue then run it through the 3d combine program after.basically i donmt want to use ffmpeg to convert to 3d i simply want to convert the video to dubois so i can then import it into 3dcombine

@@donrikk8546 I don't know 3d combine at all. But what that ffmpeg filter above does is take a left video + right video as inputs, and merges them into an anaglyph video. So if your other program can export L + R videos, you could use then use ffmpeg to convert to Dubois anaglyph video etc.

@@donrikk8546 official documentation for ffmpeg itself is at ffmpeg.org/ffmpeg.html. For something like a video tutorial on how to use it, you'd have to search around.

It's possible to do that with MiDaS frame by frame, but the resulting video will flicker, so I think it's best to use tools made specifically for video instead. “Consistent Video Depth Estimation” is near the bottom of my collected links (see video description), plus another, but I haven't used them myself. There are surely other similar AIs out there now as well.

@@WorldofDepth can u upload a tutorial for using consistent video depth estimation ? Please

@@Rocketos I don't have one to upload. Perhaps if I have time in the future.

@@WorldofDepth thanks i love your workshops

Thank you, Greg, I appreciate that! Ghosting is always tricky (and ghosting in print is another matter entirely as well). Glad this has helped.

Thank you! This workshop was for the NSA 3D convention, so I may possibly revisit this topic with those other AIs and newer ones for next year's Con. In the meantime I recommend checking out Ugo Capeto's YT channel for reviews of additional AIs.

Indeed it IS generated, per the video description. The depth is designed for laptop or smaller size display.

@@WorldofDepth didn't read description... )... My mistake

Hmm, I just tried the ‘upload version’ notebook and it worked fine. Did you miss the first code box, above “Uploading Your Image”? That is the step that installs timm. If you did run that, it may be that the bandwidth limit for a certain external file was reached, and it was temporarily unavailable, but it should notify you if that's the problem.

@@WorldofDepth , Maybe I did miss it. I'll try again. Thanks for preparing such a great presentation.

Your suspicion was correct. I missed the first step (instal timm). I was able to run it, but the final depth map was very tiny: 284x217 px. Not sure why.

@@MichaelBrownArtist ah, good. But yes, MiDaS v.3 outputs very small depth maps. I would recommend trying 1) starting with a larger input image, 2) using BMD + MiDaS v2.1 as a possible alternative, which outputs at original size, and 3) upscaling the v3 depth map and using something like a symmetrical nearest-neighbor interpolatation method to smooth it. Ugo Capeto recommends the latter; I don't have a program which offers that method, so I've use Imagemagick and the "Kuwahara edge-preserving noise filter" to pretty good results.

@@WorldofDepth , thank you.

Thank you! For this I used 2 x Sony RX100 mark II on a simple Swiss-Arca rail I put together, plus sync cables. The connector for 2 sync cables is a bit tricky to find, but otherwise these parts are widely available. I recommend the "Sony3D" discussion board on groups.io if you want full specs/details on similar rigs.

What you describe sounds more like VR 180º 3D images to me. To have a feeling of standing close to a 3D scene, I think that's the only option, unless you render in full 3D. I don't work in 180º or 360º, but check out the 3D-Con workshops and Special Interest Groups about VR, from both this year and last year.

@@WorldofDepth I wasn't thinking about super wide angle of VR180, but let's say 100-110. Just with deep scene depth. I will study the topic if it is possible to do what I described. Cheers

I use SPM. Which, by the way, produces much smoother stereopairs if you appropriately upscale your 2D image + depth map first. For example, if you're using an SPM deviation value of 60 = 6% of image width, for a 256-level depth map, you need 4267px-wide image. You could also take advantage of the 3PI AI and produce a panning video with it, then extract a stereopair. Not time efficient but would have the best inpainting.

@@WorldofDepth Hi. Thanks for your time and valuable answer. What I noticed is SPM has problems with depth recognition despite depth map seems to be ok, visually. Also, it tears away surfaces like faces when you produce 3D, even with deviation of 25 or 30 (default). Also, produced images look downscaled in quality. I have a depth map and I see in it, the depth is represented correctly. Then SPM makes flat foreground and correct background hmmm... or vertically sliced/layered depth or some things are flat and other are correct. I have spent a lot of time on this software (even with google AI working from my disk ) and never produced a great 3D photo out of depth maps made with MIDAS or LeRES and some other AI software. So, that is why I asked about some other project to produce correct images. Good point on 256-level depth maps and image size.

@@CabrioDriving Almost every AI-produced depth map needs manual corrections, I think, especially if it's an image with people's faces at any significant size. As I say in the video, it can be convenient if you're producing 2D animations that are more forgiving of the depth map, but otherwise it's always going to be work, at this stage of the tech… The sliced/layered depth problem you mention is exactly the image size issue I mentioned. Upsizing for stereopair generation and then downsizing back to original size should smooth those areas.

@@WorldofDepth Thank you for your priceless comments. 1. How did you calculate that needed resolution of 4267 pixels wide 2. what should be image width for 3% deviation? 3. What deviation % you suggest for best effect?

​@@CabrioDriving 1) If you use an 8-bit grayscale depth map that is normalized to go all the way from pure black to pure white, it has 256 different depth levels. If you want to differentiate all of those in a stereopair, then those levels will correspond to horizontally shifting parts of the original image between 0 and 255 pixels. If the maximum shift you want ( = deviation) is 6%, that means 255 pixels must be 6% (or less) of your image width, so WIDTH * .06 = 255, and WIDTH = 4250px. (My original number was slightly off.) 2) By the same method, you need an 8500px-wide image if deviation is 3%. 3) It depends on the picture and the intended use. 3% was the old rule of thumb, and that's good for TV size, I think. For laptop or phone screen size, I probably use 4.5-6% more, or rarely up to 8% for a very deep scene. For display via a projector at wall size, maybe you'd want to go down to 1.5%.

8/18/21: Files restored and working again :) Reference: github.com/vt-vl-lab/3d-photo-inpainting/issues/131

Thanks for the details. Yes, I’d heard about some of these, but do not have PS. Ugo Capeto has said he believes the PS depth map algorithm operates in a way similar to BMD.

@@WorldofDepth I actually provided the PS depth maps he used for making his comparison :-)

So the painting in of those kind of background spaces is exactly what the 3D-Photo-Inpainting AI does-the zoom-in animation at 22:42 is an example. And it does that based on everything it has learned from massive amounts of training. The SPM animation at 23:25 does do some amount of inpainting as well, but I think it's based on straight calculations and copying existing pixels, rather than on AI, and I think it's not as smooth.

Thank you Karsten!

Yes, there are so many methods. Folks get terrific results with custom 4-mirror camera attachments and such. But as most people (myself included) don't have the machining skills/means to make those, I think this is a great alternative! By the way, it's not in this video, but I use a simple DIY rig for the mirror now: worldofdepth.com/daily/201023.html

Cross-view is my default method as well, for anything much larger than phone screen. Yes, the formatting is a perennial question for 3D… I got into VR last fall and love it. You can see some 3D videos I extracted from an Oculus Quest 2 on this channel. I'll take at look at yours the next time I browse in VR.

Thank you-I appreciate that! And wow, sounds like a great show-I'll check that out, and would surely be interested in joining sometime. Email is best to contact me-bottom of the page here (don't want to put it on YT): worldofdepth.com/about.html. Cheers!

Thank you, Vivek! Cheers.

Thank you…it's fun!

Thank you. If by "fixing" you mean with a built device, see the video description link to Donald Simanek's gallery. For handholding, it's very important to be steady and consistent, and I do recommend systematic experimentation when starting out, in order to find the best mirror position and angle. Once you've determined that and can reproduce it reliably, you should be able to get consistently good stereos this way (with a few expected failures as always). See the 2nd half of this article for more on fine-tuning mirror position/angle: worldofdepth.com/tutorials/mirror.html

@@WorldofDepth Thanks Gordon!

Karsten, in case you didn't see it, I did post about a very cheap mirror + phone (or small camera) rig you can assemble yourself. It's how I take catadioptric stereos now, and the stability also enables videos! worldofdepth.com/daily/201023.html