Here is the code I used: gitlab.com/duder1966/youtube-projects/-/tree/master/OpenCV/triangulation I just checked this code March 11, 2022. Read the notes.
Hi Darwin , while running the above code am getting error like "Corrupt JPEG data: 2 extraneous bytes before marker 0xd0" . Any suggestion to fix this issue
I've seen something similar. I think the problem is that OpenCV doesn't like the data format coming from your camera (not a Python issue). I have a camera that causes OpenCV to print an error similar to that, but it doesn't cause any problems. If the error isn't fatal, you may be able change the level of error reporting in OpenCV, or you maybe able to re-direct the stderr. Try it with a different camera if you have one and see if it changes or goes away.
You need to write a small python opencv script just to test the camera. Once you can read from the camera you can use those settings for the big script. Maybe use the script from the motion capture video.
you said in the video at 21:00 "we didnt do all the fancy camera calibration" but I think all the stuffs you adjust from 11:30 to 13:20 and the test after that from 15:50 to 19:00 is another kind of "fancy calibration" actually. The difference is they do with algorithm code and you do manually. Thanks for the great video anyway
Hi i remembered i commenting and asked bunch of questions in this video years ago. Now i want to thank Mr. Clayton for sharing this video, this video is actually saved my final project. I actually already graduated and currently building a company. Thanks to you once again. You helped my future
I have replicated this project step by step and works perfectly. I keep evaluating if I can implement some of these ideas on a project that I am conceptualizing. Thanks a lot for share. I agree if the cameras are quite similar in performance the angles strategy would be the final calibration.
Cool. Check out the auto-measurement video (it's newer). It has a different way of calibrating the cameras (one camera) that might help. At some point I will merge the two projects, but I haven't had time lately.
Thanks a lot for making it. I was finding exactly that type of video; finally got it. you are very great as you are sharing your technique with us. can you make more videos on stereo vision.
So a quick question. Are angles A and B for triangulation in the X and Y directions just the angle values you get from taking the inverse tangent of pixels from center over focal length?
Will a couple of stereo cameras be sufficient for the cricket ball tracking application? Also how am I supposed to accurately superimpose the 3d coordinates obtained on a 3d model of the pitch?
Sir, I am working on a project where I am creating an Object Detection model for Autonomous Vehicles in different weather conditions. Here, depending on the information in the image, I want to vary the intensity of the weather condition filter to augment the images. The images have roads, cars, traffic signs, buildings, etc the typical Autonomous Vehicle dataset. The dataset that I am working on has images from the dashboard of the car. Now, if a car is closer, the intensity of the filter is directly proportional to the distance between the camera and the objects. What should be my approach or any specific steps to achieve this ?
Hello Clayton, thank you very much. I'm trying to fully grasp/visualize the principles. We compute, x, y an z distances (coordinates) in space from the cameras. From the verso or your sheet of paper, using the horizontal axis on the frame, we get angles A and B (cam A and cam B). Then from the recto of your sheet of paper using A and B we get the distance "d", which would be the distance (coordinate) from camera to the object along the "x" axis in space ? Then using the vertical axis on the frame, you find new A and new B and compute (recto) new "d", which would be the distance (coordinate) from camera to object along the "y" axis in space ? So "d" is computed twice, once for x in space and once for y in space ? If so, how to you compute "z" ?
The first step is as you said. Get the horizontal angles from each camera to the object and use these to compute the distance from the cameras to the object. You can then use that distance and the vertical angle from one of the cameras to locate the object on the vertical plane. That is enough to locate the object in 3D.
is it possible to know x and y co ordinates using one camera and is it possible to know angle between two co ordinates with respect to horizontal or vertical line in 2d xy plane/image
As long as you know one measurement, like the distance from the camera, or the length of an object in the frame, you can calculate other angles and distances in the frame. Mostly trigonometry. There will be some deviation from the lens distortion.
@@ClaytonDarwin objects would be circular I mean circle..circle sizes known .. background page size is known..only want to know x and y value of the centre of any Circle using one camera which is in top view
Yes, opencv can easily find circles and tell you the center point in pixels. If you know the size of the circle, then you could convert pixels to another measurement scale.
Yes, the code should work fine. However you may have to slow the frame rate down to keep the processor from running at 100% all the time and lagging. I'd start with 10 FPS, and see how that works.
From 15:48 in the video, you measure the distance between the cameras' centre-points with a ruler. When I do that, I get a slightly different distance when measuring close-up compared to further away. What factors might be causing it? And why is it necessary to get the same distance no matter where you measure?
In this case, I'm using the centroid of the largest contour in each frame. But you could do it another way, such as selecting the largest contour in one frame, then use images recognition to find the nearest match in the other frame. Or maybe a combination of both.
hello sir thanks for information i just want to ask this.I calibrated camera with chess table so is it okay to use this code with camera which is calibrated chess board ? Also in this shared code does it include camera calibration and distance(angles +triangulation) or just distance(angles+triangulation) ?
Hi Clayton, I'm planning to use a single camera in my project to detect various points of interest like banks and pharmacies and calculate their 3D coordinates to update the spatial database. Since the camera will be in motion, I'm wondering if your method is applicable and how I can adjust the code accordingly . Just to note, the camera's position is always known via GPS .
Judging the distance from a single camera is very different from the method being demonstrated here. If you knew an accurate GPS location and azimuth to the target from two different places, you could assume two cameras and apply the same process.
Thanks for making this video! Very inspirational. Just one question; how would camera calibration improve this setup? Would you get more accurate results, for example? Or would the setup be more robust to errors in the camera alignment?
In my latest project (see the laser tracker) I'm exploring this topic. I think that yes, calibration will help. But I'm also going to do some manual checks with a printed chart. The trick is getting the most accurate angles on which to base all the other measurements. With my new project I'm having trouble with parralax that I have to solve first.
Thanks for the video. I'm using object detection with YOLO. Would I need to detect the same object with 2 cameras separately before trying this triangulation technique? I am able to detect objects with one camera without issues, but how would I make sure the object being detected by the 2nd camera is the same one the 1st camera detected? (if there are multiple objects in the scene)
Yes. I would identify the object in one camera, then select a central area in the object, then find the nearest match to that in the other camera. You could also limit the search area in the second camera to the expected location. For example, the object should have about the same Y axis location in both cameras.
I haven't done anything that requires high accuracy yet, so I haven't had any problems. But certainly I can see that that could be an issue. I did have to run the cameras in separate threads, which allows basic synchronization. Otherwise they would be way off.
Dear Clayton, this very interesting tutorial is what I had been looking for; Now I want to create my own code with the mathematics that is in the video but two doubts have arisen I hope you can help me: 1. Following the tutorial from minute 5 ... What if the point is to the left of the frame between 0 and 320 px? I made the calculation of the angles using complementary angles, the formula that I use if it is on the left or right would be the following: a = number of pixels that the point moves away from the center (half of the frame "320") x = arctan (a / d (calculated distance to frame, same as the video)) having the theoretical angle x and knowing the angle of view of the camera in my case 60 I did the following operation: angle sought = 180 ° - (30 ° (angle of the middle of the frame) + 60 ° (angle where the camera has no vision) + x °) but I'm not sure about it. 2. It is still not clear to me why there are angles with respect to X and Y, would it not result in two different distances? or how the 4 angles are related. Thank you very much in advance for your time and I hope there are no very inconsistent questions.
In my example, there are four (4) distances being calculating. The distance from each camera to the object (the hypotenuse of each right triangle) on plane XY, and the distance from the X axis between the cameras to the object (the common sides of the triangles) on plane XY. And finally the hypotenuse of a triangle in 3D space from the object to a point between the cameras (compensating for the angle above the horizon). You can do the math in several ways. However, they are all based on the angles from the cameras, and the distance between the cameras. Those are your reference measurements. You should be able to calculate distance to any point identifiable in both camera frames. Work your method out on paper and check that it works on known values/examples. You may have to get your trigonometry books out. Then it will work when you code it.
hi, Clayton ! great work, but I am so curious after you calibration the two cameras, you built coordinate system in your frame center, and I could get the L/R camera's angle and the distance between the camera, but if I move the target, the angle with L/R camera should be changed, and the distance will also change. How could you realize the target coordinate system by your calculation, and I was also using Python and OpenCV to do my project, but I was trouble on this part and I want to use kalman filter to implementation motion detection...???
If the cameras are in fixed positions relative to each other, and you can determine the angles to target, then you can calculate distance. Then you can determine target position in 3d space relative to the cameras. And if you know the position of the cameras, you can locate the target relative to anything else. If the target moves, or if the cameras move, recalculate. I'm recalculating with every frame.
U got the angle measurements wrong bcoz there is distance between camera lens and sensor. This is proven by the fact that u can see beyond the lines close to the camera. What u got to do is mark 2 points on each side. Join the points with a line and extend it towards the camera. The 2 lines from each side will intersect at a point. That is your effective sensor position. U got to measure the angle at that point between the 2 lines. This will give u the correct field of view, which will be less than what u measured currently.
I saw that too. You gotta put the camera on the paper itself then mark how wide you can see say 12 inches away then do the same thing for 6 inches away. Connect the dots to form the lines. You should not be able to see outside your field of view. In the video you see the field of view lines in the frame
thanks, excellent example of triangulation. I have one question. I use camera lens and it has barrel distortion. If I use this method, lens distortion must removed ?
Yes. There are some functions in OpenCV that can help with lens distortion. How much effort depends on how much accuracy you need. My cheap setup works fine without distortion correction for what I do.
@Clayton Darwin, hi thanks for the video it is very informative, i just want to ask that is it possible to reconstruct the 3D image . i am using the multi stereo cameras and from that i am getting the image which i have to reconstruct the 3D image and separate the object from the Background. looking forward to your answer. Thanks
if we secure 2 cameras on the car , let say distance between them will be 1.5m . is it gonna work? I mean is that how tesla recognize object on camera? first object detection , then distance detection to the detected objec with method showing on your video, and then just calculating speed? did i get it right.
@@ClaytonDarwin ok thanks . one more thing if i do this with distance like few miles? for detecting and calculation distance to the ship at sea. How u think on distance over 2 miles it will work ok>?
At that distance, the angle between cameras will be very small, maybe undetectable between the two camera frames (not 1 pixel different). Probably won't work.
This is great! Question, though: what if the cameras are not fixed? For example, what if they are both pan-tilt-zoom cameras, tracking the object (i.e., always in the center of the camera frames)?
Hello Darwin. Excellent video, and thanks for sharing this great work. I have a question that I don't know, they already asked it and you answered it. This code, what kind of object does it detect? For example, if you recognize a ruler, a hand, any object, or what it seems to me, could it be something yellowish? I would appreciate your response. Greetings.
I did not understand what the distance between the camera and the screen is , that entire part. What is the screen you are talking about and how do you calculate the angle from that?
It's a hypothetical frame derived from the camera angles and camera resolution. You need that to get the hypothetical distance from the camera to the plane where the pixels are, so that you can calculate the angle.
Hi! Thanks for posting the update. Really great stuff! I was wondering, is this demo running directly off of the Rock64 board? I'm hesitating to pull the trigger on that board, but if you're getting this level of performance using two usb cameras on the Rock64 then I might just have to make an impulsive late-night purchase haha. Thanks ahead of time and keep up the great work!
This is running from my PC where I was working on it that evening. Unfortunately, I got distracted from this project by some mechanical changes I want to make (and not having the time to make them) so I haven't got it going on the Rock64 yet and don't know how it will perform. I hope to get back to it soon.
Clayton Darwin thank you for the fast response. I’ve watched some of your other videos and they’ve already convinced me to pull the trigger on the Rock64. Hopefully I’ll have an update in the near future regarding its performance. I’m looking forward to your next video. Good luck on your changes!
The X Y and Z are the distances from the center point between the two cameras along the respective axes. Together, using the Pythagorean theorem, the give us distance D to the target.
This particular video has grabbed my interest in this series. My education in mathematics is limited to high school algebra, geometry, trigonometry, with some analytic geometry and spherical trigonometry. Through my hobbies, I have kept some of the knowledge fresh, particularly geometry and trigonometry. I probably missed this from a previous video, but what computer system and programming language are you using to process the camera data real time and move stepper motors accordingly?
Hi Chuck, I almost always use a Debian flavor of Linux for my OS. For programming, I almost always use Python3 as my base programming language. All of the camera and image processing in this series is being done with the OpenCV package. It is written in C, so it's really fast, and it has a plug in for Python so that it is very easy to use. Of course, Python and OpenCV will work on Windows and OSX too. To run the stepper motors, I actually just use Python. I use the output pins on a RaspberryPi or Rock64 or ESP32 (or sometimes a USB serial to GPIO adapter) to control an h-bridge big enough to handle the amperage the motors need. This makes it very simple and cheap to do. However, when controlling pin states and h-bridges, you are limited to only whole or half-stepping the motors. No fancy multi-stepping that requires varying the amperage.
Awesome Claton, really helpful video. I am new to image processing using OpenCV and aware of using Python, currently, I am working on a small proof of concept in Golf area to find the distance between the flag pin (hole) and the ball(s) through the image's taken from high resolution camera. Currently, I am using only one camera to take the snaps frequently and will be using for image processing. At any point of time, the camera and the flag pin (hole) are fixed length and the only variation will be with ball movement. Camera, flag pin (hole) and ball will be forming a triangle and finding a way to get tan@ to get the distance from camera to flag pin and use this reference to get hold from camera to ball to find distance between flag pin (hole) and ball now. Please can you help me on how to go about it. Really appreciate, if you can help to solve this puzzle for me (using Python and OpenCV) :).
The problem is that you probably won't be a fixed distance from the ball. Even if you are a fixed distance from the hole, that won't help you locate the ball accurately because the green isn't required to be a horizontal plane. Since the ball is a specific size, you may be able to gauge distance based on size on camera frame if you have a super high resolution camera, but some kind of triangulation will probably be required. Locating the points on a triangle requires 3 measures, side angle side, or angle side angle. You have to figure out how to control for those. Might be easier to get distance with a laser range finder.
I have a question. How do you understand how much is the distance between the cameras in pixels? How do you create a unit of measurement if you have pixels and cm/m ?
If you know the exact size of the target ahead of time, you can estimate the distance using one point of view. Otherwise, two points of view, with a known distance between them is required.
Thanks for nice work. I have understand about the distance calculation from camera to the object using two cameras. Now I have following things and doing flowing steps 1-Calibrating the cameras 2- Rectified the frames 3- Calculate the Distances for each objects Using Your Approach Now I have implemented and changed your code for multi-object detection and distance calculation for each object from the cameras like you have done for single object.Now I am trying to calculate the inter-distance between the objects by getting the three coordinates X,Y,Z as you calculated for single object.After calculating the 3D XYZ coordinates for each object, I have calculated the euclidean distance using their calculated XYZ coordinates.First of All , is it right approach ?. Secondly I have problem related to change in orientation, While I rotate the objects without change relative position, distance get change between them.Why it so.Can you guide me about what is done wrong by me? I am looking forward for your response.Thank You
I’m working on a tennis ball tracker for the application of cricket. A widely renowned example would be the hawkeye. I wanted to know whether placing the two cameras wider apart would improve or decrease the accuracy.
In general, 1) wider camera spacing improves accuracy, and 2) having the target in the center of the frame improves accuracy. You have to balance the two.
Thanks for the quick response Clayton! Would you be willing to suggest a tech/hardware stack (workflow) if I sent you a 3d model of the place where it’s going to be planted? I would be eternally grateful!
you need 4 cameras at each corner of the court. If you did 6 with two at half court, you could get position but use pairs to average position and account for errors at the farthest point between them which (would be the center of the court for 4 or the center of each players half for 6 cameras). Higher MP cameras would increase accuracy, so 6 100MP cameras would give you the most accuracy, and averaging the 4 cameras (the two pairs on each side of one half court) would give you the position on that half court and help minimize errors. They sell a 100mp pi cam from arducam for $400, so itd cost ~$3200 to build the highest resolution system you could. Then keep tinkering with it.
line 120, in run frame2 = ct2.next(black=True, wait=1) line 445, in next frame = self.buffer.get(timeout=wait) AttributeError: 'NoneType' object has no attribute 'get' Any help?
Hello. Thank you for the wonderful work it's excellent work. I have one question when the code is set the show frames black color for one second then then disappeared to try to solve the problem but did not succeed
There are three first mistakes in this line Targets1 = Targetter1.targets (Frame1) and the second error in this line frame3, Contors, hierachy = cv2.findcontors (frame3, cv2.RETR_EXTERNAL,cv2 .CHAIN_APPROX_SIMPLE)an comment it is VALUE ERROR: Not ENGO VALUES TO WPPUC (expected 3, got2) and Print DONE and print WARN: 1 Terminating async callback
Your version of OpenCV is returning 2 rather than 3 values from the findcontors function. You'll have to change the code to match what is returned. I don't know what that is because my version returns 3 items.
I use the latest version of OpenCV 3.4 and the latest version of Python and use "Pycharm" to work in the windows 10 .. is your opinion of the solution to this problem Is not changing the or you need another thing .... please help me because a project that is a graduate of the university depends on this code.thanks
You need to run the findcontours function by itself and determine what it returns. I use Linux. Not Windows, so I can't tell you. The code works for me. Once you know what it returns, change the code accordingly so that it works for you. Keep looking at the errors, isolating the problem line, and making changes. That's how you troubleshoot.
Hello , I am trying to make a school project by using using this system and I want to contact you. Is the any mail adress that I can use for communication
Sir, can you help me and guide me to find out a method to measure width/height of an object through an image with a single camera? is it even possible?
really appreciate the video, i am running into an issue while trying to run your code , can you help me out? if cv2.getWindowProperty('Left Camera 1',cv2.WND_PROP_VISIBLE) < 1: AttributeError: module 'cv2' has no attribute 'WND_PROP_VISIBLE'
Hello Mr. Clayton ... I want to have the code calculate the distance for a specific object such as (a person or a car) using the haar cascade function. I am currently successful in making the program calculate the distance for the specified target but the problem is that when any other unwanted target moves (like a cat or Dog or anything) inside the image, the program also measures the distance ... and I just want it to measure the distance for the predetermined target, how can I disable the distance measuring feature for the moving object.
Use object recognition to locate your target in one camera frame. Then use a sub-frame of just the target area to locate the target in the other camera frame (OpenCV function). Then use the triangulation method. In other words, don't use motion to detect targets.
I have not come to a good result ... I still have many problems ... Could you explain me more like where to put exactly the code (Haar Cascade) in the main code .... thank you
You are going to have to write a new program. You can use mine as an example, but it won't be as simple as just replacing one part. You can also use my code from the facial recognition video.
The idea is the same, but the process will be more complicated. You will probably need to add a object recognition layer to limit detection to cars, and you will likely have to deal with multiple objects at once. It might get messy, and you may need a powerful processor to get a good frame rate.
I am using HSV filter to track an object, but i am finding difficulties in finding the pixel position values of the object I am tracking. How can I find these pixel values?
@@ClaytonDarwin Is it not possible to find the location after putting the HSV filter on? Do I have to make the program find the contours of the object first?
The same idea can be used for depth mapping. However, this uses motion to locate objects in the camera frames. You would have to use another method to identify objects. I probably won't have time to do a video about that for a while.
Traceback (most recent call last): File "", line 113, in run frame1 = ct1.next(black=True,wait=1) File "", line 206, in next frame = self.buffer.get(timeout=wait) AttributeError: 'NoneType' object has no attribute 'get' i get this error code,could help me?
videoio(MSMF): OnReadSample() is called with error status videoio(MSMF): async ReadSample() call is failed with error status: error i can't solve error
@@wkwkzl Get the camera.py code from my original targeting video. Make sure you can run both of your cameras with that, one at a time, then try the dual camera setup.
Sorry. This is the distance video. I thought this comment was from another video. You can track as many objects as you like, but you will have to come up with a different method to target them. I'm using movement of the largest object, but you could use a different method, like color or shape detection, object recognition, etc.
Hello Darwin, can you change the motion detection, instead of detecting any motion, we change it to detect motion at a certain point on the hand, hope it's okay. Please help, fix the code and update it in my comment section. thank you so much
@@ClaytonDarwin what is the . Range of detection , sir.. Can this algorithm calculate distance of objects 8-10 m away from stereo cam.. Thanks in Advance
Thank you very much for the information you provided. I have a question. Is it possible to get 3d dimension from the cameras placed on top of the soccer goal? Something like picture in this link. i.ibb.co/Mh5Q1jp/cameras.jpg
hi mr when i run your code i get these following errors can you help me : File "C:/Users/18736/PycharmProjects/Projet1076/tes1.py", line 129, in run frame1 = ct1.next(black=True, wait=1) File "C:/Users/18736/PycharmProjects/Projet1076/tes1.py", line 455, in next frame = self.buffer.get(timeout=wait) AttributeError: 'NoneType' object has no attribute 'get'
Hello sir, I need your help I am making a project in which we detect multiple objects with distance estimation using a single camera for visually impaired persons. If you can guide me I am really thankful. Please reply to me ASAP!. Thanks, sir
i have issue /targetingtools.py", line 129, in run frame1 = ct1.next(black=True,wait=1) /targetingtools.py", line 455, in next frame = self.buffer.get(timeout=wait) AttributeError: 'NoneType' object has no attribute 'get
Why is self.buffer == None? That's the problem. Did you delete it? Did you call the camera thread start() function? Did it start the thread correctly? Did you name your camera correctly (correct address)?
Hi Clayton sorry for asking. I'm getting error, the code worked but the video just popout for a second and exiting itself. The error code is, Traceback (most recent call last): File "C:/Users/Asus/PycharmProjects/liveStream/targeting_tools.py", line 133, in run targets1 = targeter1.targets(frame1) File "C:/Users/Asus/PycharmProjects/liveStream/targeting_tools.py", line 557, in targets frame3,contours,hierarchy = cv2.findContours(frame3,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE) ValueError: not enough values to unpack (expected 3, got 2) DONE [ WARN:1] global C:\projects\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (674) SourceReaderCB::~SourceReaderCB terminating async callback [ WARN:2] global C:\projects\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (674) SourceReaderCB::~SourceReaderCB terminating async callback appreciate it if you could help me out. Thanks a lot
You are using a new/different version of OpenCV. The output from cv2.findContours has changed. Go to line 557 (line the error says) and change "frame3,contours,hierarchy" to "contours,hierarchy".
I can't get your code to run. It's throwing ` File "c:/Users/.../Desktop/gdrive.py", line 130, in run frame2 = ct2.next(black=True,wait=1) File "c:/Users/dvkbp7/Desktop/gdrive.py", line 455, in next frame = self.buffer.get(timeout=wait) AttributeError: 'NoneType' object has no attribute 'get' ` Did anyone else have this issue?
yes i have same issue File "c:/Users/Meghasham Joshi/Documents/targetingtools.py", line 129, in run frame1 = ct1.next(black=True,wait=1) File "c:/Users/Meghasham Joshi/Documents/targetingtools.py", line 455, in next frame = self.buffer.get(timeout=wait) AttributeError: 'NoneType' object has no attribute 'get
Why is self.buffer == None? That's the problem. Did you delete it? Did you call the camera thread start() function? Did it start the thread correctly? Did you name your camera correctly (correct address)?
In this case, you're only getting a single point based on movement. To construct a 3d image, you would need many points. There are some functions in OpenCV that can help with that.
@@ClaytonDarwin well, thank you, any way, do you know how to generate point cloud from 2d image using laser triangulation with camera ? it's more like, have to find depth (Z axes) of object, so, we can generate 3d image. thank you in advance
Accuracy (or range) depends on the camera lens and pixel density, and on the distance between cameras. Here I'm using cheap vga cameras about 6 inches apart. Seem pretty good to 20 feet inside. Haven't had it outside yet.
great video I learned a lot! 1) will you do a video of connecting the ROCK64 to the cameras and finishing the project? 2) Do you think it's possible to make a live 2d Depth video with the rock64? and maybe later use this to make a 3D map of a house? 3)I'm new to raspberry pi and rock64, do you think it's a good idea to start straight with the rock64 if I want eventually to make this project? THANK YOU VERY MUCH, very informative and interesting!
1) Yes. I'm planning on re-starting the targeting project using the ROCK64 very soon. I just got distracted trying to finish some other projects. 2) Sure. It can be done. The only question is how fast. I don't know yet. 3) It's a trade off. The RasPi has better support and more users, but is lower power and costs more. The ROCK64 is more powerful and costs less, but has less support and users. If you know Linux pretty well, my vote is for more power and less cost. If you are new to Linux and IO pin programming, go with the RasPi.
@@ClaytonDarwin Thank you for the fast response (: I know how to work with Arduino and the GPIO pins. But I started thinking about a new project, which is a drone with a stereo camera on board that can do obstacle avoidance and mapping of the inside of a house. That's why I looked into raspberry pi and rock64. Unfortunately, I don't have experience with Linux or raspberry pi system. I thought maybe to save the money and go straight for the rock 64. what do you think? again thank you very much!
I think I would buy an older raspi (but not too old, but cheap) and learn Linux, Python, OpenCV and how to use the GPIO. You will need to know them for your project. Then upgrade to the ROCK64 when ready. Learn to use the command line for Linux. You want to run your project without a GUI to save processing.
@@ClaytonDarwin You are so helpful, thank you! the last question hehe :) can you recommend a cheap type of camera for the stereo vision that I could buy on eBay?
I have not used a stereo camera yet, so I don't have a recommendation. They are fairly expensive. I have been using 2 cheap USB cameras and mounting them together. You won't need high def for most things, vga will usually be fine and use way less processing. You will want both to have the same field of vision. If you set your raspi up as a headless server, you can connect via a network cable and then use the USBs for cameras instead of peripherals. Use ssh and command line to connect. However, most of your development you can do on a Linux desktop. It should transfer to the raspi.
this is error on ubuntu 18.04 plz help me to solve python3 -u targeting_tools.py VIDEOIO ERROR: V4L2: Could not obtain specifics of capture window. VIDEOIO ERROR: V4L: can't open camera by index 1 /dev/video1 does not support memory mapping VIDEOIO ERROR: V4L: can't open camera by index 2 Traceback (most recent call last): File "targeting_tools.py", line 129, in run frame1 = ct1.next(black=True,wait=1) File "targeting_tools.py", line 455, in next frame = self.buffer.get(timeout=wait) AttributeError: 'NoneType' object has no attribute 'get' DONE
Your cameras are not working correctly. They are probably not set up with the right parameters. Start by getting one camera to work in opencv by itself. Once you can do that, use those parameters to set up the cameras in the targeting tools script.
What about using a single camera, such as one of these ruclips.net/p/PLIfSLL2wFuaqaT_r3uMl_n7BEM6Llbizz then using know locations in the image to estimate the location of other pixels in the view? I have been trying to find an example of this but no luck yet.
Here is the code I used: gitlab.com/duder1966/youtube-projects/-/tree/master/OpenCV/triangulation I just checked this code March 11, 2022. Read the notes.
Hi Darwin , while running the above code am getting error like "Corrupt JPEG data: 2 extraneous bytes before marker 0xd0" . Any suggestion to fix this issue
I've seen something similar. I think the problem is that OpenCV doesn't like the data format coming from your camera (not a Python issue). I have a camera that causes OpenCV to print an error similar to that, but it doesn't cause any problems. If the error isn't fatal, you may be able change the level of error reporting in OpenCV, or you maybe able to re-direct the stderr. Try it with a different camera if you have one and see if it changes or goes away.
@@muthukumars50 i am getting same error can you please help me to sort this out
@@ClaytonDarwin i am actully getting same error with logitech c270 camera can u tell me how to fix this issue
You need to write a small python opencv script just to test the camera. Once you can read from the camera you can use those settings for the big script. Maybe use the script from the motion capture video.
One of the best videos of OpenCV no doubt. Simple but powerful.
Thanks for taking the time to say that. 👍
you said in the video at 21:00 "we didnt do all the fancy camera calibration" but I think all the stuffs you adjust from 11:30 to 13:20 and the test after that from 15:50 to 19:00 is another kind of "fancy calibration" actually. The difference is they do with algorithm code and you do manually. Thanks for the great video anyway
Fancy just means I don't know how to do it yet.
Thank you. Very informative! Can't find any better explanation
Hi i remembered i commenting and asked bunch of questions in this video years ago. Now i want to thank Mr. Clayton for sharing this video, this video is actually saved my final project. I actually already graduated and currently building a company. Thanks to you once again. You helped my future
Richardo, thanks for sharing this great news. Best of luck on a great career. 👍
Also, what type of company are you building? Just interested.
@@ClaytonDarwin seriouslu you helped alot Thanks!
@@ClaytonDarwin E-Commerce and Agriculture both based in Indonesia
👍
I have replicated this project step by step and works perfectly. I keep evaluating if I can implement some of these ideas on a project that I am conceptualizing. Thanks a lot for share. I agree if the cameras are quite similar in performance the angles strategy would be the final calibration.
Cool. Check out the auto-measurement video (it's newer). It has a different way of calibrating the cameras (one camera) that might help. At some point I will merge the two projects, but I haven't had time lately.
thanks a lot sir. The best thing about the video is it do explain the small but necessary details we should keep under consideration
Thanks a lot for making it. I was finding exactly that type of video; finally got it. you are very great as you are sharing your technique with us. can you make more videos on stereo vision.
Thanks. I'll try to make more as soon as I have time.
best explanation so far. It will help our robotic project. thank you
Awesome!!! So detailed explanation. Thank you SO MUCH!
Finally it worked for me on logitech c 270 stereo vision on opencv thank u so much
I am also getting 30 fps
Glad to hear it!
So a quick question. Are angles A and B for triangulation in the X and Y directions just the angle values you get from taking the inverse tangent of pixels from center over focal length?
Will a couple of stereo cameras be sufficient for the cricket ball tracking application? Also how am I supposed to accurately superimpose the 3d coordinates obtained on a 3d model of the pitch?
What a great job Man , thanks , you taught me something new ,👍🏻
Sir, I am working on a project where I am creating an Object Detection model for Autonomous Vehicles in different weather conditions. Here, depending on the information in the image, I want to vary the intensity of the weather condition filter to augment the images. The images have roads, cars, traffic signs, buildings, etc the typical Autonomous Vehicle dataset. The dataset that I am working on has images from the dashboard of the car. Now, if a car is closer, the intensity of the filter is directly proportional to the distance between the camera and the objects.
What should be my approach or any specific steps to achieve this ?
Hello Clayton, thank you very much. I'm trying to fully grasp/visualize the principles. We compute, x, y an z distances (coordinates) in space from the cameras.
From the verso or your sheet of paper, using the horizontal axis on the frame, we get angles A and B (cam A and cam B). Then from the recto of your sheet of paper using A and B we get the distance "d", which would be the distance (coordinate) from camera to the object along the "x" axis in space ? Then using the vertical axis on the frame, you find new A and new B and compute (recto) new "d", which would be the distance (coordinate) from camera to object along the "y" axis in space ? So "d" is computed twice, once for x in space and once for y in space ? If so, how to you compute "z" ?
The first step is as you said. Get the horizontal angles from each camera to the object and use these to compute the distance from the cameras to the object. You can then use that distance and the vertical angle from one of the cameras to locate the object on the vertical plane. That is enough to locate the object in 3D.
@@ClaytonDarwin I see ! Thank you.
is it possible to know x and y co ordinates using one camera and is it possible to know angle between two co ordinates with respect to horizontal or vertical line in 2d xy plane/image
As long as you know one measurement, like the distance from the camera, or the length of an object in the frame, you can calculate other angles and distances in the frame. Mostly trigonometry. There will be some deviation from the lens distortion.
@@ClaytonDarwin objects would be circular I mean circle..circle sizes known .. background page size is known..only want to know x and y value of the centre of any Circle using one camera which is in top view
Yes, opencv can easily find circles and tell you the center point in pixels. If you know the size of the circle, then you could convert pixels to another measurement scale.
hi Clayton, thank you for your video. I am wondering if it is possible to do this project on a raspberry pi with two usb camera or pi camera?
Yes, the code should work fine. However you may have to slow the frame rate down to keep the processor from running at 100% all the time and lagging. I'd start with 10 FPS, and see how that works.
@@ClaytonDarwin Thank you
Hi Clayton, i still dont get it, why you should calculate the Y? Dont you still got the distance using only X?
In two dimensions the distance from the center point is the hypotenuse of the triangle with base x and side y.
From 15:48 in the video, you measure the distance between the cameras' centre-points with a ruler. When I do that, I get a slightly different distance when measuring close-up compared to further away. What factors might be causing it? And why is it necessary to get the same distance no matter where you measure?
Theoretically the cameras should be parallel for the math to work out correctly. The more accurate you can be, the better your results will be.
Sir, how did you localize the point in the two images? Is it segmentation or something else?
In this case, I'm using the centroid of the largest contour in each frame. But you could do it another way, such as selecting the largest contour in one frame, then use images recognition to find the nearest match in the other frame. Or maybe a combination of both.
hello sir thanks for information i just want to ask this.I calibrated camera with chess table so is it okay to use this code with camera which is calibrated chess board ? Also in this shared code does it include camera calibration and distance(angles +triangulation) or just distance(angles+triangulation) ?
This code depends on knowing the viewing angle of the camera. That is the method of calibration. As long as you do that, you should be okay.
Hi Clayton, I'm planning to use a single camera in my project to detect various points of interest like banks and pharmacies and calculate their 3D coordinates to update the spatial database. Since the camera will be in motion, I'm wondering if your method is applicable and how I can adjust the code accordingly . Just to note, the camera's position is always known via GPS .
Judging the distance from a single camera is very different from the method being demonstrated here. If you knew an accurate GPS location and azimuth to the target from two different places, you could assume two cameras and apply the same process.
Thanks for making this video! Very inspirational. Just one question; how would camera calibration improve this setup? Would you get more accurate results, for example? Or would the setup be more robust to errors in the camera alignment?
In my latest project (see the laser tracker) I'm exploring this topic. I think that yes, calibration will help. But I'm also going to do some manual checks with a printed chart. The trick is getting the most accurate angles on which to base all the other measurements. With my new project I'm having trouble with parralax that I have to solve first.
Thanks for the video.
I'm using object detection with YOLO. Would I need to detect the same object with 2 cameras separately before trying this triangulation technique?
I am able to detect objects with one camera without issues, but how would I make sure the object being detected by the 2nd camera is the same one the 1st camera detected? (if there are multiple objects in the scene)
Yes. I would identify the object in one camera, then select a central area in the object, then find the nearest match to that in the other camera. You could also limit the search area in the second camera to the expected location. For example, the object should have about the same Y axis location in both cameras.
@@ClaytonDarwin Thank you. I'll try it once I get myself a dual-camera set up.
@elyakimlev Were you successful in achieving that? If so, could you please guide me?
I have read that synchronization between 2 cameras may affect the disparity map and distance measures. Did you have any problems with your cameras?
I haven't done anything that requires high accuracy yet, so I haven't had any problems. But certainly I can see that that could be an issue. I did have to run the cameras in separate threads, which allows basic synchronization. Otherwise they would be way off.
Dear Clayton, this very interesting tutorial is what I had been looking for; Now I want to create my own code with the mathematics that is in the video but two doubts have arisen I hope you can help me:
1. Following the tutorial from minute 5 ... What if the point is to the left of the frame between 0 and 320 px? I made the calculation of the angles using complementary angles, the formula that I use if it is on the left or right would be the following:
a = number of pixels that the point moves away from the center (half of the frame "320")
x = arctan (a / d (calculated distance to frame, same as the video))
having the theoretical angle x and knowing the angle of view of the camera in my case 60 I did the following operation:
angle sought = 180 ° - (30 ° (angle of the middle of the frame) + 60 ° (angle where the camera has no vision) + x °)
but I'm not sure about it.
2. It is still not clear to me why there are angles with respect to X and Y, would it not result in two different distances? or how the 4 angles are related.
Thank you very much in advance for your time and I hope there are no very inconsistent questions.
In my example, there are four (4) distances being calculating. The distance from each camera to the object (the hypotenuse of each right triangle) on plane XY, and the distance from the X axis between the cameras to the object (the common sides of the triangles) on plane XY. And finally the hypotenuse of a triangle in 3D space from the object to a point between the cameras (compensating for the angle above the horizon).
You can do the math in several ways. However, they are all based on the angles from the cameras, and the distance between the cameras. Those are your reference measurements. You should be able to calculate distance to any point identifiable in both camera frames. Work your method out on paper and check that it works on known values/examples. You may have to get your trigonometry books out. Then it will work when you code it.
hi, Clayton ! great work, but I am so curious after you calibration the two cameras, you built coordinate system in your frame center, and I could get the L/R camera's angle and the distance between the camera, but if I move the target, the angle with L/R camera should be changed, and the distance will also change. How could you realize the target coordinate system by your calculation, and I was also using Python and OpenCV to do my project, but I was trouble on this part and I want to use kalman filter to implementation motion detection...???
If the cameras are in fixed positions relative to each other, and you can determine the angles to target, then you can calculate distance. Then you can determine target position in 3d space relative to the cameras. And if you know the position of the cameras, you can locate the target relative to anything else. If the target moves, or if the cameras move, recalculate. I'm recalculating with every frame.
Awesome.!! love from India.
Great Tutorial, thank you !!
U got the angle measurements wrong bcoz there is distance between camera lens and sensor. This is proven by the fact that u can see beyond the lines close to the camera. What u got to do is mark 2 points on each side. Join the points with a line and extend it towards the camera. The 2 lines from each side will intersect at a point. That is your effective sensor position. U got to measure the angle at that point between the 2 lines. This will give u the correct field of view, which will be less than what u measured currently.
Give us a link to your video so we can see how you did it.
@@ClaytonDarwin I hv not done it. Im just saying frm the mathematics point of view, which could improve ur systems accuracy.
I saw that too. You gotta put the camera on the paper itself then mark how wide you can see say 12 inches away then do the same thing for 6 inches away. Connect the dots to form the lines. You should not be able to see outside your field of view. In the video you see the field of view lines in the frame
@@Dylan-kw8pz make a tutorial please
thanks, excellent example of triangulation. I have one question. I use camera lens and it has barrel distortion. If I use this method, lens distortion must removed ?
Yes. There are some functions in OpenCV that can help with lens distortion. How much effort depends on how much accuracy you need. My cheap setup works fine without distortion correction for what I do.
Clayton Darwin thanks!
Hi Clayton, is it possible to use this approach with cameras with different FoV angles?
Yes. Just more math.
@Clayton Darwin, hi thanks for the video it is very informative, i just want to ask that is it possible to reconstruct the 3D image . i am using the multi stereo cameras and from that i am getting the image which i have to reconstruct the 3D image and separate the object from the Background. looking forward to your answer. Thanks
There is a built-in function/method. See opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_calib3d/py_depthmap/py_depthmap.html
if we secure 2 cameras on the car , let say distance between them will be 1.5m . is it gonna work? I mean is that how tesla recognize object on camera? first object detection , then distance detection to the detected objec with method showing on your video, and then just calculating speed? did i get it right.
Theoretically, sure, but for longer distances being that far apart. Make sure the alignment is right.
@@ClaytonDarwin thx ) that is the princeple how tesla drive ?
I don't really know. They may use LIDAR or something. Probably more than one system.
@@ClaytonDarwin ok thanks . one more thing if i do this with distance like few miles? for detecting and calculation distance to the ship at sea. How u think on distance over 2 miles it will work ok>?
At that distance, the angle between cameras will be very small, maybe undetectable between the two camera frames (not 1 pixel different). Probably won't work.
How far away does the accuracy work? like in the center walking away
Accuracy will depend on camera pixel density and quality of calibration.
Excellent work ! . I have a question : Are the X, Y, Z, and D coordinates in pixel units?
Inches.
This is great! Question, though: what if the cameras are not fixed? For example, what if they are both pan-tilt-zoom cameras, tracking the object (i.e., always in the center of the camera frames)?
This code would not work. However it could be done if you knew the pan/tilt angles and the distance between the cameras. A lot more complicated.
very useful video. thank you !
Hello Darwin. Excellent video, and thanks for sharing this great work. I have a question that I don't know, they already asked it and you answered it. This code, what kind of object does it detect? For example, if you recognize a ruler, a hand, any object, or what it seems to me, could it be something yellowish? I would appreciate your response. Greetings.
In this case I'm just detecting motion. But with opencv you can also detect objects, shapes, and colors.
I did not understand what the distance between the camera and the screen is , that entire part. What is the screen you are talking about and how do you calculate the angle from that?
It's a hypothetical frame derived from the camera angles and camera resolution. You need that to get the hypothetical distance from the camera to the plane where the pixels are, so that you can calculate the angle.
@claytonDarwin : Great explanation, i have a doubt, if it's just X,Y i'm bothered about, i'm better off with just one camera right?
X and Y change depending on Z. If Z is fixed, then one camera will work. See my other video on measurement. I have an example.
@@ClaytonDarwin i checked your video, but can i work without having a known distance to calibrate in the screen?
You have to have a known measurement in the formula.
Hi! Thanks for posting the update. Really great stuff!
I was wondering, is this demo running directly off of the Rock64 board? I'm hesitating to pull the trigger on that board, but if you're getting this level of performance using two usb cameras on the Rock64 then I might just have to make an impulsive late-night purchase haha.
Thanks ahead of time and keep up the great work!
This is running from my PC where I was working on it that evening. Unfortunately, I got distracted from this project by some mechanical changes I want to make (and not having the time to make them) so I haven't got it going on the Rock64 yet and don't know how it will perform. I hope to get back to it soon.
Clayton Darwin thank you for the fast response. I’ve watched some of your other videos and they’ve already convinced me to pull the trigger on the Rock64. Hopefully I’ll have an update in the near future regarding its performance.
I’m looking forward to your next video. Good luck on your changes!
Thanks. Looking forward to the Rock64.
thanks for the video, good information, I have one question, what does the values in x, y tells us? . could you explain me a bit briefly?
The X Y and Z are the distances from the center point between the two cameras along the respective axes. Together, using the Pythagorean theorem, the give us distance D to the target.
This particular video has grabbed my interest in this series. My education in mathematics is limited to high school algebra, geometry, trigonometry, with some analytic geometry and spherical trigonometry. Through my hobbies, I have kept some of the knowledge fresh, particularly geometry and trigonometry. I probably missed this from a previous video, but what computer system and programming language are you using to process the camera data real time and move stepper motors accordingly?
Hi Chuck, I almost always use a Debian flavor of Linux for my OS. For programming, I almost always use Python3 as my base programming language. All of the camera and image processing in this series is being done with the OpenCV package. It is written in C, so it's really fast, and it has a plug in for Python so that it is very easy to use. Of course, Python and OpenCV will work on Windows and OSX too. To run the stepper motors, I actually just use Python. I use the output pins on a RaspberryPi or Rock64 or ESP32 (or sometimes a USB serial to GPIO adapter) to control an h-bridge big enough to handle the amperage the motors need. This makes it very simple and cheap to do. However, when controlling pin states and h-bridges, you are limited to only whole or half-stepping the motors. No fancy multi-stepping that requires varying the amperage.
Hi Darwin. Why is it that when I actually check, Y and Z show the correct value and Z shows the wrong value?
The X coordinate origin is moved to the right camera
This is a program I wrote one weekend 5 years ago. I don't have time to support it. You're just going to have to figure it out yourself.
Hey hi...can we do 3d mapping by using webcams with raspberry pi
Yes. I haven't done much with it, but there is no reason you can't. Processing speed might be an issue.
Awesome Claton, really helpful video. I am new to image processing using OpenCV and aware of using Python, currently, I am working on a small proof of concept in Golf area to find the distance between the flag pin (hole) and the ball(s) through the image's taken from high resolution camera. Currently, I am using only one camera to take the snaps frequently and will be using for image processing. At any point of time, the camera and the flag pin (hole) are fixed length and the only variation will be with ball movement. Camera, flag pin (hole) and ball will be forming a triangle and finding a way to get tan@ to get the distance from camera to flag pin and use this reference to get hold from camera to ball to find distance between flag pin (hole) and ball now. Please can you help me on how to go about it. Really appreciate, if you can help to solve this puzzle for me (using Python and OpenCV) :).
The problem is that you probably won't be a fixed distance from the ball. Even if you are a fixed distance from the hole, that won't help you locate the ball accurately because the green isn't required to be a horizontal plane. Since the ball is a specific size, you may be able to gauge distance based on size on camera frame if you have a super high resolution camera, but some kind of triangulation will probably be required. Locating the points on a triangle requires 3 measures, side angle side, or angle side angle. You have to figure out how to control for those. Might be easier to get distance with a laser range finder.
I have a question. How do you understand how much is the distance between the cameras in pixels? How do you create a unit of measurement if you have pixels and cm/m ?
How did you get this? camera_separation = 5 + 15/16
Pixels are used to get angles. Angles and the inter camera distance are used to get measurements in units.
With a ruler. Imperial inches.
hello , actually your work is very nice , but i want to ask you can we measure the distance but for one camera not two ? Thanks in advance
If you know the exact size of the target ahead of time, you can estimate the distance using one point of view. Otherwise, two points of view, with a known distance between them is required.
i got this error plz help.......AttributeError: 'NoneType' object has no attribute 'get'
You haven't provided enough information from the error message.
Thanks for nice work. I have understand about the distance calculation from camera to the object using two cameras.
Now I have following things and doing flowing steps
1-Calibrating the cameras
2- Rectified the frames
3- Calculate the Distances for each objects Using Your Approach
Now I have implemented and changed your code for multi-object detection and distance calculation for each object from the cameras like you have done for single object.Now I am trying to calculate the inter-distance between the objects by getting the three coordinates X,Y,Z as you calculated for single object.After calculating the 3D XYZ coordinates for each object, I have calculated the euclidean distance using their calculated XYZ coordinates.First of All , is it right approach ?. Secondly I have problem related to change in orientation, While I rotate the objects without change relative position, distance get change between them.Why it so.Can you guide me about what is done wrong by me? I am looking forward for your response.Thank You
I'll respond to your email as soon as I'm able.
I’m working on a tennis ball tracker for the application of cricket. A widely renowned example would be the hawkeye. I wanted to know whether placing the two cameras wider apart would improve or decrease the accuracy.
In general, 1) wider camera spacing improves accuracy, and 2) having the target in the center of the frame improves accuracy. You have to balance the two.
Thanks for the quick response Clayton! Would you be willing to suggest a tech/hardware stack (workflow) if I sent you a 3d model of the place where it’s going to be planted? I would be eternally grateful!
I don't have the qualifications/experience to do that.
you need 4 cameras at each corner of the court. If you did 6 with two at half court, you could get position but use pairs to average position and account for errors at the farthest point between them which (would be the center of the court for 4 or the center of each players half for 6 cameras). Higher MP cameras would increase accuracy, so 6 100MP cameras would give you the most accuracy, and averaging the 4 cameras (the two pairs on each side of one half court) would give you the position on that half court and help minimize errors.
They sell a 100mp pi cam from arducam for $400, so itd cost ~$3200 to build the highest resolution system you could. Then keep tinkering with it.
Thank you for this amazing video.
I need to ask what is the maximum distance range in this project ?
I don't know yet. Still working on it. I haven't taken it outside.
I am preety new to this , how can we get those green Horizontal and Vertical lines in the camera view as shown in video? Thank You..!
OpenCV has drawing functions for lines, circles, text, etc. that you can use to modify frames.
@@ClaytonDarwin oh okay, great Thank You
i think another method would be to find height ranges at some distances. It may be possible
Sure. This is just one way to do it when you have no references.
@@ClaytonDarwin okay I get it. Some people on Facebook have done it by taking refrences and ratios and that would change everytime camera is adjusted.
hello sir, by using this code can we detect objects like yolo detects and find the distance
I'm not familiar with Yolo.
line 120, in run
frame2 = ct2.next(black=True, wait=1)
line 445, in next
frame = self.buffer.get(timeout=wait)
AttributeError: 'NoneType' object has no attribute 'get'
Any help?
If frame is None, your camera can't be read. Probably you have the wrong address.
Hello. Thank you for the wonderful work it's excellent work. I have one question when the code is set the show frames black color for one second then then disappeared to try to solve the problem but did not succeed
It sounds like your cameras (you need 2) are not set up correctly. Read the exception error string. What does it say?
There are three first mistakes in this line Targets1 = Targetter1.targets (Frame1)
and the second error in this line
frame3, Contors, hierachy = cv2.findcontors (frame3, cv2.RETR_EXTERNAL,cv2 .CHAIN_APPROX_SIMPLE)an comment it is
VALUE ERROR: Not ENGO VALUES TO WPPUC (expected 3, got2) and Print DONE and print WARN: 1 Terminating async callback
Your version of OpenCV is returning 2 rather than 3 values from the findcontors function. You'll have to change the code to match what is returned. I don't know what that is because my version returns 3 items.
I use the latest version of OpenCV 3.4 and the latest version of Python and use "Pycharm" to work in the windows 10 .. is your opinion of the solution to this problem Is not changing the or you need another thing .... please help me because a project that is a graduate of the university depends on this code.thanks
You need to run the findcontours function by itself and determine what it returns. I use Linux. Not Windows, so I can't tell you. The code works for me. Once you know what it returns, change the code accordingly so that it works for you. Keep looking at the errors, isolating the problem line, and making changes. That's how you troubleshoot.
Hello , I am trying to make a school project by using using this system and I want to contact you. Is the any mail adress that I can use for communication
You can get my email from my RUclips page.
@Clayton Darwin I actually could not find your email, can you please write it
Use the "About" tab. There is a view email address link.
Sir, can you help me and guide me to find out a method to measure width/height of an object through an image with a single camera? is it even possible?
ruclips.net/video/1CVmjTcSpIw/видео.html
really appreciate the video,
i am running into an issue while trying to run your code , can you help me out?
if cv2.getWindowProperty('Left Camera 1',cv2.WND_PROP_VISIBLE) < 1:
AttributeError: module 'cv2' has no attribute 'WND_PROP_VISIBLE'
I was running on Linux. What are you using? Not every option works for every operating system.
Hello Mr. Clayton ...
I want to have the code calculate the distance for a specific object such as (a person or a car) using the haar cascade function. I am currently successful in making the program calculate the distance for the specified target but the problem is that when any other unwanted target moves (like a cat or Dog or anything) inside the image, the program also measures the distance ... and I just want it to measure the distance for the predetermined target, how can I disable the distance measuring feature for the moving object.
Use object recognition to locate your target in one camera frame. Then use a sub-frame of just the target area to locate the target in the other camera frame (OpenCV function). Then use the triangulation method. In other words, don't use motion to detect targets.
I have not come to a good result ... I still have many problems ... Could you explain me more like where to put exactly the code (Haar Cascade) in the main code .... thank you
You are going to have to write a new program. You can use mine as an example, but it won't be as simple as just replacing one part. You can also use my code from the facial recognition video.
Salaam Ali Mohsen Can you give me your code
In the description and first comment.
Great thanks for such a wonderful work and outcomes. It'll be very cool if there will be a chance to share the code.
Send me an email.
Can I use 1 camera to calculate all the value X, Y, Z, Dis?
Generally speaking, no. There are some tricks you can use for special situations. Like my measurement video.
@@ClaytonDarwin thank you Darwin. Hopefully your idea helped me find the coordinates of the tomatoes on the tree :D :D
If you know the distance from the camera to the target, you can use just 1 camera. See my auto-measuring video.
@@ClaytonDarwin No, I need all the value x, y, z, and distance and then add to the function that calculate inverse kinematic of my 6 axis robot arm
Thanks, this vdo show me how machine know object distance.
hi Clayton thanks for making this video, i wonder what if i want to detect a car and measure the distance
The idea is the same, but the process will be more complicated. You will probably need to add a object recognition layer to limit detection to cars, and you will likely have to deal with multiple objects at once. It might get messy, and you may need a powerful processor to get a good frame rate.
Excellent work!
i have a question actually need your help. can i measure between two points via this method in real world scale?
evreka :)
I have another video about measurement. You would have to combine the two methods. It would be complicated.
@@ClaytonDarwin i guess i dont need it. cosine theorem will work...
I am using HSV filter to track an object, but i am finding difficulties in finding the pixel position values of the object I am tracking. How can I find these pixel values?
I think an HSV filter only separates the color. It doesn't provide a location.
@@ClaytonDarwin Is it not possible to find the location after putting the HSV filter on? Do I have to make the program find the contours of the object first?
Yes. The HSF filter will isolate the color, but you will still have to identify a target and find a contour using other methods.
@@ClaytonDarwin I will try that :) Thank you!
how to solve for the Z value?
Use trigonometry with the variables that you have, like I did in the video.
is it possible to measure the dimensions the feature of the product
Yes. Probably so, depending on what features. You could probably just use one camera if you set it at a fixed distance or the product is a known size.
can this code be used for depth maping
could u do a tutorial on it
The same idea can be used for depth mapping. However, this uses motion to locate objects in the camera frames. You would have to use another method to identify objects. I probably won't have time to do a video about that for a while.
Hello Darwin. Why can we detect motion but calculate slowly, and sometimes cannot calculate at all?
I don't understand your question.
@@ClaytonDarwin Calculate at slow speed, then display delay
You can change the frame rate.
@@ClaytonDarwin What is the unit when calculating xyz coordinates?
Whatever you use for calibration.
Traceback (most recent call last):
File "", line 113, in run
frame1 = ct1.next(black=True,wait=1)
File "", line 206, in next
frame = self.buffer.get(timeout=wait)
AttributeError: 'NoneType' object has no attribute 'get'
i get this error code,could help me?
Your camera is probably not connected. Make sure you are using the correct address.
videoio(MSMF): OnReadSample() is called with error status
videoio(MSMF): async ReadSample() call is failed with error status:
error i can't solve error
@@wkwkzl Get the camera.py code from my original targeting video. Make sure you can run both of your cameras with that, one at a time, then try the dual camera setup.
hi sir, so is it possible to use this method and detect the distance of multiple objects, if so can you make a tutorial on it
This does not work for distance. I have another video on using triangulation to estimate distance.
@@ClaytonDarwin could you please send link sir
Sorry. This is the distance video. I thought this comment was from another video. You can track as many objects as you like, but you will have to come up with a different method to target them. I'm using movement of the largest object, but you could use a different method, like color or shape detection, object recognition, etc.
@@ClaytonDarwin ok sir so thank you for replying
@@ClaytonDarwin and sir what is minimum distance that the cameras have to be from each other
Hello Darwin, can you change the motion detection, instead of detecting any motion, we change it to detect motion at a certain point on the hand, hope it's okay. Please help, fix the code and update it in my comment section. thank you so much
It could be done. I don't have time to do it.
@@ClaytonDarwin Please help me, I'm working on a project to graduate from a university in Vietnam
sir, how many FPS are you getting and on what specs are you running this code on ?\
The FPS is written on the frame under the other specs. I think this was on an I3 processor.
@@ClaytonDarwin what is the . Range of detection , sir..
Can this algorithm calculate distance of objects 8-10 m away from stereo cam..
Thanks in Advance
I'm not sure. I've only used it in my house. I think if you have better cameras that would work fine.
@@ClaytonDarwin thanks. For your time
Thank you very much for the information you provided. I have a question. Is it possible to get 3d dimension from the cameras placed on top of the soccer goal? Something like picture in this link. i.ibb.co/Mh5Q1jp/cameras.jpg
Sure. It should work if you can get the image recognition to work fast enough for sports.
hi mr when i run your code i get these following errors can you help me :
File "C:/Users/18736/PycharmProjects/Projet1076/tes1.py", line 129, in run
frame1 = ct1.next(black=True, wait=1)
File "C:/Users/18736/PycharmProjects/Projet1076/tes1.py", line 455, in next
frame = self.buffer.get(timeout=wait)
AttributeError: 'NoneType' object has no attribute 'get'
I don't think your cameras are set up correctly. Make sure you can use your cameras via OpenCV before you try this program.
@@ClaytonDarwin ,Yes I usually use my camera with opencv, but I have only one camera that's 0 that I use every time
@@kohdev2488 You need 2 cameras.
@@ClaytonDarwin thank you it works with 2 cameras, how to adapt a camera, because I need the distance from an object for my project
@@kohdev2488 Triangulation requires 2 cameras.
Hello sir, I need your help I am making a project in which we detect multiple objects with distance estimation using a single camera for visually impaired persons. If you can guide me I am really thankful. Please reply to me ASAP!. Thanks, sir
Detecting distance from one camera will be difficult. You will have to know the size of the target object to estimate distance.
Why distance measure is.so slow
Lots of processing requirements.
Could you do a tutorial on depth mapping with openCV?
Email me and I'll send you the code.
i have issue
/targetingtools.py", line 129, in run
frame1 = ct1.next(black=True,wait=1)
/targetingtools.py", line 455, in next
frame = self.buffer.get(timeout=wait)
AttributeError: 'NoneType' object has no attribute 'get
Why is self.buffer == None? That's the problem. Did you delete it? Did you call the camera thread start() function? Did it start the thread correctly? Did you name your camera correctly (correct address)?
@@ClaytonDarwin can u tell me how to use self.start() in program call
Where the research paper you use?
I'm just making it up as I go along. You can find most of the ideas in Wikipedia.
@@ClaytonDarwin What is the keyword to find those ideas in Wikipedia??
Triangulation. Resection. Intersection.
Hi Clayton sorry for asking. I'm getting error, the code worked but the video just popout for a second and exiting itself. The error code is,
Traceback (most recent call last):
File "C:/Users/Asus/PycharmProjects/liveStream/targeting_tools.py", line 133, in run
targets1 = targeter1.targets(frame1)
File "C:/Users/Asus/PycharmProjects/liveStream/targeting_tools.py", line 557, in targets
frame3,contours,hierarchy = cv2.findContours(frame3,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
ValueError: not enough values to unpack (expected 3, got 2)
DONE
[ WARN:1] global C:\projects\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (674) SourceReaderCB::~SourceReaderCB terminating async callback
[ WARN:2] global C:\projects\opencv-python\opencv\modules\videoio\src\cap_msmf.cpp (674) SourceReaderCB::~SourceReaderCB terminating async callback
appreciate it if you could help me out.
Thanks a lot
You are using a new/different version of OpenCV. The output from cv2.findContours has changed. Go to line 557 (line the error says) and change "frame3,contours,hierarchy" to "contours,hierarchy".
@@ClaytonDarwin hi thanks, I'm Sorry btw for not read the description on the video first.
Don't worry about it. I should really replace that code.
I could not find the code in the website
Sorry. My website is being reworked. I'll have it back up soon.
I can't get your code to run. It's throwing
`
File "c:/Users/.../Desktop/gdrive.py", line 130, in run
frame2 = ct2.next(black=True,wait=1)
File "c:/Users/dvkbp7/Desktop/gdrive.py", line 455, in next
frame = self.buffer.get(timeout=wait)
AttributeError: 'NoneType' object has no attribute 'get'
`
Did anyone else have this issue?
I don't recognize the gdrive.py script? What is that from?
yes i have same issue
File "c:/Users/Meghasham Joshi/Documents/targetingtools.py", line 129, in run
frame1 = ct1.next(black=True,wait=1)
File "c:/Users/Meghasham Joshi/Documents/targetingtools.py", line 455, in next
frame = self.buffer.get(timeout=wait)
AttributeError: 'NoneType' object has no attribute 'get
Why is self.buffer == None? That's the problem. Did you delete it? Did you call the camera thread start() function? Did it start the thread correctly? Did you name your camera correctly (correct address)?
Gosh, why don't I learn this in school?
Can you measure the dimensions of a face?
Yes, but you would have to adapt the algorithm to that specific task.
nice, from X Y Z value that obtained, how to reconstruct that image to 3d ?
In this case, you're only getting a single point based on movement. To construct a 3d image, you would need many points. There are some functions in OpenCV that can help with that.
@@ClaytonDarwin well, thank you, any way, do you know how to generate point cloud from 2d image using laser triangulation with camera ? it's more like, have to find depth (Z axes) of object, so, we can generate 3d image. thank you in advance
I haven't tried it yet.
Sir can we use here raspberry pi camera
Probably. Might be slow, and you'll have to modify the code some.
Awesome ...
Thanks.
Hello sir we can implement this on Raspberey pi 4 b
Yes. It may run slower (fewer frames per second) but it should work fine.
Do i need same resolution of camera? Thanks men
You should be able to run it with different camera types, as long as you set them up correctly.
@@ClaytonDarwin okay i set it already thanks for that. but i get this error 'not enough values to unpack (expected 3, got 2)'
Check the description. Answer is there.
Where is your maximum range of distance?
Accuracy (or range) depends on the camera lens and pixel density, and on the distance between cameras. Here I'm using cheap vga cameras about 6 inches apart. Seem pretty good to 20 feet inside. Haven't had it outside yet.
great video I learned a lot!
1) will you do a video of connecting the ROCK64 to the cameras and finishing the project?
2) Do you think it's possible to make a live 2d Depth video with the rock64? and maybe later use this to make a 3D map of a house?
3)I'm new to raspberry pi and rock64, do you think it's a good idea to start straight with the rock64 if I want eventually to make this project?
THANK YOU VERY MUCH, very informative and interesting!
1) Yes. I'm planning on re-starting the targeting project using the ROCK64 very soon. I just got distracted trying to finish some other projects.
2) Sure. It can be done. The only question is how fast. I don't know yet.
3) It's a trade off. The RasPi has better support and more users, but is lower power and costs more. The ROCK64 is more powerful and costs less, but has less support and users. If you know Linux pretty well, my vote is for more power and less cost. If you are new to Linux and IO pin programming, go with the RasPi.
@@ClaytonDarwin Thank you for the fast response (: I know how to work with Arduino and the GPIO pins. But I started thinking about a new project, which is a drone with a stereo camera on board that can do obstacle avoidance and mapping of the inside of a house. That's why I looked into raspberry pi and rock64. Unfortunately, I don't have experience with Linux or raspberry pi system. I thought maybe to save the money and go straight for the rock 64. what do you think?
again thank you very much!
I think I would buy an older raspi (but not too old, but cheap) and learn Linux, Python, OpenCV and how to use the GPIO. You will need to know them for your project. Then upgrade to the ROCK64 when ready. Learn to use the command line for Linux. You want to run your project without a GUI to save processing.
@@ClaytonDarwin You are so helpful, thank you! the last question hehe :) can you recommend a cheap type of camera for the stereo vision that I could buy on eBay?
I have not used a stereo camera yet, so I don't have a recommendation. They are fairly expensive. I have been using 2 cheap USB cameras and mounting them together. You won't need high def for most things, vga will usually be fine and use way less processing. You will want both to have the same field of vision.
If you set your raspi up as a headless server, you can connect via a network cable and then use the USBs for cameras instead of peripherals. Use ssh and command line to connect.
However, most of your development you can do on a Linux desktop. It should transfer to the raspi.
this is error on ubuntu 18.04 plz help me to solve
python3 -u targeting_tools.py
VIDEOIO ERROR: V4L2: Could not obtain specifics of capture window.
VIDEOIO ERROR: V4L: can't open camera by index 1
/dev/video1 does not support memory mapping
VIDEOIO ERROR: V4L: can't open camera by index 2
Traceback (most recent call last):
File "targeting_tools.py", line 129, in run
frame1 = ct1.next(black=True,wait=1)
File "targeting_tools.py", line 455, in next
frame = self.buffer.get(timeout=wait)
AttributeError: 'NoneType' object has no attribute 'get'
DONE
Your cameras are not working correctly. They are probably not set up with the right parameters. Start by getting one camera to work in opencv by itself. Once you can do that, use those parameters to set up the cameras in the targeting tools script.
@@ClaytonDarwin thank u so much i am willing to use logitech 270
could I find your code on GitHub?
No, But I'll put it back on my website soon.
What about using a single camera, such as one of these ruclips.net/p/PLIfSLL2wFuaqaT_r3uMl_n7BEM6Llbizz then using know locations in the image to estimate the location of other pixels in the view? I have been trying to find an example of this but no luck yet.
Check the auto measure video. Similar.
nice！