Best place to get the code is at my WEB site on this page: toptechboy.com/jetson-nano/ai-on-the-jetson-nano-lesson-51-improving-nvidia-jetson-inference-library-for-rpi-camera/ or, you can try these lines directly: ss
For the August 31, 2020 version of jetson-inference, gstCamera.cpp has changed. Here's what I did to make it work. Check the version of jetson-inference based on the first date appearing in the CHANGELOG.md: The grep command finds all the lines with "##" in the changelog that have the dates and the head command takes the first one. cat@JetsonCat:~$ cd ~/Downloads/jetson-inference/ cat@JetsonCat:~/Downloads/jetson-inference$ grep "##" CHANGELOG.md | head -1 ## August 31, 2020 cat@JetsonCat:~/Downloads/jetson-inference$ cd utils/camera/ cat@JetsonCat:~/Downloads/jetson-inference/utils/camera$ sha1sum gstCamera.cpp 4edffea5db7b5778a8dfdb9aac56b69a1adb7d85 gstCamera.cpp I use cp -p to save the file modify time and a time stamp to back up source files. cat@JetsonCat:~/Downloads/jetson-inference/utils/camera$ cp -p gstCamera.cpp gstCamera.cpp202101101830 Launch gedit to edit. cat@JetsonCat:~/Downloads/jetson-inference/utils/camera$ gedit gstCamera.cpp Find "nvarguscamerasrc sensor" (menu > Search > Find... or ctrl-f) It's line 144 for the version I'm using. Replace whole line with the following line. Be aware the line is likely wrapped around several time in gedit. Use two tabs for the leading white space. Copy/paste often converts them to spaces.: ss
Using 2 Raspberry Pi Cameras. I cannot thank you enough for your foray into modifying a library. I have never felt able to attempt that step. Now I am ready to step out on my own.
Rasperry Pi Camera is better for performance, it connects directly to GPU, that fps differences comes from there, also more portable, project friendly. Thanks for fixing that bug and all of your good work. You are my biggest tutor and insparation.
So I am on Jetpack 4.4--and am wondering if this has been fixed? In looking at the same file, it appears that section of the code has been edited. Still worth trying to change?
Hi Paul, using the Pi camera but on JetPack 4.5 and used the instructions from @Mike Thompson posted below. The wbmode that worked for my warm white light in the room was wbmode=1. The resolution that keeps the correct aspect ratio is 1280x966 (not 1280x720). Thank you!
Thanks for your wonderful tutorials, Mr.McWhorter. You are a great tutor. Unfortunately I didn't manage to improve my Jetson Inference Library as it seems that the ccp file changed since you premiered this lesson. I tried to live dangerously and tweak the settings following your advise, which in my version are in line 144 (as also outlined in another comment by Derrick). The modified file did compile with the make command without any errors, however the raspberry pi displayed image remained cropped and nothing changed. :( I think that I'll have to live with the cropped image as I don't have a webcam but only a dual raspberry pi cam.
The challenge i have is that these things are changing quicker than I could ever hope to update the videos. In a future lesson this is what I suggest. Read the frame using openCV. Then convert to cudafromnumpy. then you have a frame the inference libraries can use. Do the recognition on the converted frame. The display the original frame, with boxes and labels based on what you learned from the inference line. In a lesson or two from this one I explain.
It seams not working anymore. They change the gstCamera.cpp file by adding #ifdef ENABLE_NVMM... I tryed to manage around but at compilation it can't find the mWidth and mHeight.
Interesting lesson Paul, on the 2g that file has been changed a lot. The patches you put in would not even compile. I did play with sizes and I can set it where ever I want though the aspect ratio gets off. If I cut the 1280 x 720 to 640 x 360 the aspect ratio is good in cuda or cv2. Running 30 fps in both. cv2 version does better if the browser is not running. Thanks Paul
Yes, I think in a later lesson I show my real preferred way, which is read the frame in openCV and then convert to the jetson inference format. Also, check out one of the Jetson XAVIER lessons where I show how to tweak the GSTREAMER string to get much better images from the piCam
Hi Paul, I followed your lesson to access the gstCamera.cpp file. I'm trying to get my rtsp stream working and wasn't able to find rtspscr in the cpp file. Does that mean the current gstCamera does not support rtsp? thanks in advance.
My gstcamer.cpp file is "newer" I think. It has a 2017 comment header. The line that need to be replace was on line 144 in my version and I had to modify the code we are inserting to the following for compatibility: ss
I just want to bring it to your attention that in not where you should have that error : correct it in the gstrCamera.h file static const uint32_t DefaultWidth and DefaultHeight that is where they set that value
My lessons on Arduino IMU show how to communicate between python and arduino over serial. That is what you are trying to do. Look into those lessons and see how to pass data from arduino to and from python. Easy Easy Easy to do.
after wasting an entire day trying to flip the pi cams via the program, I finally took 3 minutes and flipped the cameras over and twisted up my ribbons to make them physically right side up. NOT HAPPY! very OCD! and twisted cables make me uncomfortable.
HI Paul, i followed your tutorial step by step and tried to do a make on the build directory but i got errors and could not proceed further. ERRORS are as follows; [ 83%] Linking CXX shared library ../../../aarch64/lib/python/3.6/jetson_utils_python.so /usr/bin/ld: cannot find -lnpymath collect2: error: ld returned 1 exit status utils/python/bindings_python_3.6/CMakeFiles/jetson-utils-python-36.dir/build.make:230: recipe for target 'aarch64/lib/python/3.6/jetson_utils_python.so' failed make[2]: *** [aarch64/lib/python/3.6/jetson_utils_python.so] Error 1 CMakeFiles/Makefile2:1281: recipe for target 'utils/python/bindings_python_3.6/CMakeFiles/jetson-utils-python-36.dir/all' failed make[1]: *** [utils/python/bindings_python_3.6/CMakeFiles/jetson-utils-python-36.dir/all] Error 2 Makefile:129: recipe for target 'all' failed make: *** [all] Error 2 Have tried to shutdown and reboot the nano, but result is the same. I am not sure what happened here, but i was still able to run the usual codes in code-oss. So i guess i did not break anything. Any advice would be good.
Hi Paul, after searching the web for a solution, i discovered that the numpy version that comes along with Jetpack 4.4 was 1.19, however, the numpy version i had was 1.16 for some reason i do not know why. Maybe its due to my other installs that may have changed by numpy version to 1.16. Anyway, after upgrading the numpy version back to 1.19 and did a make again and then sudo make install, everything went smoothly. I was also able to run the same code in code-oss and try out all sort of width and height
@@paulmcwhorter Get a Contigo vacuum insulated mug with push to drink valve and lock. When I knock mine over, there's a loud clunk but not even one precious drop of coffee is wasted. Save the Coffee! and by extension - the cost of a keyboard pays for a lot of coffee! (mxkeys multi-host keyboard that I switch between my PC and the Jetson Nano.)
Very useful. Thank you. A small trivia: I noticed that u put the width as 3264 and height as 2464. The Pi camera v2 actually has a max resolution of 3280 (width) x 2464 (height). So should the height be put as 3280 instead of 3264 in the change in gstreamer? Update: The latest Jetpack I see the following string : ss
The answer is you have to pole your camera, and have it tell you what parameters it accepts. If you have not already, install v4l2. Then in the linux terminal put in v4l2-ctl --list-formats-ext. It will return accpeted sizes and frame rates. If you dont match those offered, you can have troubles.
If you look at your mic, you are getting a bunch of weird colors on it. I am getting the same thing. I am not sure, but I think the source probably isn't RGBA. Using v4l2 to inspect the camera shows the Pixel Format :'RG10" 10 bit Bayer RGRG/GBGB. As my intellesense stopped working, I am not really sure what options I can use for cam.CaptureRGBA and cv2.COLOR_RGBA2BGR.
@D Johnson, After making the correction to the NVIDA jetson-inteference code, I too am seeing blue speckles on black objects. Maybe the next tutorial will correct the problems we both are seeing? Regards, TCIII
@@TheRealFrankWizza I have the same problem.. really blotchy blue speckles only when using Jetson Infefence. I also noticed the color problem with Paul's mic. Something ain't right.
Thanks for the fix. I've never heard or understood what support, if any, NVIDIA gets from the Raspberry Pi foundation. The foundation treats the camera source code as their crown jewels so they can sell $3 worth of electronics for $25. They even put a crypto chip on the camera (V2) to prevent cloning. And, why doesn't NVIDIA support the cheaper V1 camera? Maybe a secret agreement with RPI foundation? I like the Pi camera but I wonder if it's really worth the extra cost. Lately I've been working with the ESP32 cam, board plus camera ~$6.
![Seungmin "그렇게, 천천히, 우리(As we are)" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/kAzmhLHePqU/mqdefault.jpg)
Best place to get the code is at my WEB site on this page:
toptechboy.com/jetson-nano/ai-on-the-jetson-nano-lesson-51-improving-nvidia-jetson-inference-library-for-rpi-camera/
or, you can try these lines directly:
ss
For the August 31, 2020 version of jetson-inference, gstCamera.cpp has changed. Here's what I did to make it work.
Check the version of jetson-inference based on the first date appearing in the CHANGELOG.md:
The grep command finds all the lines with "##" in the changelog that have the dates and the head command takes the first one.
cat@JetsonCat:~$ cd ~/Downloads/jetson-inference/
cat@JetsonCat:~/Downloads/jetson-inference$ grep "##" CHANGELOG.md | head -1
## August 31, 2020
cat@JetsonCat:~/Downloads/jetson-inference$ cd utils/camera/
cat@JetsonCat:~/Downloads/jetson-inference/utils/camera$ sha1sum gstCamera.cpp
4edffea5db7b5778a8dfdb9aac56b69a1adb7d85 gstCamera.cpp
I use cp -p to save the file modify time and a time stamp to back up source files.
cat@JetsonCat:~/Downloads/jetson-inference/utils/camera$ cp -p gstCamera.cpp gstCamera.cpp202101101830
Launch gedit to edit.
cat@JetsonCat:~/Downloads/jetson-inference/utils/camera$ gedit gstCamera.cpp
Find "nvarguscamerasrc sensor" (menu > Search > Find... or ctrl-f)
It's line 144 for the version I'm using.
Replace whole line with the following line. Be aware the line is likely wrapped around several time in gedit. Use two tabs for the leading white space. Copy/paste often converts them to spaces.:
ss
@@mikethompson5119 It still shows some delay in the raspberry pi camera
@@mikethompson5119 Thanks Mike, it worked on my JetPack 4.5!
@@mikethompson5119 I am on JetPack 4.5, and it works. Thank you Mike!
@@mikethompson5119 Thanks a lot Mike. Works without a hitch.
With my notifications on, I can't wait to watch my favorite channel, Paul McWhorter. Thank you so much.
Using 2 Raspberry Pi Cameras. I cannot thank you enough for your foray into modifying a library. I have never felt able to attempt that step. Now I am ready to step out on my own.
Rasperry Pi Camera is better for performance, it connects directly to GPU, that fps differences comes from there, also more portable, project friendly. Thanks for fixing that bug and all of your good work. You are my biggest tutor and insparation.
Good job on editing that file. Definitely makes a difference. Thanks, Paul.
Paul with the fresh cut. I see you.
So I am on Jetpack 4.4--and am wondering if this has been fixed? In looking at the same file, it appears that section of the code has been edited. Still worth trying to change?
I've got the same question 😃
Working on a raspberry pi Camera! Thank you for the lesson!
Hi Paul, using the Pi camera but on JetPack 4.5 and used the instructions from @Mike Thompson posted below. The wbmode that worked for my warm white light in the room was wbmode=1. The resolution that keeps the correct aspect ratio is 1280x966 (not 1280x720). Thank you!
Thanks for your wonderful tutorials, Mr.McWhorter. You are a great tutor. Unfortunately I didn't manage to improve my Jetson Inference Library as it seems that the ccp file changed since you premiered this lesson. I tried to live dangerously and tweak the settings following your advise, which in my version are in line 144 (as also outlined in another comment by Derrick). The modified file did compile with the make command without any errors, however the raspberry pi displayed image remained cropped and nothing changed. :( I think that I'll have to live with the cropped image as I don't have a webcam but only a dual raspberry pi cam.
The challenge i have is that these things are changing quicker than I could ever hope to update the videos. In a future lesson this is what I suggest. Read the frame using openCV. Then convert to cudafromnumpy. then you have a frame the inference libraries can use. Do the recognition on the converted frame. The display the original frame, with boxes and labels based on what you learned from the inference line. In a lesson or two from this one I explain.
It seams not working anymore. They change the gstCamera.cpp file by adding #ifdef ENABLE_NVMM... I tryed to manage around but at compilation it can't find the mWidth and mHeight.
Interesting lesson Paul, on the 2g that file has been changed a lot. The patches you put in would not even compile. I did play with sizes and I can set it where ever I want though the aspect ratio gets off. If I cut the 1280 x 720 to 640 x 360 the aspect ratio is good in cuda or cv2. Running 30 fps in both. cv2 version does better if the browser is not running. Thanks Paul
Yes, I think in a later lesson I show my real preferred way, which is read the frame in openCV and then convert to the jetson inference format. Also, check out one of the Jetson XAVIER lessons where I show how to tweak the GSTREAMER string to get much better images from the piCam
Hi Paul, I followed your lesson to access the gstCamera.cpp file. I'm trying to get my rtsp stream working and wasn't able to find rtspscr in the cpp file. Does that mean the current gstCamera does not support rtsp? thanks in advance.
My gstcamer.cpp file is "newer" I think. It has a 2017 comment header.
The line that need to be replace was on line 144 in my version and I had to modify the code we are inserting to the following for compatibility:
ss
Derrick Muncy @DerrickMuncy - thanks. That was very helpful. If you do not mind, how would you modify it to allow for a flip method input?
@@jeraldgooch6438
Works well with Derrick's changes including flip method
Could I ask if you have latency with your modifications at a resolution of 1280x720 ? I have a 2 second delay. Have you changed something else ?
Good on ya Derrick!
Well done for sorting this
@@chrisjohnston1791 What change did you make to get the flip to work? Thanks.
Hi Paul. Dusty has made same changes by 8th July 2020 in this and other related files. Best regards.
I think my update is for the newer file. See above.
Great videos, been doing 4 a day. I have 2 problems with my RPi cam, image is upside-down and the frame rate is just below 10. What am I doing wrong?
I just want to bring it to your attention that in not where you should have that error : correct it in the gstrCamera.h file static const uint32_t DefaultWidth and DefaultHeight that is where they set that value
Hello Paul,
Could you please do some tutorials on how to interface communication between Arduino, Raspberry Pi and Jetson Nano via Serial?
My lessons on Arduino IMU show how to communicate between python and arduino over serial. That is what you are trying to do. Look into those lessons and see how to pass data from arduino to and from python. Easy Easy Easy to do.
@@paulmcwhorter Thank you very much. Is this compatible with both GPIO and USB serial or only one of them?
I spent entirely too much time trying to figure out how to flip the pi cam. Hope we get to fix hat in this video.
after wasting an entire day trying to flip the pi cams via the program, I finally took 3 minutes and flipped the cameras over and twisted up my ribbons to make them physically right side up. NOT HAPPY! very OCD! and twisted cables make me uncomfortable.
HI Paul, i followed your tutorial step by step and tried to do a make on the build directory but i got errors and could not proceed further.
ERRORS are as follows;
[ 83%] Linking CXX shared library ../../../aarch64/lib/python/3.6/jetson_utils_python.so
/usr/bin/ld: cannot find -lnpymath
collect2: error: ld returned 1 exit status
utils/python/bindings_python_3.6/CMakeFiles/jetson-utils-python-36.dir/build.make:230: recipe for target 'aarch64/lib/python/3.6/jetson_utils_python.so' failed
make[2]: *** [aarch64/lib/python/3.6/jetson_utils_python.so] Error 1
CMakeFiles/Makefile2:1281: recipe for target 'utils/python/bindings_python_3.6/CMakeFiles/jetson-utils-python-36.dir/all' failed
make[1]: *** [utils/python/bindings_python_3.6/CMakeFiles/jetson-utils-python-36.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2
Have tried to shutdown and reboot the nano, but result is the same.
I am not sure what happened here, but i was still able to run the usual codes in code-oss. So i guess i did not break anything.
Any advice would be good.
Hi Paul, after searching the web for a solution, i discovered that the numpy version that comes along with Jetpack 4.4 was 1.19, however, the numpy version i had was 1.16 for some reason i do not know why. Maybe its due to my other installs that may have changed by numpy version to 1.16. Anyway, after upgrading the numpy version back to 1.19 and did a make again and then sudo make install, everything went smoothly. I was also able to run the same code in code-oss and try out all sort of width and height
Thanks, Paul. It seems I do not have the permission to modify gstCamera.cpp file. Could you help me with that please?
Use sudo gedit . . . try that
Hey Paul you drink a lot of coffee, have you ever spilled your coffee? 😂
Yes, sometimes I spill coffee on keyboard, and it kills the keyboard
Paul McWhorter it happens to the best of us
@@paulmcwhorter Get a Contigo vacuum insulated mug with push to drink valve and lock. When I knock mine over, there's a loud clunk but not even one precious drop of coffee is wasted. Save the Coffee! and by extension - the cost of a keyboard pays for a lot of coffee! (mxkeys multi-host keyboard that I switch between my PC and the Jetson Nano.)
Very useful. Thank you. A small trivia: I noticed that u put the width as 3264 and height as 2464. The Pi camera v2 actually has a max resolution of 3280 (width) x 2464 (height). So should the height be put as 3280 instead of 3264 in the change in gstreamer?
Update: The latest Jetpack I see the following string :
ss
The answer is you have to pole your camera, and have it tell you what parameters it accepts. If you have not already, install v4l2. Then in the linux terminal put in v4l2-ctl --list-formats-ext. It will return accpeted sizes and frame rates. If you dont match those offered, you can have troubles.
If you look at your mic, you are getting a bunch of weird colors on it. I am getting the same thing. I am not sure, but I think the source probably isn't RGBA. Using v4l2 to inspect the camera shows the Pixel Format :'RG10" 10 bit Bayer RGRG/GBGB.
As my intellesense stopped working, I am not really sure what options I can use for
cam.CaptureRGBA
and
cv2.COLOR_RGBA2BGR.
@D Johnson, After making the correction to the NVIDA jetson-inteference code, I too am seeing blue speckles on black objects. Maybe the next tutorial will correct the problems we both are seeing? Regards, TCIII
@@thomascoyle3715 I was seeing this before the Correction, but only when using Jetson Inference.
@@TheRealFrankWizza I have the same problem.. really blotchy blue speckles only when using Jetson Infefence. I also noticed the color problem with Paul's mic. Something ain't right.
Thanks for the fix.
I've never heard or understood what support, if any, NVIDIA gets from the Raspberry Pi foundation. The foundation treats the camera source code as their crown jewels so they can sell $3 worth of electronics for $25. They even put a crypto chip on the camera (V2) to prevent cloning. And, why doesn't NVIDIA support the cheaper V1 camera?
Maybe a secret agreement with RPI foundation?
I like the Pi camera but I wonder if it's really worth the extra cost. Lately I've been working with the ESP32 cam, board plus camera ~$6.
One of the cameras I use, which came with the jetbot kit, is marked V1.3.
I don't know if I should buy Jetson nano or raspberry pi 4B
I want to learn Artificial intelligence and Linux
Absolutely get the jetson nano. the pi does not have a GPU. Really need a nano
using a raspberry pi cam and cheapo web cam