Это видео недоступно.
Сожалеем об этом.
How to Create a Custom Object Detection Model with TensorFlow
HTML-код
- Опубликовано: 17 авг 2024
- Hey guys welcome back, Ben again! Today we are continuing the project we left off on last time. This time we are learning to detect custom objects using TensorFlow's Object Detection API. This video goes over installing LabelImg for creating our labels, generating csv files, and creating TFrecord files. The next video in this series will be training our custom model and testing it.
How to Train a Custom Model: • How to Train a Custom ...
Commands:
python generate_tfrecord.py --csv_input=data/test_labels.csv --output_path=data/test.record --image_dir=images/
python generate_tfrecord.py --csv_input=data/train_labels.csv --output_path=data/train.record --image_dir=images/
Useful Links:
My GitHub: github.com/Ben...
LabelImg: github.com/tzu...
datitran's GitHub with raw files: github.com/dat...
Fatkun Batch Downloader: chrome.google....
sentdex's RUclips: / @sentdex
![EST Gee - NEVER NEED (Feat. Selfpaid Savage) [Official Video]](http://i.ytimg.com/vi/f2cRtr52HXI/mqdefault.jpg)
I love how you uploaded the whole video and did not edit out you errors. It helped me a lot.
Commands should be:
python generate_tfrecord.py --csv_input=data/test_labels.csv --output_path=data/test.record --image_dir=images/test/
python generate_tfrecord.py --csv_input=data/train_labels.csv --output_path=data/train.record --image_dir=images/train/
im getting error in the train command.. the test command runs perfectly
This is great, it's hard to find a good tutorial for custom object detection :) very good work!! Thanks Ben!
Very clear video, and very helpful. Some (minor) suggestions - list clearly which versions you are using (I started with Tensorflow 2 because I started here and not your install vide and it didn't work, but everything worked fine when I used 1.15 -- I could have missed the warning though). Can you also upload your notes as a pdf? They are difficult to see in how you stream now. Plus it's nice to have a notes sheet. Also, maybe make your terminal fill at least half the screen (Windows/Super + left/right arrow) with at least 1-2 font sizes larger? Great video, though
zoom, please
if anyone's getting the error image not found or no such directory use this
python generate_tfrecord.py --csv_input=data/train_labels.csv --output_path=data/train.record --image_dir=images/train
and similarly for test
got the same error and fixed it on own :) and then saw ur comment
King, you are the best!
you should have all your images with annotations in the directory "images"
GOD you saved me from hell. bless you!
HERO!
Traceback (most recent call last):
File "generate_tfrecord.py", line 26, in
from object_detection.utils import dataset_util
ModuleNotFoundError: No module named 'object_detection'
??????????what should i do?????????
same
Thank you, 3 hours to find your video, your video was the only that works for i, thank you again!
just subscribed, very helpful videos, thank you so much man ! You helped me a lot !
getting some pandas error in generate_tfrecord.py...can nyone help or can you tell how to execute that generate_tfrecord.py file in google colab....
please can anyone help?
AttributeError: module 'tensorflow' has no attribute 'app' any one have a solution for this ? i tried downgrading tensorflow but it wouldent let me
Thank You Bro. That was very Helpful.👍👍 Ur a gr8 Teacher.😎😎
Thank you so much for this helpful video! Really helped for my semester 6 mini project :)
can this work with raspberry pi 4?
after you convert it in a tflite mode yes
Hello, I want to do this process not for a single category but for 2 different categories, for example, apple and pear, how can I do it with 2 categories?
Thanks for the video, great work!
Ama just subscribe real quick cause you talking bout it..tech tech let's go
Excellent work sir. Thanks for sharing.
Sir, can i know which research paper are you used for this project as reference?
Kindly let me know about it sir.
Does anyone know why I have an error of 'no module named object_detection' please? It's when I try to generate the TFRecords. I am following step by step and am using PyCharm. Thanks
try pip3/pip install object_detection
Great work, thanks.
Is possible to convert this custom model in Tensorflow Lite (runnable on Android device)?
I believe to convert it there is just a script you have to run. I haven’t messed with TF lite yet but I’ll make a video about it when I do!
@@LazyTech825 If you made a video how to convert tflite then it would be better for us
Hey Ben, thanks for the video. Could you release the step by step document?
I've been editing it over time while trying to fix up, I will upload a more "complete" version of it soon
generate_tfrecord needs the following updates to run with TensorFlow 2:
(Warning! I'm still having trouble getting the training to start, and I'm still unsure if the errors I got later on are still stemming from my fixed generate_tfrecord. All I know is that these changes will not generate an error upon running the generate_tfrecord file.)
tf.app has been removed, but you can just as easily, with minimal changes, use absl-py package instead.
pip install absl-py
then on generate_tfrecord file, do:
import absl
Now, where on the file it says tf.app.flags, simply replace with absl.flags
and on the last line of the file, where it says tf.app.run(), simply replace it with absl.app.run(main)
All right, now two last minor changes:
on line 88, where it says tf.python_io.TFRecordWriter(FLAGS, output_path), change it to:
tf.io.TFRecordWriter(FLAGS, output_path)
and on line 47, where it says tf.gfile.GFile(....., simply change it to tf.io.gfile.GFile(.......
You're all set!!
The TensorFlow team does excellent work making sure all changes from TF1 to TF2 are not scary at all!
hi i already did the step that you're addressing. But now i encountered another problem. After i run the generate_tfrecord.py, it shows this:
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a NewWriteableFile: : The system cannot find the path specified.
; No such process
Can you help me with this? Thanks !!
@@rafaeladi00 Sorry, I don't know about this error... Usually your best bet is searching the error message on Google. Chances are there is some forum thread out there where someone had the same problem and someone else has reached out with the solution. Especially on the Stack Overflow forum!
But keep in mind you might be getting deeper in a hole of making a chimera of TensorFlow 2 fixes for a TensorFlow 1 guide, and who knows what other incompatibilities you will find after you solve this one.
So my true recommendation is moving away from TensorFlow 1 guides and starting to get fully TensorFlow 2 projects to work on your computer (if it won't take too long to run it), or on Google Colab.
Since my previous comment, I found an official TensorFlow 2 tutorial for Object Detection on Google Colab. You can find it by searching "TensorFlow 2 meets the Object Detection API". It's a blog post, and on it, there is a bullet point: "Colab demonstrations of eager mode compatible few-shot training and inference" which contains two links for great Google Colab tutorials which you can then modify for your own projects.
I myself have spent the last couple months exploring and getting familiar with this code by modifying it a bit, and this Lazy Tech's video was super useful as a guide for how to input my own dataset (in my case, people holding cellphones!). I am slowly getting familiar with the code, as it takes some patience to start understanding in detail what is going. It's after that that you can start making more advanced tunings and modifications to the training model beyond changing the dataset to one your own. I've also used other guides as reference that I found with simple google searches.
I'm still hoping Lazy Tech makes another video for TF2 object detection, as I loved this one very much, but for now the official tutorial I mentioned earlier is the best resource I've found.
Good luck with your machine learning, and always keep it ethical! ;)
@@rafaeladi00 Did you get a solution to your issue? I am having the same problem
Thanks, this helped!
I noticed a typo/mistake in your instructions: the argument for tf.io.TFRecordWriter() should be FLAGS.output_path.
@@rafaeladi00 Maybe I'm too late for you, but this might help someone else.
It looks like the script can't find the path for something. Are you sure that the paths are correct when you are running the script?
python generate_tfrecord.py --csv_input=data/test_labels.csv --output_path=data/test.record --image_dir=images/
With the paths in above script call you must also run it from the same directory where images and data folders are.
Must be a lot of eagle eyes out there. I don't have magnifying glass big enough to see your tutorial
Good work, my man(like Denzel says it).When is the next part coming out?. And can u explain the main code in the next part line by line perhaps?
Other tensorflow tutorials(by other creators) are convoluted for me, thanks.
Thanks! Hopefully the next part will be out on Wednesday. I can try to explain some more things in the code, but to be honest a lot of it is above my level as a programmer. Most of the process is just editing minor parts of the code to fit your system which allows it to be an (sorta) easy API to use
Don't know if anyone will be reading this in the future, but how would you label occluded objects (objects blocked by another object)?
QMimeDatabase: Error loading internal MIME data
An error has been encountered at line 1 of : Premature end of document.:
please can you help us on how to find confusion matrix for ssd-mobnet model.? please
Good work man, one question for multi class object detection what changes we should do in generate_tfrecord file ?
Ben, thank you for your work.
Does anyone deal with the similar issue? I am trying to generate tfrecord but the error occurs: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbf in position 99: invalid start byte.
i guess it could be some invalid char in your xml files. give a check on the sanity of your files.
Thanks , but how to convert tflite or SSd?
can i resize the image after labeling them ?
you can not do that because after resize the xml generated need to be re adjust as per new resize image. It's better to resize image first and then label. or use roboflow for ease
Hello, does including negative photos to my dataset improve the detection of custom object?
i always get the error when trying to execute the generate.tfrecord. can anybody help me with this?
What error do you get? Not sure if I can help but I'll give it a shot if you've got more details. One thing I had a lot of trouble with was the version - I think for this to work you need to use tensorflow 1.15 - Tensorflow 2 does *not* work with this tutorial. That will lead to lots of errors in generate.tfrecord
@@nicolebaldy3535 im getting a error on tensorflow 1.15 that says "File "generate_tfrecord.py", line 23, in
from object_detection.utils import dataset_util
ModuleNotFoundError: No module named 'object_detection'"
Would be nice if you would explain how to use TF2.x. Like your description is showing " how to in 2020.. "
Ironically I made this video a few days before they released support for tf2... ouch. Fortunately this version still works! I will be making a new video using tf2 methods in the future.
Hi. I got an error when i'm trying to convert the xml to csv.
python xml_to_csv.py
Traceback (most recent call last):
File "xml_to_csv.py", line 4, in
import pandas as pd
ModuleNotFoundError: No module named 'pandas'
Can you help me with that?
Hello, it seems that you are missing the module pandas. If you're using an anaconda environment just do "conda install pandas" or you can just type "pip install pandas"
maybe install pandas in python site packages?
Hey I'm getting an error that says 'no module named pyqt5' and 'no module named sip', despite having installed them on my pc.
Could you please tell me what I should do now?
Wow its incredible! Imagine if you could learn biggest map and higher resolution :D
Nice work man
Any suggestions for me? I detected an object but I want to register or align it automatically. Any thoughts on how I can do this?
Hey great job Ben nice videos !
If someone gets the tensorflow module errror it seems like the versions of tensorflow dont work together so try to change the line inside files
import tensorflow as tf to import tensorflow.compat.v1 as tf and it will work
@@pythonisa79 Check the tensorflow version that you are using on your anaconda virtual environment. Note that this project works with tensorflow 1 and python prior 3.7 for example python 3.6 , if you have also a python 3.8 version on your PC it will automatically download tensorflow 2 which messes with your project. Make sure to delete the tensorflow 2 version and to have only one Python Version in your PC I recommend 3.6 for this project
thanks so much
Hi, I have encountered a problem when trying to run python labelImg.py. I get the following error: "qt.qpa.plugin: Could not load the Qt platform plugin "windows" in "" even though it was found". To solve it, first I set the enviroment variable "QT_PLUGIN_PATH" to my "c:\ProgramData\Anaconda3\Library\plugins" folder, but nothing changed. So, I tried to copy the platforms directory both into the labelImg folder and in Python.exe folder but I got no results. How can I get rid of this error? Thanks in advance
p.s.: my operating system is Windows 10 64
Hey Ben, these videos are exactly what I was looking for. Are you going to do the training in colab or locally? Im trying to make it work with colab and google drive, but since I cannot just execute the model_main_tf2.py, I tried running it in an .ipynb within the object detection folder with
!python "//content//gdrive//My Drive//models-master//research//object_detection/model_main_tf2.py" \
--pipeline_config_path= {pipeline.config} \
--model_dir={training} \
--alsologtostderr \
--num_train_steps=1 \
--num_eval_steps=1
,which does not work as there are a lot of problems with the paths.
It would be great if you would show how to implement the training online and I cannot wait for the next part of the series!
model_main_tf2.py is used for running object detection using tensorflow 2 which just added support, this video series uses tensorflow 1. Speaking of the online training, my new video that came out today goes over this! Hope it helps
Great video bro
Hey Ben, i have a question. I am creating a model for my project to detect vehicles. You labelled them all as 'cloth_mask'. I wanna label them seperately like bus truck etc. At 17:35 in line 33 you write only cloth_mask. When i label them differently what should i write in line 33. Please help me! Thanks !
thanks for this tutorial, great work. you helped me a lot. i'll subscribe your channel bro
Please zoom your screen otherwise there is no point uploading this video.
Hey! What if I got my data set pictures from my own camera and not the web. Would I still need the xml to cvs step?
Haha nevermind. The xml files are created from the labl program!
Hey bro im getting an error after this command even if im in the correct directory :
pyrcc5 -o libs/resources.py resources.qrc
it says :
'C:/Users/Rahul' is not recognized as an internal or external command,
operable program or batch file.
please help
Hey Rahul, I had the same problem, it seems that it does not like white spaces in the paths. If your user folder contains a white space you need to change that. Beware that making this change is a very delicate process and could really mess up your system!
Great work bro!!
Hi. I am trying to follow the steps as shown by you in the video. Initially I was getting error with regards to "no module named object_detection" which got solved after I did pip install tensorflow-object-detection-api. After this I am getting error "DuplicateFlagError: The flag csv_input is defined twice. First from, second from. Description from first occurance:C:\Users\user\Desktop\Substation Detection\data\train_labels.csv". Can you please let me know what's the issue. I am totally a newbie to object detection and currently I have installed tensorflow version 2.4.1 on my windows 10 machine....
Nice video, suscribed
Thanks!
Wonderful video, The last commands you run to create record file. Can you run it in Google Colab?
got error in generate_tfrecord.py file
from object_detection.utils import dataset_util
ModuleNotFoundError: No module named 'object_detection'
I had the same error. In Anaconda I had to
pip install pandas
Which updated a package, then I got an error, "AttributeError: module 'tensorflow' has no attribute 'app'"
That was resolved by changing:
import tensorflow as tf
to
import tensorflow.compat.v1 as tf
@@nicholaswinger3163 did not work for me, where did you find compat v1?
Should all images in same format and same size?
i have got an error like 'UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 117: invalid start byte' while trying to get train tfrecord. does anyone know the solution ?
i found it. 'İ'(uppercase dotted I) is not supported. so filename shouldn't contain this latter
How to increase confidence level ?
good job guy
How did you setup video wallpaper and can you please give me a wallpaper link? It's look too cool to me. Thanks for the video.
It’s an app called “Wallpaper Engine”
@@LazyTech825 Thanks for your kind reply.
import pandas as pd
ModuleNotFoundError: No module named 'pandas'
i am getting this error
You need to install the pandas library. Pip3 install pandas
This is all over the place. Once you get your mistakes fixed, show us how it's supposed to look.
Hey guys! When I run python xml_to_csv.py, it doesn't show anything inside in test_label.csv(There are pictures in the test folder btw). Also, it is written only in column A in train_label.csv. How can i solve all these?
you need the xml in test folder.. not the images
@@rapscalliondave thanks, but i already solved the problem :)
@@ondereser1126 : hi Onder: how did you resolve this problem. i'm too getting the same. Thanks in advance!!
@@wasimakram-ze8oq i forget the solution :(
@@wasimakram-ze8oq if you (and perhaps others) are still looking for the answer, it's done in excel. you need to go to the data tab, next click on text to column. the in the window make sure it's on determined, then click next and deselect tab and select comma, click on complete and then it should have worked.
Hi Ben, do you know what could cause for train_record and test_record don't show up? The code ran fine without any errors and also my csv files have data in them.
Not sure what's happening but I checked the folder and now both are there but what's the delay for? I ran it again to make sure and the delay was still there
is there any alternative for that hand labeling?
the computer can't learn if you don't teach it
you can use makesense ai and roboflow where you can use assisted ai model to help user for fast up manual annotation.
Hey Ben, I am not able to resolve this error can you please help me out with this. Here's the error I keep getting:
NewRandomAccessFile failed to Create/Open: C:\Users\Avinash\Documents\Codes and DataSets\Deep Learning\tf_object_detection_custom_dataset\images/img (112).jpg
: The system cannot find the file specified.
; No such file or directory
create a folder named outputs
@Abhishek Suga i'm getting this error.I tried the command from the pycharm terminal ." ..... failed to Create/Open: images/images.jpg. My images folder has no file named "images.jpg" tho. Need help.
Hey Avinash & Ben, I am getting the same error while creating the tfrecords for test and train. It says -
File "C:\Users\DELL\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\python\lib\io\file_io.py", line 84, in _preread_check
compat.as_bytes(self.__name), 1024 * 512)
tensorflow.python.framework.errors_impl.NotFoundError: NewRandomAccessFile failed to Create/Open: images/18.jpg : The system cannot find the file specified.
; No such file or directory
Can you help me understand the error along with a solution?
Also don't know where to make the folder named outputs?(created at several place but still the error persists)
@@rajatshrivastav I am getting the exact same thing. Have you found the issue yet?
Here is how I solved this issue. There is a link to more help on this StackOverflow answer. stackoverflow.com/questions/64377948/tensorflow-notfounderror-when-trying-to-generate-tf-record-files-to-build-custo
Thanks for the tutorial, can i use cnn vgg16 from your tutorial?
Hey Ben can you share the research paper pls..
Hi! Cool tutorial. I do however get the following error when I want to label the second image:
Traceback (most recent call last):
File "labelImg.py", line 1355, in openNextImg
self.loadFile(filename)
File "labelImg.py", line 1096, in loadFile
self.showBoundingBoxFromAnnotationFile(filePath)
File "labelImg.py", line 1114, in showBoundingBoxFromAnnotationFile
filedir = filePath.split(basename)[0].split("/")[-2:-1][0]
IndexError: list index out of range
Any possible help?
how to measure custom object size
can I use something else instead of labeling the object, for example can I if the object is founded make it to give me a sound ?
Not that I know of natively, you could always just use some logic functions that makes something else happen when that object label appears
That command which you ran on test record i am getting an error its not workin on either pycharm or on either anaconda cmdd
me too.
Found any solution?
@@hashtable_yt I would urge you to follow this guy ruclips.net/channel/UCKDK_XjVaM_pfrUNrtnCS-Q Everything on his channel worked for me . If you want i can help you personally as well
I think what you are doing is so manually into the model training, do we have any solution to auto annotate the image?
you can use makesense ai and roboflow for this.
Hey Ben, greetings from one of your subscriber. Recently I was looking for some guidance for creating a new object ( rain clouds ) for the object detection ML module for a weather station project of mine. I need your help for this project it would be very kind of you to help me with this project. Thank you in advance.
how can i two class names in generate_tfrecord.py
I get the error below while running the commands in description:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 82: invalid start byte
Anyone knows the solution?
check the csv file by opening in notepad ++ and check if there is any garbage data.
kindly resize window, we cant see font.
Awesome but whats the website to download that bulk pics
just type mask on google
Hello :) Thanks for ur video ! But I need help from someone...
When I try to generate the tfrecord files I am facing an error...
My terminal gives me back this error: \Python\Python39\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 79, in _preread_check
self._read_buf = _pywrap_file_io.BufferedInputStream(
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 106: invalid continuation
byte
Can someone help me to resolve this error ? Thanks in advance, Jules !
can you minimize a lot so that it will create more challenge for us to see, i am going to rise brightness level so it will be more challenging.
good video and everything but everything is in one colomn
has anybody an idea
Can someone tell me if there's a way to verify tf records file?
There are scripts included in the object detection folder that allow you to do this. I have not used these before but the official GitHub probably has more information about them
@@LazyTech825 When I exported my graph, the variables folder came out to be empty. Can you please help?
Not a good tutorial to be honest. Many failures and plot holes (like why didnt the command work in doa prompt but works on PyCharm etc), making it hard to follow.
Thank you for the feedback, I’m working hard on making higher quality tutorials for the future!
so small, i can't see you screen, please make you desktop with the bigger font
Does anyone did the tutorial from EdjeElctronics hahaah?
can you make a vid for tf 2.0
Planning on making one
bro upload marks folder so we can test the training !
python is so cumbersome with its import errors all the time, I hate it...
9:46
bota mais empenho meu
horrible tutorial...you don't even have file extensions displayed in windows explorer....how can you code on windows
Do you have any suggestions on what I could do better for next time? I’m still pretty new to this. For the file extensions I just prefer them that way, I can just see the name of the file type in explorer.
I cannot run the final code
File "generate_tfrecord.py", line 24, in
from object_detection.utils import dataset_util
ModuleNotFoundError: No module named 'object_detection'
what should i do?
I have the same issue :(
Exactly I have, too. and that issue is that has in tensorflow version 2.x.x, but this video use 1.15.0. What should I do? I could't install 1.15.0 with pip.
I fixed that changed python version what matched with tensorflow version 1.15.0, so following next conda command.
> conda create -n object python=3.6
> conda activate object
> conda pip install tensorflow==1.15.0 ... (and what you need libs)
and you know that copy from models/research/object_detection(it is contain "github.com/tensorflow/models.git") folder to execute python folder
@@user-mk5xx1nh4j 고마워요.. 저도 한국인.. ㅋㅋㅋㅋ