I have been watching your entire series on Mask RCNN this afternoon and learned so much from your video, Sreeni! Super grateful for your wonderful work here! I'll be following your code tomorrow step by step :)
I'm sure the is more elegant ways of referencing paths than the crude form I employed here, but it fixed my problems. Running the code in full, no problem was detected. It just wasn't generating the weights. Going line by line or groups of lines by group of lines, following the video, I corrected some things that were wrong, printing things to see were the paths were pointing to. My changes are crude, so you will have to manually change the paths, as your path will not be like mine. All this on the coco style In my file 'F:\joaopedro\downloads\python_for_microscopists-master\286-Object detection using mask RCNN - end to end\286-marbles_maskrcnn_coco_style_labels.py', I had to make the following changes to make it work: line before change: dataset_train.load_data('marble_dataset/train/labels/marbles_two_class_coco_json_format.json', 'marble_dataset/train') line after change: dataset_train.load_data('F:/joaopedro/downloads\python_for_microscopists-master/286-Object detection using mask RCNN - end to end/marble_dataset/train/labels/marbles_two_class_coco_json_format.json', 'F:/joaopedro/downloads\python_for_microscopists-master/286-Object detection using mask RCNN - end to end/marble_dataset/train') similarly for the 'COCO_WEIGHTS_PATH = '... line, the line after the change became: COCO_WEIGHTS_PATH = 'F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/coco_weights/mask_rcnn_coco.h5' Also added lines to save the weights (I'm not sure how Sreenivas was able to save it without those lines...), below the last line of the training section of the code, 'model.train(dataset_train, dataset_train, learning_rate=config.LEARNING_RATE, epochs=3, layers='heads')': # Save the trained model weights ##print(os.path.join(DEFAULT_LOGS_DIR, "mask_rcnn_trained_weights.h5")) model_path = os.path.join(DEFAULT_LOGS_DIR, "mask_rcnn_trained_weights.h5") model.keras_model.save_weights(model_path) print(f"Trained weights saved to {model_path}") -==-=- Now on the inference (which I put in a separate file), I've made these changes: line before change: model = MaskRCNN(mode='inference', model_dir='logs', config=cfg) line after the change: model = MaskRCNN(mode='inference', model_dir='F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/logs', config=cfg) line before change: model.load_weights('logs/mask_rcnn_trained_weights.h5', by_name=True) line after change: model.load_weights('F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/logs/mask_rcnn_trained_weights.h5', by_name=True) -==-=-=- Now on the section 'Show detected objects in color and all others in B&W' I changed 1 line (below '#line changed 1')and added another (below '#line added 2') to specify an output folder, otherwise it was outputting in my C: in my user. The changed code is below: ############################################## #Show detected objects in color and all others in B&W def color_splash(img, mask): """Apply color splash effect. image: RGB image [height, width, 3] mask: instance segmentation mask [height, width, instance count] Returns result image. """ # Make a grayscale copy of the image. The grayscale copy still # has 3 RGB channels, though. gray = skimage.color.gray2rgb(skimage.color.rgb2gray(img)) * 255 # Copy color pixels from the original color image where mask is set if mask.shape[-1] > 0: # We're treating all instances as one, so collapse the mask into one layer mask = (np.sum(mask, -1, keepdims=True) >= 1) splash = np.where(mask, img, gray).astype(np.uint8) else: splash = gray.astype(np.uint8) return splash import skimage def detect_and_color_splash(model, image_path=None, video_path=None): assert image_path or video_path # line added 2 output_dir = 'F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/viz/' # Image or video? if image_path: # Run model detection and generate the color splash effect #print("Running on {}".format(img)) # Read image img = skimage.io.imread(image_path) # Detect objects r = model.detect([img], verbose=1)[0] # Color splash splash = color_splash(img, r['masks'])
What could be wrong if the training stops at the message "Epoch 1/30"? There is no loss information or anything loading like in your case. This occurs after model.train has been run. The process where it gets stuck is: get_next_batch() -> get() -> get() -> wait() -> wait() -> wait(), so I believe it doesn't get the required responses just yet or it's looping.
Hello sir... Wonderful lecture... I tried the same... Couldnt see where the trained weights are stored... So they are not loaded during the 'inference' and shows Train mAP value as 0.000. Please address the issue...
Hi Sreeni i really like your videos and thanks a lot. But I have a scenario here if you want to detect whether a person is turning right or left or back what is the best approach you suggest?
hi, thank you for your helpful video; i jae a question , hope you can give me a guide. i tried to apply this code, first it was the problem of tensorflow 1x version not being abled in colab, then i made a virtual env using lower python version and using tensorflow 1x , it appeared with other errors such as skimage and code being old! i wish you have sth for me to do it,,, the other help that i want is about models detecting polygon with polygon annotations; i want to detect polygon lables exactly as polygons, not as bounding boxes (when the model detect my thing, it shows it as a polygon label not a bounding box label), do you have any clue for me? or do you know on which models should i work? i'll be so thankful anyway💕💕
Hi Sreeni sir, thank you for sharing mask rcnn workflow, I was using it, I am struggling to deploy it using redis as this model has custom objects and class, could you please help
Your video is totally informative, totally love it. However, I am having a small problem with the model= MaskRCNN line, it keeps telling me that type object "Config" has no attribute "image_shape". Can you please tell me what has gone wrong and what is the best way to fix it? Thanks a lot and I'm looking forward to your reply!!
your presentation is nice and pretty good. would please show how to classify image(Remote sensing image such as sentinel 1,2 or land sat) by CNN in jupyter notebook
Hi, Excellent explanation step by step to understand MaskRCNN. Pls, let us know how we can calculate the accuracy of generated mask instead of the mAP of object detection?
Thanks for video. I have trained MaskRNN model with my custom dataset. Could you tell me about how to measure large image where few objects span to another patch. In that case, objects metrics are not accurate.
Hello, I need to use a model to do semantic segmentation of human organs from CT scans (3D). I have an important question, do you think I should test both Mask RCNN and Unets? I am well documented on U nets since it seems to be used a lot for these medical tasks. I used Mask RCNN with Detectron2 in the past but have no clue how to code one from scratch. Do you think Mask RCNN is not well suited when you have only few data?
Where can I find a description of the COCO file? And where do I find the requirements.txt information? Please also provide information on various auxiliary programs and versions that need to be installed.
Hi Sreeni, im following the tutorial and doing some modifications to adjust it to my needs, but im not able to use my system's GPU, do you have any recomendation? i have cuda 11.7 installed but i dont know if i need to do another extra step. Thanks for your content!
On Medium there is a good guide on how to install 'Install CUDA and CUDNN on Windows & Linux'. CUDNN had to be installed manually, as there is no installer for older versions.
You need to look up the compatibilities between the programs. My GPU is RTX 2060 Super. cdnn used by me: cuDNN v7.6.5 (November 5th, 2019), for CUDA 10.1 (had to remove the developer link... but in the 'cuDNN Archive' you can find it) cuda used: 10.1 python used: 3.7.11 tensorflow-2.2.0 Here you can find cuda and cdnn compatibility, search on google, as I cannot put links here: tensorflow tested_build_configurations Another resource that gives a little bit of context, search on google: stack exchange Dlerror: cudnn64_7.dll not found
I am constantly getting the error "You are using the default load_mask(), maybe you need to define your own one." But I already did custom it . Someone please help
I used this to train my custom dataset on Vehicle Damage Segmentation but it's results are very poor. I trained it for 100 epochs but still it cannot work correctly on training images. Please help me to solve this. Thank you in advance
hey, just a question i have an issue with training maskrcnn on my custom dataset after reaching 100 epochs the training restart from the beginning any idea how to resolve this i followed the exact same tutorial. would really appreciate it im stuck any solution please
Hi, Thanks for your video! I get a maP=0.0 while I was implementing this video. Do you know what may be the different? I would appreciate if you can share your mask_rcnn_marble_cfg_0003.h5 file to Github so that I can try to run and see if maP=0.0 to figure out my problem. Thanks!
Hello bro when I run the map I always get load_image_gt() got an unexpected keyword argument 'use_mini_mask' Can you tell me whether you also have got this type of error.And can you suggest me how to solve this
Hello bro I always get this error load_image_gt() got an unexpected keyword argument 'use_mini_mask' When I run the map score from inference part Can you help me how I can solve this
Similar problem here, on the coco style code. Did you find a solution? In my case not even the 'logs' folder is being created automatically, as I think is supposed to. Even puting an exact and direct path for the variable did not work.
Dear Sir, I am using this code for my own image object detection(the image is .png format). However, I encountered a issue which confused me a lot. it is at detection stage, where "detected = model.detect([marbles_img])", and I got errors saying "mold_image return images.astype(np.float32) - config.MEAN_PIXEL ValueError: operands could not be broadcast together with shapes (1024,1024,4) (3,)" it seems that my image has 4 channels so I checked that png indeed has 4 channels. Therefore, I changed all my png files to jpg files and re ran the whole training process. However, when the code executes at the same location, it still reports the same errors. Do you have any idea what caused this error? I appreciate your kind help. Thank you!
ValueError: Error when checking input: expected input_image_meta to have shape (16,) but got array with shape (15,) Are anyguy have this ploblem ? And how can i fix it
I have been watching your entire series on Mask RCNN this afternoon and learned so much from your video, Sreeni! Super grateful for your wonderful work here! I'll be following your code tomorrow step by step :)
Hey!
Your videos are always long enough, informative and beyond what anybody does. Very polite and understanding delivery. Kudos!
I appreciate that!
Your videos on Mask RCNN is great thank you!
I'm sure the is more elegant ways of referencing paths than the crude form I employed here, but it fixed my problems. Running the code in full, no problem was detected. It just wasn't generating the weights. Going line by line or groups of lines by group of lines, following the video, I corrected some things that were wrong, printing things to see were the paths were pointing to. My changes are crude, so you will have to manually change the paths, as your path will not be like mine. All this on the coco style
In my file 'F:\joaopedro\downloads\python_for_microscopists-master\286-Object detection using mask RCNN - end to end\286-marbles_maskrcnn_coco_style_labels.py', I had to make the following changes to make it work:
line before change:
dataset_train.load_data('marble_dataset/train/labels/marbles_two_class_coco_json_format.json', 'marble_dataset/train')
line after change:
dataset_train.load_data('F:/joaopedro/downloads\python_for_microscopists-master/286-Object detection using mask RCNN - end to end/marble_dataset/train/labels/marbles_two_class_coco_json_format.json', 'F:/joaopedro/downloads\python_for_microscopists-master/286-Object detection using mask RCNN - end to end/marble_dataset/train')
similarly for the 'COCO_WEIGHTS_PATH = '... line, the line after the change became:
COCO_WEIGHTS_PATH = 'F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/coco_weights/mask_rcnn_coco.h5'
Also added lines to save the weights (I'm not sure how Sreenivas was able to save it without those lines...), below the last line of the training section of the code, 'model.train(dataset_train, dataset_train, learning_rate=config.LEARNING_RATE, epochs=3, layers='heads')':
# Save the trained model weights
##print(os.path.join(DEFAULT_LOGS_DIR, "mask_rcnn_trained_weights.h5"))
model_path = os.path.join(DEFAULT_LOGS_DIR, "mask_rcnn_trained_weights.h5")
model.keras_model.save_weights(model_path)
print(f"Trained weights saved to {model_path}")
-==-=-
Now on the inference (which I put in a separate file), I've made these changes:
line before change:
model = MaskRCNN(mode='inference', model_dir='logs', config=cfg)
line after the change:
model = MaskRCNN(mode='inference', model_dir='F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/logs', config=cfg)
line before change:
model.load_weights('logs/mask_rcnn_trained_weights.h5', by_name=True)
line after change:
model.load_weights('F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/logs/mask_rcnn_trained_weights.h5', by_name=True)
-==-=-=-
Now on the section 'Show detected objects in color and all others in B&W' I changed 1 line (below '#line changed 1')and added another (below '#line added 2') to specify an output folder, otherwise it was outputting in my C: in my user. The changed code is below:
##############################################
#Show detected objects in color and all others in B&W
def color_splash(img, mask):
"""Apply color splash effect.
image: RGB image [height, width, 3]
mask: instance segmentation mask [height, width, instance count]
Returns result image.
"""
# Make a grayscale copy of the image. The grayscale copy still
# has 3 RGB channels, though.
gray = skimage.color.gray2rgb(skimage.color.rgb2gray(img)) * 255
# Copy color pixels from the original color image where mask is set
if mask.shape[-1] > 0:
# We're treating all instances as one, so collapse the mask into one layer
mask = (np.sum(mask, -1, keepdims=True) >= 1)
splash = np.where(mask, img, gray).astype(np.uint8)
else:
splash = gray.astype(np.uint8)
return splash
import skimage
def detect_and_color_splash(model, image_path=None, video_path=None):
assert image_path or video_path
# line added 2
output_dir = 'F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/viz/'
# Image or video?
if image_path:
# Run model detection and generate the color splash effect
#print("Running on {}".format(img))
# Read image
img = skimage.io.imread(image_path)
# Detect objects
r = model.detect([img], verbose=1)[0]
# Color splash
splash = color_splash(img, r['masks'])
#line changed 1
file_name = output_dir + "splash_{:%Y%m%dT%H%M%S}.png".format(datetime.datetime.now())
skimage.io.imsave(file_name, splash)
elif video_path:
import cv2
# Video capture
vcapture = cv2.VideoCapture(video_path)
width = int(vcapture.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(vcapture.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = vcapture.get(cv2.CAP_PROP_FPS)
# Define codec and create video writer
file_name = "splash_{:%Y%m%dT%H%M%S}.avi".format(datetime.datetime.now())
vwriter = cv2.VideoWriter(file_name,
cv2.VideoWriter_fourcc(*'MJPG'),
fps, (width, height))
count = 0
success = True
while success:
print("frame: ", count)
# Read next image
success, img = vcapture.read()
if success:
# OpenCV returns images as BGR, convert to RGB
img = img[..., ::-1]
# Detect objects
r = model.detect([img], verbose=0)[0]
# Color splash
splash = color_splash(img, r['masks'])
# RGB -> BGR to save image to video
splash = splash[..., ::-1]
# Add image to video writer
vwriter.write(splash)
count += 1
vwriter.release()
print("Saved to ", file_name)
detect_and_color_splash(model, image_path="F:/joaopedro/downloads/python_for_microscopists-master/286-Object detection using mask RCNN - end to end/marble_dataset/val/test4.jpg")
######################################################
=-=-=-=-
thank you so much for this. I've been quite confused on how the weights were getting saved
Very useful presentation. Nice to see an end-to-end example!
Glad it was helpful!
Keep on learning from your great lectures.
Thank you for the presentation, makes the whole thing so much clearer.
Glad it was helpful!
Fantastic video! I'm going to run through the code in detail next. Many thanks.
highly informative presentation and explanation!!
What could be wrong if the training stops at the message "Epoch 1/30"? There is no loss information or anything loading like in your case. This occurs after model.train has been run.
The process where it gets stuck is:
get_next_batch() -> get() -> get() -> wait() -> wait() -> wait(), so I believe it doesn't get the required responses just yet or it's looping.
Hello sir... Wonderful lecture... I tried the same... Couldnt see where the trained weights are stored... So they are not loaded during the 'inference' and shows Train mAP value as 0.000. Please address the issue...
Hi Sreeni i really like your videos and thanks a lot. But I have a scenario here if you want to detect whether a person is turning right or left or back what is the best approach you suggest?
hi, thank you for your helpful video; i jae a question , hope you can give me a guide.
i tried to apply this code, first it was the problem of tensorflow 1x version not being abled in colab, then i made a virtual env using lower python version and using tensorflow 1x , it appeared with other errors such as skimage and code being old!
i wish you have sth for me to do it,,,
the other help that i want is about models detecting polygon with polygon annotations;
i want to detect polygon lables exactly as polygons, not as bounding boxes (when the model detect my thing, it shows it as a polygon label not a bounding box label), do you have any clue for me? or do you know on which models should i work? i'll be so thankful anyway💕💕
Hi Sreeni sir, thank you for sharing mask rcnn workflow, I was using it, I am struggling to deploy it using redis as this model has custom objects and class, could you please help
nice tutorial, thank you.what's about the confusion matrix
Your video is totally informative, totally love it. However, I am having a small problem with the model= MaskRCNN line, it keeps telling me that type object "Config" has no attribute "image_shape". Can you please tell me what has gone wrong and what is the best way to fix it? Thanks a lot and I'm looking forward to your reply!!
your presentation is nice and pretty good. would please show how to classify image(Remote sensing image such as sentinel 1,2 or land sat) by CNN in jupyter notebook
Hi,
Excellent explanation step by step to understand MaskRCNN. Pls, let us know how we can calculate the accuracy of generated mask instead of the mAP of object detection?
Thanks for video. I have trained MaskRNN model with my custom dataset. Could you tell me about how to measure large image where few objects span to another patch. In that case, objects metrics are not accurate.
Hello, I need to use a model to do semantic segmentation of human organs from CT scans (3D). I have an important question, do you think I should test both Mask RCNN and Unets? I am well documented on U nets since it seems to be used a lot for these medical tasks. I used Mask RCNN with Detectron2 in the past but have no clue how to code one from scratch. Do you think Mask RCNN is not well suited when you have only few data?
I have learned a lot from your videos. Could you please upload any video on Cell Nuclei segmentation using Mask R-CNN?
Hi, thanks for the video. Can we get coco format output for prediction?
Hi Sreeni, Is the mask r-cnn model in the Tensorflow model zoo the same as Matterport?
Great course! What is the best way for small object detection?
This works for small and large objects. Although, if your small objects are roundish, give StarDist a try.
how can I update or use my own weights (.h5) once I have trained my own models? from where and how can I implement new own ones? Thanks!
It is not saving the weights after training... no folder 'logs' is being created... Trying to find a solution, no apparent error message...
The coco style code.
I'm going to try running the VGG style to see.
I solved it. There is a comment explaining quickly what I did.
I am getting the train mAP as 0.000. Could you please let me know what could be the reason behind this?
amazing session, thnx a lot
can't find the code on Github?
sir please make a video about nnunet with costum dataset on colab.
Where can I find a description of the COCO file? And where do I find the requirements.txt information? Please also provide information on various auxiliary programs and versions that need to be installed.
Please make a video for object detection using Faster R-CNN from annotation to end.
Hi Sreeni, im following the tutorial and doing some modifications to adjust it to my needs, but im not able to use my system's GPU, do you have any recomendation? i have cuda 11.7 installed but i dont know if i need to do another extra step. Thanks for your content!
For that you have to upgrade the TF version but it has some dependencies on skimage so you cannot do it. Have you trained on CPU?
On Medium there is a good guide on how to install 'Install CUDA and CUDNN on Windows & Linux'. CUDNN had to be installed manually, as there is no installer for older versions.
You need to look up the compatibilities between the programs. My GPU is RTX 2060 Super.
cdnn used by me: cuDNN v7.6.5 (November 5th, 2019), for CUDA 10.1 (had to remove the developer link... but in the 'cuDNN Archive' you can find it)
cuda used: 10.1
python used: 3.7.11
tensorflow-2.2.0
Here you can find cuda and cdnn compatibility, search on google, as I cannot put links here: tensorflow tested_build_configurations
Another resource that gives a little bit of context, search on google: stack exchange Dlerror: cudnn64_7.dll not found
Can you make a video to do the whole process in cloud I mean google collab
I am constantly getting the error "You are using the default load_mask(), maybe you need to define your own one." But I already did custom it . Someone please help
I used this to train my custom dataset on Vehicle Damage Segmentation but it's results are very poor. I trained it for 100 epochs but still it cannot work correctly on training images. Please help me to solve this. Thank you in advance
Sir please give some content for point cloud data processing by deep learning methods
hey, just a question i have an issue with training maskrcnn on my custom dataset after reaching 100 epochs the training restart from the beginning any idea how to resolve this i followed the exact same tutorial. would really appreciate it im stuck any solution please
Sir can you tell why are you loading the coco weights we have,nt trained the model yet and also coc o weights are not included in mask repo folder
How much data is required for such kind of transfer learning? Is 40 - 50 images enough?
Usually 50 objects per class is a good starting point. The more the merrier!
Hi, Thanks for your video! I get a maP=0.0 while I was implementing this video. Do you know what may be the different? I would appreciate if you can share your mask_rcnn_marble_cfg_0003.h5 file to Github so that I can try to run and see if maP=0.0 to figure out my problem. Thanks!
Hello bro when I run the map I always get
load_image_gt() got an unexpected keyword argument 'use_mini_mask'
Can you tell me whether you also have got this type of error.And can you suggest me how to solve this
Hello, I've an error when I try to import the model. The error says that there is no module named 'keras.engine'. How can i fix it?
Hello bro I always get this error
load_image_gt() got an unexpected keyword argument 'use_mini_mask'
When I run the map score from inference part
Can you help me how I can solve this
Does anyone knows why I cannot find the logs of the trained model? I created a logs folder but it is empty, what am I doing wrong? Please help
Similar problem here, on the coco style code. Did you find a solution? In my case not even the 'logs' folder is being created automatically, as I think is supposed to. Even puting an exact and direct path for the variable did not work.
Sir how we will calculate accuracy here..?
Dear Sir, I am using this code for my own image object detection(the image is .png format). However, I encountered a issue which confused me a lot. it is at detection stage, where "detected = model.detect([marbles_img])", and I got errors saying
"mold_image
return images.astype(np.float32) - config.MEAN_PIXEL
ValueError: operands could not be broadcast together with shapes (1024,1024,4) (3,)"
it seems that my image has 4 channels so I checked that png indeed has 4 channels. Therefore, I changed all my png files to jpg files and re ran the whole training process. However, when the code executes at the same location, it still reports the same errors.
Do you have any idea what caused this error? I appreciate your kind help. Thank you!
How to train with binary mask?
ValueError: Error when checking input: expected input_image_meta to have shape (16,) but got array with shape (15,) Are anyguy have this ploblem ? And how can i fix it
It's late, but I'll post it, maybe it will help someone, perhaps the number of specified classes is different
NUM_CLASSES = 1 + count_your_classes
Does RCNN work like Yolo?
The actual way it works is different but if object segmentation is what you need then both Mask R-CNN and Yolo will do the job.
i am using google colab and got an error AttributeError: module 'tensorflow.python.framework.ops' has no attribute '_TensorLike'
please help me
I'm still waiting for your reply
I think that need use oldest version tensorflow.