Unet for Multi-class Segmentation
HTML-код
- Опубликовано: 5 окт 2024
- Here is the codebase and Blog on how to modify U-net for Multi-class semantic segmentation
Blog: towardsdatasci...
Github: github.com/soh...
For beginners in Medical Image Processing here is a list of data sets and papers:
sites.google.c...
Code for binary segmentation: github.com/soh...
Previous U-net videos: • Image Segmentation wit...
• How to Create a Succes...
#######################################################################
Debugging instructions:
1. The code github.com/soh... performs BINARY SEMANTIC SEGMENTATION only.
2. The code github.com/soh... performs MULTI-CLASS SEMANTIC SEGMENTATION. Please ensure ALL code is located in the same location for this to work. The model file is model_depth_3.py or model_depth_4.py.
Please see section on "How to avoid Errors in Code"
github.com/soh...
@Sohini Mam....
Thanks alot for this amazing codebase.
Thanks for your guidance.
It was really tough for me to implement this, but your constant help and replies made me through and I am able to run the UNet model as per your codebase.
For all learners...
I faced multiple problems:
1. Understand the codebase completely and arrange your dataset as explained in the video by Sohini Mam.
2. I have problems in starting Tensorboard, first learn the TensorBoard usage.
3. There might be problems in callbacks, you can try by removing the callbacks.
All these problems were solved with Sohini mam's help.
Once again mam,
thanks a lot and god bless you.
🙂
Most welcome. Thanks and stay tuned for new videos on unet++
Thank you so much
And glory be to almighty God who directed me to your channel
Glad you are here. Please do let me know if there are specific topics you would like me to cover.
@@AIwithSohini hello!
Would you do a tutorial explaining impact of using a deep CNN like VGG or ResNet as backbone for U_net either binary or multi class segmentation.
Thanks in advance for your support to the community!
@@helloyes4795 I have started a QA on my google site. Here is your answer, if you have follow up questions, please do ask here. sites.google.com/site/sohiniroychowdhury/ask-sohini-qa-from-ai-with-sohini?authuser=0
Thank you very much mam for sharing such a brief a useful information in a short time
Thanks for liking
Thank you so much madam.. Can you do a video on style transfer based on U-Net (CLIPstyler), that would be a great help
That’s a great idea for a video. Thanks
Thanks for valuable and informative content as usual, Sohini mam
Thanks so much for the support
@@AIwithSohini Kindly discuss the recent trends, repos, skills expected and impact of AI in biomedical engineering/applications as requested before by preparing a content in your free time/others.
@@venkatesanr9455 Hello there. I have now updated my website to reflect the datasets you mention here. sites.google.com/site/sohiniroychowdhury/data-and-research-samples?authuser=0
@@venkatesanr9455 I will add the link in description box as well. Hope this helps.
@@AIwithSohini Thanks a lot Sohini Mam and you are doing great job for the community.
Thank you for this video. Could you please explain me how to prepare the mask data of satellite for multi-class (annotations)
Thanks a lot for your question. V7 is a great platform and free to academic and researchers that allows for multi-class annotation. Check out my video here ruclips.net/video/TAYuwd6N18s/видео.html. Thanks and stay tuned!
Thanks for the video!
Thanks and stay tuned
very fantastic explanation and code, so from where i get the images and its GT, i didnot found in the link shown in the github repo. please provide the data which you used. Thank you so much
Thank you so much for joining the community. Stay tuned
Thank you very much for this informative video. Please let me know how to get the RGB masks for multi class segmentation. which tool to use for the annotation.
Hello there are thanks for your question. I have made several videos on image annotation so please feel free to check out my other annotation videos. For now I believe Labelme would fit your use case. Here is the video ruclips.net/video/0ktMyscdiC8/видео.html. Thanks and stay tuned.
@@AIwithSohini Thank you for the reply madam, I tried Labelme but I am getting the same color for most of the classes after I converted the JSON file of individual images to a png file (RGB mask). I followed the method you explained in ruclips.net/video/0ktMyscdiC8/видео.html at 10:54 minute. I have selected different colors for each class while annotating in the Labelme tool. Also, I need the corresponding RGB value for each class.
Please suggest to me the solution madam.
Madam,Is this code automatically generate colormap for mask or we have to prepare colormap with the dataset manually
Hello and thanks for the question. The code generates 3 dimensional images RGB planes if you read with PIL library. For more than 3 classes you will need to convert to .npy files where each plane dimension is each class. So no color map needed here. Hope this helps. Thanks for watching
Thank you madam
here you have used 3 classes so its easy to save 3 channeled image, what about 10 classes. No library is allowing me to save image more than 3 channels, i had to use np.savez() which saves file in .npz format, i am not sure if this is acceptable for Imagedatagenerator() and my unet model or not
Hello, yes you are right that you need to change format for more than 3 classes and image data generator can handle multi channel. Good luck
Which tool to use for generating the masks?
I typically use GIMP. Here is a tutorial to use GIMP ruclips.net/video/fx-qA9lvWPM/видео.html
Hello ! Thanks a lot for the informations and the tutorial !
I have a problem on a very specific Epoch and step_epoch (it's the 6th epoch and the 13th step), the error comes back each time I reach this moment...
The problem is : "InvalidArgumentError : Graph execution error :" and then refers to files.
The bottom message is :
"Node : 'gradient_tape/model/concatenate/concatOffset'
All dimensions except 3 must match. Input 1 has shape [3 32 32 512] and doesn't match input 0 with shape [3 64 64 512].
[[{{node gradient_tape/model/concatenate/ConcatOffset}}]] [Op:__interference_train_function_4405]"
I tried thew things but my guess is that there is something "simple" that I don't see... Any idea ?
Thanks for the help !
Hello there. It seems like an image sizing issue. So the size of image being returned at the end of a certain step is not matching what is expected. Two ways of debugging here. One resize your image to exactly the size shown in the example to work. Two, check out my other video here for added pointers ruclips.net/video/M4EvfyoVhb4/видео.html
@@AIwithSohini Hello again ! Thanks for your answer ! :)
I tried to resize all the images but still got the same issue... Maybe I misunderstood, should I resize the images in a specific Def or in the very beginning of the program ?
One other thing is that my accuracy seems good for a 4rth Epoch but the loss is very big (1418s 71s/step - loss: 10740230144.0000 - accuracy: 0.4291) and it's right before the program stop.. Is it still a matter of sizing issue to you ?
Thanks again for your time, I will try the second solution ! :)
Hello,
while runing the code with 100 epochs i've had this error "none in summary histogram for: conv 2D/kernel_0 [op:WriteHistogramSummary]"
Can anyone please help?
Knowing that I used the same dataset as you.
It seems your data is run out. So go for smaller batch size and lesser epochs like 50 and see.
I did but it gave me very bad images and wrong predictions
Hello
I did not watch the video yet
But i want to do 2 class segmentation with u-net
And i have the data set ready, what i want to segment is optic cup and optic disc , its the almost the same dataset
Will this video help me?
Thanks alot
Hello there. This video and this one ruclips.net/video/M4EvfyoVhb4/видео.htmlsi=_t5hxjMAf_m9iIX0 will be useful for you to walk through the code. Good luck and stay tuned
thanks i will watch it :)@@AIwithSohini
I am getting some error in cell 16 can you please help
Thank you so much for the informative video series. It helps a lot. But, I'm confused about one point. In this video, you try to segment 3 classes. On the other hand, In the code of multi class.ipynb, this line of code is written '' def SaveResultwImage(img_path,save_path,npyfile,target_size=img_size,flag_multi_class = True,num_class = 2):" I think num_ckass should be 3 instead of 2 ?! I've have a problem with dice_loss. It only segments background because the effect of it to the lost is more than %80. So, the final image is only red. For custom dataset, What do u suggest to arrange label images such as each channel shows different segmentation like background, cat, dog at each channel. 1 shows the segmentation object in the image at each layer. The other pixels show 0. Example. in the first channel of the image only dog area is 1,others will show the rest of the ii,age.
Other question, why do don;t want to split the validation set///?
Hello. You are right that in all there are 3 classes, but only 2 foreground classes. The third class (blue pixels in groundtruth) are the background only. Since I did not want to render the background pixels in SaveResultwImage, thats why I use the two foreground planes only. For custom data set, your assumption is correct. Each foreground category must be a plane (value =1) all other pixels are value=0. Similarly create a plane for cat and all other objects and the last plane must be background. The only downside is you cannot visualize the groundtruth as standard image planes then. For your cat/dog example see this github github.com/HZCTony/U-net-with-multiple-classification
In this case validation data set would have taken more time to process (since using data generator). But you should apply validation set in standard workflow. Good luck and do stay tuned!
Ma'am how do I stack 5 binary masks into 1 mask for multiclass segmentation? (Considering some of the classes overlap as well).
I had tried out your code but I required all the 5 classes in different colors for the single mask to be used.
Thanks in advance
Hello, you will have to create your own Step 1, where the masks are generated with one object per plane. The only difference will be you cannot visualize the groundtruth as an image (so load_img will not work) since you now will have 5 planes. Check out my blog towardsdatascience.com/a-machine-learning-engineers-tutorial-to-transfer-learning-for-multi-class-image-segmentation-b34818caec6b, where I mention how to extend to more classes. Good luck!
@@AIwithSohini thank you Ma'am
hello, how can i address the masks with overlapped pixels. I have 15 class 15 binary masks. But some pixels fall into multiple classes. Can i use binary cross entropy instead of categorical cross entropy and process masks seperately?
thank you
Hello. Check this out stats.stackexchange.com/questions/207794/what-loss-function-for-multi-class-multi-label-classification-tasks-in-neural-n
Ideally yes you should use binary cross entropy loss in case of multi class multi label problems as you mention. Good luck and stay tuned
@@AIwithSohini thank you
@@AIwithSohini Thank you, it works correctly. I have one more doubt regarding using binary cross entropy loss. In the second case, I have gaussian filtered mask. For example, i have 15 class/15 channel masks where the pixel is not 0 or 1 but range(0 to 1) since I use gaussian filter to the create the mask. I tried the binary cross entropy loss, it works well. But i dont understand how it works, since its not one hot encoded. Can you please tell me the theory behind it.
@@rs9130 Hello. Here is a good explanation for your query stats.stackexchange.com/questions/370179/why-binary-crossentropy-can-be-used-as-the-loss-function-in-autoencoders. Hope this helps and stay tuned.
hello ma'am. Can you please tell me wich image we are suppose to have in th GT folder?
Hi. So the 3 colored images corresponding to the foreground and background should be in folder GT. GT means ground truth.
@@AIwithSohini Thank you ma'am
how do we classify into progressive and non progressive wounds ?
Once you have data in the proper format, you need a simple classifier like Resnet or InceptionV3 etc.
Please be with us so that we can atleast take ur help
Thanks a lot for your support. Please do share any issues and I will try my best to help.
Thanks for the tutorial mam and also there is one doubt like how to extract the predicted classes for unet , for example like in detectron2 print(outputs["instances"].pred_classes) can give you the predicted classes.
Hello there. So unet produces images as output with region maps corresponding to the predicted class labels. So if problem has 3 classes, the output will be 3-d image where each dimension reflects foreground region labs vs background. So no need to query pred classes here. Hope this helps.
@@AIwithSohini Thanks for your kind reply , actually there is a user requirement to store predicted classes in database so i switched to mask rcnn using detectron2 because in unet i dont know to extract the predicted classes for storing in database.
good evening madam , when I try to run the code, i am getting the following error, please help in this regard,
ModuleNotFoundError Traceback (most recent call last)
in
5 import skimage.transform as trans
6 import numpy as np
----> 7 import tensorflow as tf
8 from tf.keras.models import *
9 from tf.keras.layers import *
ModuleNotFoundError: No module named 'tensorflow'
You need to install tensorflow first. So if you are using colab include the following line before this cell
! pip install tensorflow
@@AIwithSohini i am using anaconda , already used pip install tensor flow ,, but it is not installing
For anaconda pip is not recommended. Within anaconda you should do a condo install tensorflow
Also when done installing check the installation by typing
Python
Import tensorflow as tf
If you get an error that means tensorflow is not installed. In that case you need to use colab for now. There may be an issue with your system in that case. Good luck.
Good Morning , While executing Unet_binary _db1 file , following error I am getting in Cell [26] in jupyter notebook as TypeError: Invalid shape (256, 256, 1) for image data
and If I ignore above cell ,
CELL [30] in mode.fit following error :
Errors may have originated from an input operation.
Input Source operations connected to node dice_coef_loss/mul:
IteratorGetNext (defined at :3).
and also how input data is arranged .... in the Program .... can please help me to solve the above errors.
Hello for binary segmentation please use the other repo from my previous video ruclips.net/video/hK-RHbU3rng/видео.html
@@AIwithSohini ok , Thank you
I used the same code (binary segmentation repo mentioned above ) for retina exudate detection , But not give good results , Can you help me in complete the task. Can I share my code and dataset?
you are very cute