How to Cut Out Object Segmentations from Detectron2

This video is a response to some questions we received regarding how to use the mask outputs of an instance segmentation neural net. In this one, Adam explains how to cut out mask objects found with Detectron2, then paste them on top of an image. Everything is done in Google Colab with common Python libraries.
Recommended Links
Code: gist.github.co...
🤩 We've launched our Patreon page! / immersivelimit
If you are able to, we'd love to have your support so we can keep more videos like this coming. Patrons will enable us to continue this work and will directly impact the quality and quantity of our RUclips videos.
Follow These For Updates +
Immersive Limit LinkedIn: / immersive-limit
Immersive Limit Facebook: / immersivelimit
Immersive Limit Twitter: / immersivelimit
Subscribe to the RUclips channel too!
Connect with Us! +
Adam's Twitter: / aktwelve
Adam's LinkedIn: / arkelly12
Kayla's Twitter: / kayladkelly
Kayla's LinkedIn: / kayladk
More Ways to Connect +
www.immersiveli...

[@danilogsch]

exactly what I needed

[@JavierSanchez-yc8qo]

Thank you sir! This is exactly what I needed to know and your walk through was awesome! You saved me hours if not days of me trying to figure this out on my own.

[@vineetbhat9731]

Thanks a lot ! I modified it as follows to get just the person instances and feed it to a face recognition algorithm.
person_instances = []
for one_mask, cls, conf in zip(pred_masks_np, pred_cls, pred_conf):
if names[int(cls)] == "person" and conf >= threshold:
# Get the true bounding box of the mask (not the same as the bbox prediction)
segmentation = np.where(one_mask == True)
x_min = int(np.min(segmentation[1]))
x_max = int(np.max(segmentation[1]))
y_min = int(np.min(segmentation[0]))
y_max = int(np.max(segmentation[0]))

cropped = Image.fromarray(nimg[y_min:y_max, x_min:x_max, :], mode='RGB') # Create a cropped image from just the portion of the image we want
mask = Image.fromarray((one_mask * 255).astype('uint8')) # Create a PIL image out of the mask
cropped_mask = mask.crop((x_min, y_min, x_max, y_max)).convert('RGB') # Crop the mask to match the cropped image
paste_position = (0, 0) # Choose a paste position
# Create a new alpha mask as large as the composite and paste the cropped mask
alpha_mask = Image.new('L', cropped.size, color = 0)
alpha_mask.paste(cropped_mask, paste_position)
composite = Image.composite(cropped, cropped_mask, alpha_mask) # Compose the foreground and background using the alpha mask
composite_array = np.array(composite)
person_instances.append(composite_array[:, :, ::-1])

[@ola0x98]

Thanks, you explained it better.

[@inyeung6481]

Is it possible to crop both the person and the horse out? Is the instances sorted by the objects that are most likely to be foreground?

[@jasonwheeler2986]

No need for hard numbers when setting paste_position; instead, set paste_position = (x_min, y_min)

[@aukehofman9378]

Very useful video! Can you also do this without alpha mask? So let's say, place them on top of each other to do further processing?

[@rajeshroyal5922]

How to crop that person separately

[@cypher5317]

how can i do the same thing with Yolov8 segmentation ?

[@bolarinwaoreoluwa5268]

Hiiii... Very good tutorial... But I'm having issues modifying this source code... I'll like to extract only the exact object found in the mask region and not put it anywhere...
And you mentioned something about this a little bit to the end of this video but I can't just seem to implement it 🥺🥺🥺☹️☹️

[@muddassarjilani4637]

great video, how can we find the area of each mask and select only the required one

[@ImmersiveLimit]

Area is just counting the pixels in the mask. You can use numpy for that.

[@fabioconti2379]

ti amo