Last Updated on 05/05/2026 by Eran Feit
Object detection can tell you where an object is, but it falls short when you need the exact pixel boundaries. If you are struggling to move beyond basic bounding boxes, this Mask R-CNN Python tutorial for instance segmentation is exactly what you need. In this guide, we will bridge the gap between theoretical computer vision and practical implementation. You will learn how to configure a pre-trained model, properly process image tensors, and extract highly accurate, pixel-perfect masks for distinct objects in your images. Let’s dive into the code and mechanics behind state-of-the-art image segmentation.
A good mask rcnn tutorial should also explain why instance segmentation is different from the other common computer vision tasks.
From a Python workflow perspective, Mask R-CNN is especially approachable because you can run it directly from widely used libraries.
In real-world use, instance segmentation is often about presentation and usability as much as accuracy.
What is Instance Segmentation vs. Semantic Segmentation? Before diving into the code, it is crucial to understand where Mask R-CNN sits in the computer vision hierarchy. While both techniques involve assigning labels to pixels, the level of granularity and the way they handle individual objects are fundamentally different.
Semantic Segmentation: The “Categorical” Approach Semantic segmentation treats all pixels of the same class as a single entity. If an image contains five different cars, a semantic segmentation model will highlight every pixel belonging to “car” in the same color. It does not distinguish between individual vehicles; it simply maps out the “territory” of that class within the frame. This is commonly used in medical imaging (identifying a tumorous region) or land-cover mapping.
Instance Segmentation: The “Individual” approach Instance segmentation—the core of this Mask R-CNN Python tutorial —goes a step further. It combines the strengths of Object Detection and Semantic Segmentation . It doesn’t just identify that “cars” are present; it identifies “Car 1,” “Car 2,” and “Car 3” as distinct individuals. Each instance is given its own unique mask and bounding box.
The Key Differences at a Glance:
Feature Semantic Segmentation Instance Segmentation (Mask R-CNN) Object Distinction Groups all objects of a class together. Detects and separates individual objects. Primary Goal Understand the scene layout. Understand the individual components. Complexity High (Pixel-level classification). Very High (Detection + Pixel-level Masking). Common Use Case Autonomous driving (Road vs. Sidewalk). Robotics (Picking up a specific tool).
Why Mask R-CNN is the Standard Mask R-CNN (Region-based Convolutional Neural Network) is widely considered the gold standard for instance segmentation because it effectively adds a “mask branch” to the Faster R-CNN architecture. While the model detects the bounding box (localization) and the class (classification), it simultaneously predicts a binary mask for each detected object. This multi-task learning approach ensures that the model isn’t just guessing where an object is, but is understanding its precise geometric shape down to the pixel level.
A friendly mask rcnn tutorial for instance segmentation in Python A mask rcnn tutorial is most useful when it keeps the goal simple and concrete.
The main target of this kind of tutorial is building a clean inference pipeline.
At a high level, the pipeline has three layers.
The visualization step is where instance segmentation becomes immediately useful.
Mask RCNN Tutorial Running a Complete Mask R-CNN Pipeline in Python This tutorial code is designed to help you go from a raw image to a fully visualized instance segmentation result using a pretrained Mask R-CNN model.
At a high level, the code loads a Mask R-CNN model that was trained on the COCO dataset and prepares it for evaluation.
A key goal of the code is turning raw model output into something visually meaningful.
The final stage of the pipeline focuses on usability and repeatability.
Link to the video tutorial here
Code for the tutorial here or here
Master Computer Vision
Follow my latest tutorials and AI insights on my
Personal Blog .
Beginner Complete CV Bootcamp
Foundation using PyTorch & TensorFlow.
Get Started → Interactive Deep Learning with PyTorch
Hands-on practice in an interactive environment.
Start Learning → Advanced Modern CV: GPT & OpenCV4
Vision GPT and production-ready models.
Go Advanced →
Mask R-CNN instance segmentation overview Mask R-CNN is one of the most practical ways to get high-quality instance segmentation without training a model from scratch.
The goal is simple.
This approach is perfect for prototyping.
By the end of the code walkthrough, you will have a clean end-to-end script.
Setting up Your Python Environment for Mask R-CNN This part is all about creating a stable Python environment that matches the exact versions used in the tutorial.
You are also installing OpenCV for visualization and saving results.
### Create a fresh Conda environment named Pytorch251 with Python 3.12 so the tutorial dependencies stay isolated. conda create -- name Pytorch251 python = 3.12 ### Activate the new environment so all installations apply to it. conda activate Pytorch251 ### Check your CUDA compiler version to confirm CUDA support on your machine. nvcc -- version ### Install PyTorch, TorchVision, and TorchAudio with CUDA 12.4 support so inference can run on GPU when available. # Cuda 12.4 conda install pytorch == 2.5 . 1 torchvision == 0.20 . 1 torchaudio == 2.5 . 1 pytorch - cuda = 12.4 - c pytorch - c nvidia ### Install OpenCV for image loading, drawing masks, showing windows, and saving results. # install pip install opencv - python == 4.10 . 0 . 84 Loading the Pre-Trained Mask R-CNN Model and ResNet Backbone This code block is the setup step that makes the whole Mask R-CNN pipeline possible.
The first goal here is importing the right libraries.MaskRCNN_ResNet50_FPN_Weights enum, you make sure the correct pretrained weights and configuration are loaded safely and consistently.
Next, the code selects the default pretrained weights and creates the Mask R-CNN model with those weights.
Finally, the code puts the model into evaluation mode and moves it to the GPU if possible.model.eval() is important because it disables training behavior and ensures the model runs deterministically for inference.torch.cuda.is_available() lets the same script work on both GPU and CPU machines while automatically taking advantage of faster CUDA inference when it exists.
### Import PyTorch for tensors, device handling, and running inference. import torch ### Import TorchVision for pretrained detection and segmentation models. import torchvision ### Import the official Mask R-CNN weights enum so we can load pretrained weights safely. from torchvision . models . detection import MaskRCNN_ResNet50_FPN_Weights ### Choose the default pretrained weights for Mask R-CNN ResNet50-FPN. weights = MaskRCNN_ResNet50_FPN_Weights . DEFAULT ### Load the pretrained Mask R-CNN model from TorchVision using the selected weights. model = torchvision . models . detection . maskrcnn_resnet50_fpn ( weights = weights ) ### Switch the model to evaluation mode so it behaves correctly for inference. model . eval () # Set the model to evaluation mode ### If CUDA is available, move the model to GPU for faster predictions. if torch . cuda . is_available (): model = model . to ( ' cuda ' ) # Move the model to GPU if available The environment is now ready and reproducible.
Why we do this: Before feeding our image into the Mask R-CNN architecture, we must convert it into a tensor and scale the pixel values. Pre-trained Mask R-CNN models expect images normalized to a specific mean and standard deviation based on the COCO dataset. Failing to properly normalize your inputs will result in poor feature extraction by the neural network’s backbone, leading to dropped detections or wildly inaccurate segmentation masks.
Image Preprocessing: Tensors and COCO Dataset Normalization This part prepares the label map so your results have readable class names.
You also build the core inference function that takes an image and returns masks, bounding boxes, and class labels.
### Define the full COCO instance category list so predicted label IDs can be converted into readable class names. COCO_INSTLANCE_CATEGORY_NAMES = [ ' __background__ ' , ' person ' , ' bicycle ' , ' car ' , ' motorcycle ' , ' airplane ' , ' bus ' , ' train ' , ' truck ' , ' boat ' , ' traffic light ' , ' fire hydrant ' , ' N/A ' , ' stop sign ' , ' parking meter ' , ' bench ' , ' bird ' , ' cat ' , ' dog ' , ' horse ' , ' sheep ' , ' cow ' , ' elephant ' , ' bear ' , ' zebra ' , ' giraffe ' , ' N/A ' , ' backpack ' , ' umbrella ' , ' N/A ' , ' N/A ' , ' handbag ' , ' tie ' , ' suitcase ' , ' frisbee ' , ' skis ' , ' snowboard ' , ' sports ball ' , ' kite ' , ' baseball bat ' , ' baseball glove ' , ' skateboard ' , ' surfboard ' , ' tennis racket ' , ' bottle ' , ' N/A ' , ' wine glass ' , ' cup ' , ' fork ' , ' knife ' , ' spoon ' , ' bowl ' , ' banana ' , ' apple ' , ' sandwich ' , ' orange ' , ' broccoli ' , ' carrot ' , ' hot dog ' , ' pizza ' , ' donut ' , ' cake ' , ' chair ' , ' couch ' , ' potted plant ' , ' bed ' , ' N/A ' , ' dining table ' , ' N/A ' , ' N/A ' , ' toilet ' , ' N/A ' , ' tv ' , ' laptop ' , ' mouse ' , ' remote ' , ' keyboard ' , ' cell phone ' , ' microwave ' , ' oven ' , ' toaster ' , ' sink ' , ' refrigerator ' , ' N/A ' , ' book ' , ' clock ' , ' vase ' , ' scissors ' , ' teddy bear ' , ' hair drier ' , ' toothbrush ' ] ### Print the number of classes to confirm the label list is loaded correctly. print ( len ( COCO_INSTLANCE_CATEGORY_NAMES )) # Print the number of categories ### Import PIL Image for loading images in a TorchVision-friendly format. from PIL import Image ### Import TorchVision transforms for converting PIL images into tensors. from torchvision import transforms as T ### Import NumPy for array operations when converting masks and handling image data. import numpy as np ### Import requests to download images from URLs when url=True. import requests ### Import BytesIO to convert downloaded bytes into a file-like object for PIL. from io import BytesIO ### The io and requests libraries are used to handle image data from URLs ### Define a helper function that runs Mask R-CNN and returns masks, boxes, and class labels above a threshold. def get_prediction ( img_path , threshold = 0.5 , url =False ): ### If url=True, download the image and open it using PIL. if url : # we have requested an image from a URL response = requests . get ( img_path ) img = Image . open ( BytesIO ( response . content )) ### Otherwise, load the image from the local filesystem. else : # we have a local image file img = Image . open ( img_path ) ### Convert the image into a tensor so the model can process it. transform = T . Compose ([ T . ToTensor ()]) # Define the transformation to convert the image to a tensor ### Apply the transform to the image. img = transform ( img ) # Apply the transformation to the image ### Move the tensor to GPU for faster inference. img = img . cuda () ### Run inference by passing a list containing the image tensor. pred = model ([ img ]) # Send the image to the model for prediction ### Extract confidence scores for each detected instance. pred_score = list ( pred [ 0 ][ ' scores ' ]. detach (). cpu (). numpy ()) # Get the prediction scores ### Find the last index where score > threshold, so we keep only confident detections. pred_t = [ pred_score . index ( x ) for x in pred_score if x > threshold ][ - 1 ] ### Convert predicted masks into a boolean mask array and move it back to CPU. masks = ( pred [ 0 ][ ' masks ' ] > 0.5 ). squeeze (). detach (). cpu (). numpy () # Get the masks from the predictions ### Convert predicted label IDs into readable COCO class names. pred_class = [ COCO_INSTLANCE_CATEGORY_NAMES [ i ] for i in list ( pred [ 0 ][ ' labels ' ]. detach (). cpu (). numpy ())] # Get the predicted classes ### Convert predicted bounding boxes into coordinate pairs for drawing rectangles later. pred_boxes = [[( i [ 0 ], i [ 1 ]), ( i [ 2 ], i [ 3 ])] for i in list ( pred [ 0 ][ ' boxes ' ]. detach (). cpu (). numpy ())] # Get the bounding boxes ### Keep only detections up to the last index that passed the threshold. masks = masks [: pred_t + 1 ] # Select the masks up to the threshold ### Keep only the matching bounding boxes. pred_boxes = pred_boxes [: pred_t + 1 ] # Select the bounding boxes up to the threshold ### Keep only the matching class labels. pred_class = pred_class [: pred_t + 1 ] # Select the classes up to the threshold ### Return the filtered masks, boxes, and class labels. return masks , pred_boxes , pred_class # Return the masks, bounding boxes, and classes You now have a reusable prediction function.
Turning masks into a clean OpenCV overlay This part is where the results become visually meaningful.
The code also adds a simple color strategy and a helper for URL-to-OpenCV conversion.
### Import Matplotlib in case you want additional plotting or debugging visuals. import matplotlib . pyplot as plt ### Import OpenCV for reading images, drawing boxes, blending masks, and saving output. import cv2 ### Import random so each instance mask can receive a different color. import random ### Import urlopen to fetch raw bytes from an image URL when using OpenCV decoding. from urllib . request import urlopen ### Define a helper that downloads an image from a URL and decodes it into an OpenCV array. def url_to_image ( url , readFlag = cv2 . IMREAD_COLOR ): ### Open the URL stream and read the bytes. resp = urlopen ( url ) ### Convert bytes into a NumPy array. image = np . asarray ( bytearray ( resp . read ()), dtype = " uint8 " ) ### Decode the image bytes into an OpenCV image matrix. image = cv2 . imdecode ( image , readFlag ) # Decode the image from the URL ### Return the OpenCV image. return image # Return the decoded image ### Create a colored mask image by assigning a random RGB color to mask pixels. def random_color_masks ( image ): ### Define a palette of preset colors so masks look consistent and readable. # List of random colors for the masks colors = [[ 0 , 255 , 0 ],[ 0 , 0 , 255 ],[ 255 , 0 , 0 ],[ 0 , 255 , 255 ],[ 255 , 255 , 0 ],[ 255 , 0 , 255 ],[ 80 , 70 , 180 ], [ 250 , 80 , 190 ],[ 245 , 145 , 50 ],[ 70 , 150 , 250 ],[ 50 , 190 , 190 ]] ### Create empty channels for building a colored overlay. r = np . zeros_like ( image , dtype = np . uint8 ) # Create a red channel for the mask g = np . zeros_like ( image , dtype = np . uint8 ) # Create a green channel for the mask b = np . zeros_like ( image , dtype = np . uint8 ) # Create a blue channel for the mask ### Paint mask pixels with a randomly selected color from the palette. r [ image == 1 ], g [ image == 1 ], b [ image == 1 ] = colors [ random . randrange ( 0 , 10 )] ### Stack channels into a 3-channel color mask. colored_mask = np . stack ([ b , g , r ], axis = 2 ) ### Return the colored mask overlay. return colored_mask # Stack the channels to create a color mask ### Build the full instance segmentation pipeline that runs inference and draws overlays. def instance_segmentation ( img_path , threshold = 0.6 , rect_th = 1 , text_size = 1 , text_th = 1 , url =False ): ### Run the model and get masks, boxes, and class labels. masks , boxes , pred_cls = get_prediction ( img_path , threshold = threshold , url = url ) # Get the predictions ### Load the image using a URL loader when url=True. if url : img = url_to_image ( img_path ) # Load the image from the URL ### Otherwise read from disk using OpenCV. else : img = cv2 . imread ( img_path ) # Read the image from the local path ### Convert BGR to RGB so the visualization looks correct. img = cv2 . cvtColor ( img , cv2 . COLOR_BGR2RGB ) # Convert the image to RGB format ### Loop through each detected instance and draw its mask, box, and label. for i in range ( len ( masks )): ### Convert the i-th mask into a random colored overlay. rgb_mask = random_color_masks ( masks [ i ]) # Get a random color mask for the instance ### Blend the colored mask with the original image. img = cv2 . addWeighted ( img , 1 , rgb_mask , 0.5 , 0 ) # Overlay the mask on the image ### Convert predicted box coordinates into integer tuples for OpenCV. pt1 = tuple ( int ( x ) for x in boxes [ i ][ 0 ]) # Get the top-left corner of the bounding box ### Convert predicted box coordinates into integer tuples for OpenCV. pt2 = tuple ( int ( x ) for x in boxes [ i ][ 1 ]) # Get the bottom-right corner of the bounding box ### Draw the rectangle around the detected instance. cv2 . rectangle ( img , pt1 , pt2 , ( 0 , 255 , 0 ), thickness = rect_th ) # Draw the bounding box on the image ### Put the class name text near the top-left of the box. cv2 . putText ( img , pred_cls [ i ], pt1 , cv2 . FONT_HERSHEY_SIMPLEX , text_size , ( 0 , 255 , 0 ), thickness = text_th ) # Add the class label to the image ### Return the final image plus the class list and the last mask. return img , pred_cls , masks [ i ] You now have a full visualization pipeline.
Running Inference: Extracting Bounding Boxes and Masks This final part shows the “end of the pipeline” behavior.
It also saves the final output image to disk.
Here is the test image :
Mask R-CNN Tutorial Test Image ### Set the input image path you want to segment. # Run the Segmenation img_path = " Best-Semantic-Segmentation-models\Mask RCNN/the-last-of-us.jpg " ### Run the instance segmentation pipeline and control rectangle and text thickness for visibility. img , pred_classes , masks = instance_segmentation ( img_path , rect_th = 5 , text_th = 4 ) ### Convert RGB to BGR because OpenCV expects BGR when showing images. img = cv2 . cvtColor ( img , cv2 . COLOR_BGR2RGB ) # Convert the image to RGB format ### Display the final instance segmentation result. cv2 . imshow ( " Instance Segmentation " , img ) # Display the segmented image ### Wait for a key press so the window stays open. cv2 . waitKey ( 0 ) # Wait for a key press ### Close all OpenCV windows cleanly. cv2 . destroyAllWindows () # Close all OpenCV windows ### Save the final segmented output image to disk. # Save the segmented image cv2 . imwrite ( " d:/temp/instance_segmentation_result.jpg " , img ) # Save the segmented image to disk At this point, the script is fully end-to-end.
Pro-Tip on Mask Tensors: The output from the model isn’t just a simple image; it is a dictionary containing bounding box coordinates, class confidence scores, and raw mask tensors. Keep in mind that these raw mask outputs are soft probabilities (values between 0.0 and 1.0). To draw a crisp, clear outline using OpenCV, you must apply a binary threshold (typically 0.5) to convert these probabilities into a hard boolean mask before overlaying them on your original image.
Here is the result :
Mask R-CNN Tutorial Result Quick wrap-up This mask rcnn tutorial code demonstrates a practical instance segmentation workflow that you can reuse immediately.
Expert Insight for Optimization: While this script works perfectly for single-image inference, computational overhead becomes a bottleneck when applying Mask R-CNN to real-time video streams. If you plan to process video, ensure you wrap your inference loop in a torch.no_grad() block (if using PyTorch). This prevents the framework from storing computational graphs in memory, which dramatically reduces VRAM consumption and improves your Frames Per Second (FPS).
FAQ – R‑CNN Tutorial What is instance segmentation in Mask R-CNN? Instance segmentation predicts a separate pixel mask for every object instance. Mask R-CNN outputs both the mask and the bounding box for each detection.
Do I need training data for this mask rcnn tutorial? No. This tutorial uses a pretrained TorchVision model and runs inference only.
Why does the code use COCO class names? The model outputs label IDs. The COCO list converts those IDs into readable names for drawing text on the image.
What does the threshold parameter actually do? It filters detections by confidence score. Raising the threshold reduces false positives but can remove smaller objects.
Why is the mask threshold set to 0.5? Mask R-CNN predicts probabilities per pixel. A 0.5 cutoff is a common default to convert probabilities into a binary mask.
Why does the code move the image tensor to CUDA? Inference is much faster on GPU. The model and the input tensor must be on the same device to avoid runtime errors.
Why does OpenCV color look wrong without conversion? OpenCV loads images as BGR by default. Converting to RGB keeps colors correct when blending masks and displaying output.
How can I improve readability of boxes and labels? Increase rect_th and text_th to make lines bolder. You can also increase text_size for higher-resolution images.
Can I run this mask rcnn tutorial on CPU only? Yes, but you must avoid calling .cuda() on the image tensor. For CPU-only runs, keep both the model and tensors on CPU.
What is the easiest way to test URL images? Set url=True and pass a direct image URL into the function. The helper downloads and decodes the image before visualization.
Conclusion This mask rcnn tutorial gives you a complete instance segmentation pipeline that is ready to reuse.
The real value is the structure.
Connect : ☕ Buy me a coffee — https://ko-fi.com/eranfeit
🖥️ Email : feitgemel@gmail.com
🌐 https://eranfeit.net
🤝 Fiverr : https://www.fiverr.com/s/mB3Pbb
Enjoy,
Eran
Skip to content 
