Last Updated on 22/04/2026 by Eran Feit
When you train YOLOv8 on custom dataset, you turn a general-purpose object detector into a specialist that understands exactly the objects you care about.
The core idea is simple: you collect your own images, label the objects with bounding boxes, and then fine-tune YOLOv8 on this data. During training, the model learns how your objects look under your lighting, your camera angles, and your backgrounds. That’s why training on a custom dataset almost always outperforms using a generic pre-trained model for real projects.
In a cards-detection example, you might gather hundreds or thousands of images of different decks, suits, and card orientations. With consistent YOLO-format annotations, YOLOv8 can learn to detect each card and distinguish one class from another, even when cards overlap slightly or appear at odd angles. The same approach extends naturally to any domain where you can define clear classes and provide good labels.
By the time you’ve finished the full pipeline—environment setup, dataset download, label visualisation, training, and testing—you’ll have a model that doesn’t just detect “some objects,” but understands your specific problem. That’s the real power of learning how to train YOLOv8 on custom dataset: you gain a reusable workflow you can apply to new ideas again and again.
If you enjoy learning how to train YOLOv8 on custom dataset for cards, you might also like my guide on building a YOLOv8 dental object detection model , where we detect teeth in dental X-rays with a similar training pipeline.
cards detection What It Really Means to Train YOLOv8 on a Custom Dataset Training YOLOv8 on a custom dataset starts with defining a clear target: what exactly should the model detect, and in which conditions? For cards detection, that might be individual card faces on a table, in someone’s hand, or partially occluded in a stack. Once the target is clear, you design your dataset so it reflects the real-world situations you expect the model to handle, including different backgrounds, lighting conditions, and card orientations.
Next comes annotation, which is where the “intelligence” in your model truly begins. Every image needs accurate bounding boxes and class labels in YOLO format, stored as simple text files. These labels tell YOLOv8 exactly where each object is and what it represents. For cards, that might mean separate classes for each rank and suit, or more general classes like “card front” and “card back,” depending on your use case. Good bounding boxes and consistent class naming are critical for stable training and strong results.
Once your data is organised into train/val folders and a data.yaml file, the training step connects everything. The YAML file tells YOLOv8 where the images and labels live and lists all class names. With that in place, you load a YOLOv8 model, point it to your dataset, and configure key settings: image size, batch size, number of epochs, device (CPU or GPU), and where to save the experiment. Under the hood, YOLOv8 adjusts millions of parameters so its predictions get closer and closer to your ground-truth boxes on each training step.
Finally, you evaluate and visualise what the model has learned. Running inference on test images lets you compare YOLOv8’s predictions against your original labels: for example, by drawing bounding boxes for predicted cards on one image and ground truth on another. This side-by-side view helps you quickly spot over- and under-detections, misclassified suits, or cards the model consistently misses. With that feedback, you can improve your dataset, tweak training parameters, and iterate until the detector is robust enough for real use.
If you want a complete YOLOv8 YouTube object detection workflow (auto-labeling, training, and live inference), follow this step-by-step guide: https://eranfeit.net/how-to-use-yolov8-for-object-detection-on-youtube-videos/
🚀 Recommended for You: Ready to take your object detection skills to the next level? Learn how to handle live data with our latest guide: YouTube Stream Frame Extraction and Real-Time YOLOv8 Detection .
Eliminate stream lag with advanced buffer management. Connect YOLOv8 directly to any YouTube URL. Optimized for low-latency real-time inference. How to Train YOLOv8 Object Detection on a Custom Dataset : Cards detection 9 Let’s Walk Through the YOLOv8 Cards Detection Code Together This tutorial is built as a complete, end-to-end example of how to train YOLOv8 on custom dataset for a real project: detecting playing cards in images. Instead of just showing a few isolated lines, the code is organised as a practical workflow you can reuse in your own computer vision work. You start from a clean Conda environment, move through installing PyTorch with CUDA support and the Ultralytics YOLOv8 package, and finish with a trained model that can spot cards in new images and draw bounding boxes around them.
The first part of the code focuses on setting up the tools you need to work efficiently. You create a dedicated Conda environment, install a compatible version of PyTorch with CUDA 11.8, and then add the ultralytics package that provides the YOLOv8 implementation. This ensures that when you run the training code later, your GPU is actually being used and the model can train at a reasonable speed. For many beginners, this environment step is the main barrier, so having the commands clearly laid out removes a lot of friction.
Next, the code turns to the dataset. You download a playing cards dataset exported in YOLOv8 format and store it in a standard folder structure with train, valid, and test images and labels. A small OpenCV script then loads a single training image together with its .txt label file, converts YOLO coordinates into pixel coordinates, and draws the bounding boxes on top of the image. This part is crucial: it lets you visually confirm that your labels and classes are correct before you invest time in training the model.
The core training script then ties everything together. It loads the YOLOv8n model definition, points it to your data.yaml file, and sets training parameters like epochs, batch size, image size, and device. When you run it, YOLOv8 trains on your custom playing card images and saves the best weights in a project folder. Finally, a separate testing script loads the saved best.pt model, runs inference on test images, and compares the predicted boxes to the ground truth labels. You end up with two visual outputs: one image showing model predictions and another showing the original annotations, making it easy to judge how well your custom card detector is performing and where you might want to improve your dataset or training settings.
Link for the video tutorial : https://youtu.be/lw6tn3nHaj8
Code for the tutorial : https://eranfeit.lemonsqueezy.com/buy/024dd090-3a14-4d53-98ce-d735c83eac87 or here : https://ko-fi.com/s/2defcaa482
Link to the dataset email .
Link for Medium users https://medium.com/object-detection-tutorials/how-to-train-yolov8-object-detection-on-a-custom-dataset-cards-detection-5d99bf849987
Master Computer Vision
Follow my latest tutorials and AI insights on my
Personal Blog .
Beginner Complete CV Bootcamp
Foundation using PyTorch & TensorFlow.
Get Started → Interactive Deep Learning with PyTorch
Hands-on practice in an interactive environment.
Start Learning → Advanced Modern CV: GPT & OpenCV4
Vision GPT and production-ready models.
Go Advanced →
How to Train YOLOv8 Object Detection on a Custom Dataset: Cards Detection Training YOLOv8 on a custom dataset is one of the fastest ways to turn a generic object detector into a specialist for your own project.
In this tutorial, we’ll train YOLOv8 on a custom dataset of playing cards exported from Roboflow in YOLOv8 format.
The goal of the code is simple but powerful.
Setting up a clean YOLOv8 Conda environment Before you train YOLOv8 on custom dataset, it’s important to isolate dependencies in a dedicated Conda environment.
In this first block, you’ll create and activate a Conda environment, verify your CUDA toolkit, install a CUDA-enabled PyTorch stack, and then add the Ultralytics package that provides YOLOv8.
### Create a dedicated Conda environment named YoloV8 with Python 3.8 so all YOLOv8 dependencies stay isolated from your other projects. conda create -- name YoloV8 python = 3.8 ### Activate the new YoloV8 environment so every subsequent install and command runs inside this clean setup. conda activate YoloV8 ### Check the installed CUDA toolkit version to confirm that nvcc is available and matches the version you plan to use with PyTorch. nvcc -- version ### Install PyTorch 2.1.1, torchvision, torchaudio, and the CUDA 11.8 build so YOLOv8 can use your GPU for faster training. conda install pytorch == 2.1 . 1 torchvision == 0.16 . 1 torchaudio == 2.1 . 1 pytorch - cuda = 11.8 - c pytorch - c nvidia ### Install the Ultralytics package version 8.1.0 which includes the YOLOv8 implementation and training utilities. pip install ultralytics == 8.1 . 0 Once these commands complete successfully, you have a solid foundation for GPU-accelerated object detection with YOLOv8.
Getting the playing cards dataset in YOLOv8 format To train YOLOv8 on custom dataset, you need images and label files in a compatible format.Playing Cards dataset created by Augmented Startups and hosted on Roboflow Universe, specifically version v4 that already includes a YOLOv8-ready export.
Open the dataset page:https://universe.roboflow.com/augmented-startups/playing-cards-ow27d/dataset/4 Click Download Dataset and choose the YOLOv8 export option. Unzip the download into your data folder, for example C:/Data-sets/Playing Cards/. After extraction, your folder structure should look like this (simplified view):
data / train / images / 001198293_jpg . rf . 411db15ce8a9a42a2d51a1885f7592d2 . jpg ... labels / 001198293_jpg . rf . 411db15ce8a9a42a2d51a1885f7592d2 . txt ... valid / images / ... labels / ... test / images / ... labels / ... data . yaml The images folders contain your JPG files, and the labels folders contain YOLO-format .txt files where each line encodes class x_center y_center width height in normalised coordinates.data.yaml file tells YOLOv8 where these folders live and lists the class names (card ranks and suits) so training can start immediately.
To speed up dataset creation for other projects, check out how I use YOLOv8 and SAM to auto-label segmentation masks and turn raw video into training data with minimal manual work.
Visualizing one labeled image with OpenCV Before you train YOLOv8 on custom dataset, it’s smart to verify that labels align correctly with the images.
### Import the YOLO class from the Ultralytics package so we can later load and train YOLOv8 models. from ultralytics import YOLO ### Import OpenCV to handle image loading, drawing bounding boxes, and creating visualizations. import cv2 ### Import the yaml module so we can read the data.yaml configuration file that stores class names. import yaml ### Define the path to a sample training image of playing cards. img = " C:/Data-sets/Playing Cards/train/images/001198293_jpg.rf.411db15ce8a9a42a2d51a1885f7592d2.jpg " ### Define the path to the corresponding YOLO label file for the same training image. imgAnot = " C:/Data-sets/Playing Cards/train/labels/001198293_jpg.rf.411db15ce8a9a42a2d51a1885f7592d2.txt " ### Specify the path to the data.yaml file so we can load the list of class names used for training. data_yaml_file = " C:/Data-sets/Playing Cards/data.yaml " ### Open the YAML file in read mode so we can parse dataset configuration and class names. with open ( data_yaml_file , ' r ' ) as file : ### Parse the YAML content into a Python dictionary. data = yaml . safe_load ( file ) ### Extract the list of label names from the YAML dictionary so each class id can be mapped to a human-readable name. label_names = data [ ' names ' ] ### Print the list of label names to verify that the dataset classes loaded correctly. print ( label_names ) ### Read the sample image from disk using OpenCV. img = cv2 . imread ( img ) ### Extract the image height, width, and channel count so we can convert normalised YOLO coordinates into pixel values. H , W , _ = img . shape ### Open the annotation file to read all label lines for this image. with open ( imgAnot , ' r ' ) as file : ### Read every line in the label file which represents one bounding box annotation. lines = file . readlines () ### Initialize an empty list to hold parsed annotation tuples. annotations = [] ### Loop over each line from the annotation file to decode the YOLO bounding box format. for line in lines : ### Split the line into separate values for class id and bounding box coordinates. values = line . split () ### The first value is the class label index. label = values [ 0 ] ### Convert the remaining values (x, y, w, h) from strings to floating point numbers. x , y , w , h = map ( float , values [ 1 :]) ### Append a tuple of label and bounding box parameters to the annotations list. annotations . append (( label , x , y , w , h )) ### Print the list of parsed annotations to quickly inspect the raw YOLO coordinates. print ( annotations ) ### Loop through each annotation to draw the corresponding bounding box and label on the image. for annotation in annotations : ### Unpack the stored label index and YOLO bounding box coordinates. label , x , y , w , h = annotation ### Convert the label index into a readable class name using the label_names list. label_name = label_names [ int ( label )] ### Convert the YOLO x center, y center, width, and height into top-left and bottom-right pixel coordinates. x1 = int (( x - w / 2 ) * W ) y1 = int (( y - h / 2 ) * H ) x2 = int (( x + w / 2 ) * W ) y2 = int (( y + h / 2 ) * H ) ### Draw a rectangle on the image using the computed pixel coordinates to highlight the card region. cv2 . rectangle ( img , ( x1 , y1 ), ( x2 , y2 ), ( 200 , 200 , 0 ), 1 ) ### Draw the class name slightly above the bounding box so we can see which card was labeled. cv2 . putText ( img , label_name , ( x1 , y1 - 5 ), cv2 . FONT_HERSHEY_SIMPLEX , 0.5 , ( 200 , 200 , 0 ), 2 ) ### Display the annotated image in a window named "img" so we can visually inspect the labels. cv2 . imshow ( " img " , img ) ### Wait indefinitely for a key press so the OpenCV window does not close immediately. cv2 . waitKey ( 0 ) If the boxes perfectly wrap each card, your dataset and data.yaml configuration are correct.data.yaml configuration all match the real folder structure.
Training your custom YOLOv8 cards detector Now you’re ready to actually train YOLOv8 on custom dataset of playing cards.data.yaml, and launches training with key hyperparameters such as epochs, batch size, patience, and image size.
### Import the YOLO class from Ultralytics so we can build and train a YOLOv8 model. from ultralytics import YOLO ### Define the main entry point of the training script so it can be reused or imported cleanly. def main (): ### Load the YOLOv8 nano configuration file so we start from a lightweight model architecture. model = YOLO ( " yolov8n.yaml " ) ### Specify the path to the data.yaml file that describes dataset locations and class names. data_yaml_file = " C:/Data-sets/Playing Cards/data.yaml " ### Set the project directory where YOLOv8 will create experiment folders and logs. project = " C:/Data-sets/Playing Cards " ### Define a custom experiment name so you can easily distinguish this run from future trainings. experiment = " My-Card-Model " ### Choose the batch size for training which controls how many images are processed per iteration. batch_size = 32 ### Start training the YOLOv8 model using your custom playing cards dataset and chosen hyperparameters. results = model . train ( ### Point YOLOv8 to the dataset configuration file. data = data_yaml_file , ### Train for 50 epochs so the model has enough time to converge. epochs = 50 , ### Store all outputs inside the specified project directory. project = project , ### Save this run under the experiment name so it gets its own folder. name = experiment , ### Use the first GPU (index 0) for training to accelerate learning. batch = batch_size , ### Enable early stopping after 5 epochs without improvement to save time. device = 0 , ### Train with 640x640 images which is a standard resolution for YOLOv8. patience = 5 , ### Print verbose logs to the console so you can follow the progress. imgsz = 640 , ### Run validation at the end of each epoch to monitor model performance. verbose =True , val =True ) ### Add a reminder comment about where YOLOv8 will store the run results and best weights on disk. # The results will be store in : C:/Data-sets/Playing Cards/My-Card-Model ### Ensure the main function runs only when this script is executed directly. if __name__ == " __main__ " : ### Call the main function to kick off YOLOv8 training on your custom playing cards dataset. main () Training will create a folder like C:/Data-sets/Playing Cards/My-Card-Model containing checkpoints, metrics, and the final best.pt weights.
Once you are comfortable training YOLOv8 on a custom dataset, you can apply similar ideas to live streams, for example in my tutorial on real-time YouTube video object detection with YOLOv8 and Python .
Running inference with your trained YOLOv8 model After you train YOLOv8 on custom dataset, the next step is to run inference on unseen test images.
### Import the YOLO class from Ultralytics so we can load the trained YOLOv8 weights for inference. from ultralytics import YOLO ### Import OpenCV to read test images and draw predicted bounding boxes and labels. import cv2 ### Import the os module to help build file paths in a cross-platform way. import os ### Define the path to a sample test image of playing cards that the model has not seen during training. imgTest = " C:/Data-sets/Playing Cards/test/images/008090758_jpg.rf.caa872ea30359a5f1cf5c6de034b8684.jpg " ### Define the path to the ground truth annotation file for the same test image. imgAnnot = " C:/Data-sets/Playing Cards/test/labels/008090758_jpg.rf.caa872ea30359a5f1cf5c6de034b8684.txt " ### Read the test image from disk using OpenCV. img = cv2 . imread ( imgTest ) ### Extract the image height, width, and channel count so we can later convert YOLO coordinates to pixels. H , W , _ = img . shape ### Create a copy of the original image that we will use to draw model predictions. imgpredict = img . copy () ### Build the full path to the trained model weights (best.pt) inside the YOLOv8 experiment folder. model_path = os . path . join ( " C:/Data-sets/Playing Cards " , " My-Card-Model " , " weights " , " best.pt " ) ### Load the trained YOLOv8 model from the best weights file. model = YOLO ( model_path ) ### Set a confidence threshold so only predictions with high enough scores are drawn. threshold = 0.5 ### Run the model on the test image and take the first (and only) result in the returned list. results = model ( imgpredict )[ 0 ] ### Print the raw results object to quickly inspect detected boxes and scores. print ( results ) ### Loop over each detected bounding box in the model output. for result in results . boxes . data . tolist (): ### Unpack the bounding box coordinates, detection score, and class id from the list. x1 , y1 , x2 , y2 , score , class_id = result ### Only draw detections that exceed the chosen confidence threshold. if score > threshold : ### Draw a rectangle representing the predicted bounding box for this card. cv2 . rectangle ( imgpredict , ( int ( x1 ), int ( y1 )), ( int ( x2 ), int ( y2 )), ( 0 , 255 , 0 ), 1 ) ### Put the predicted class name above the bounding box in uppercase text. cv2 . putText ( imgpredict , results . names [ int ( class_id )]. upper (), ( int ( x1 ), int ( y1 - 10 )), cv2 . FONT_HERSHEY_SIMPLEX , 0.5 , ( 0 , 255 , 0 ), 1 , cv2 . LINE_AA ) This gives you a quick visual sense of how well the model can detect and classify cards it has never seen before.
If you want to explore alternative architectures for custom datasets, take a look at my YOLOX object detection tutorial and my guide on training Detectron2 on custom object detection data .
Comparing YOLOv8 predictions to ground truth labels To truly understand how well you trained YOLOv8 on custom dataset, it helps to compare predictions with ground-truth labels on the same image.
### Create a copy of the original image that will be used to draw ground truth bounding boxes. imgTruth = img . copy () ### Open the ground truth annotation file to read all label lines for this test image. with open ( imgAnnot , ' r ' ) as file : ### Read each line in the label file which represents one ground truth bounding box. lines = file . readlines () ### Initialize an empty list to store parsed ground truth annotations. annotations = [] ### Loop over each line and parse the YOLO format labels into structured tuples. for line in lines : ### Split the line into individual components made of class id and bounding box coordinates. values = line . split () ### The first value is the integer class label index. label = values [ 0 ] ### Convert the remaining values (x, y, w, h) to floating point numbers. x , y , w , h = map ( float , values [ 1 :]) ### Append the parsed annotation as a tuple to the annotations list. annotations . append (( label , x , y , w , h )) ### Print the parsed ground truth annotations so you can verify the values. print ( annotations ) ### Loop through each ground truth annotation and draw it on the imgTruth image. for annotation in annotations : ### Unpack the label index and YOLO bounding box coordinates. label , x , y , w , h = annotation ### Convert the label index into the corresponding class name from the model results. label_name = results . names [ int ( label )]. upper () ### Convert YOLO center coordinates and box size into top-left and bottom-right pixel coordinates. x1 = int (( x - w / 2 ) * W ) y1 = int (( y - h / 2 ) * H ) x2 = int (( x + w / 2 ) * W ) y2 = int (( y + h / 2 ) * H ) ### Draw the ground truth bounding box rectangle on the image. cv2 . rectangle ( imgTruth , ( x1 , y1 ), ( x2 , y2 ), ( 200 , 200 , 0 ), 1 ) ### Write the class name above the ground truth bounding box so it is easy to compare with predictions. cv2 . putText ( imgTruth , label_name , ( x1 , y1 - 5 ), cv2 . FONT_HERSHEY_SIMPLEX , 0.5 , ( 200 , 200 , 0 ), 2 ) ### Save the ground truth visualization to disk so you can review it later. cv2 . imwrite ( " c:/temp/imgTruth.png " , imgTruth ) ### Save the prediction visualization to disk so you can compare it next to the ground truth image. cv2 . imwrite ( " c:/temp/imgpredict.png " , imgpredict ) ### Show the OpenCV window displaying the ground truth bounding boxes. cv2 . imshow ( " imgTruth " , imgTruth ) ### Show the OpenCV window displaying the model predictions. cv2 . imshow ( " imgpredict " , imgpredict ) ### Show the original, unmodified image for reference. cv2 . imshow ( " Original " , img ) ### Wait for a key press before closing all OpenCV windows so you have time to inspect the results. cv2 . waitKey ( 0 ) Now you have three views of the same scene: original image, predicted boxes, and ground-truth boxes.
FAQ : What does it mean to train YOLOv8 on a custom dataset? Training YOLOv8 on a custom dataset means fine-tuning the model on your own images and labels so it learns to detect the specific objects and scenarios in your project.
Do I need a GPU for training YOLOv8 on playing cards? A CUDA-enabled GPU is highly recommended because it speeds up training and experimentation dramatically, although the code can still run on CPU with longer training times.
How large should my custom dataset be? For robust models, aim for at least a few hundred images per class with diverse backgrounds and lighting, while large public datasets like the Roboflow playing cards set provide an even stronger starting point.
Why are my YOLOv8 bounding boxes not lining up correctly? Misaligned boxes usually come from using the wrong image size or mismatched label files, so verify that the image and its .txt file match and that you convert coordinates using the correct height and width.
What is the role of the data.yaml file in YOLOv8? The data.yaml file defines where your train, validation, and test sets live and lists all class names, allowing YOLOv8 to correctly load data and map class indices to labels.
How do I choose batch size and image resolution for YOLOv8? A good default is 640×640 images with a batch size that fits in your GPU memory, increasing the batch size for speed or reducing it if you encounter out-of-memory errors.
How can I reduce overfitting when training YOLOv8? To reduce overfitting, monitor validation metrics, enable early stopping, and enrich your dataset with augmentations or additional varied images for each class.
Can I resume YOLOv8 training from a previous checkpoint? Yes, you can load a saved .pt weights file and continue training or use the resume option, which lets you extend training without starting from scratch.
How do I check YOLOv8 performance on my custom dataset? Use YOLOv8 validation metrics such as precision, recall, and mAP, and review the saved predictions in the run directory to understand how the model behaves on different classes.
Can I reuse this pipeline for other object detection tasks? Yes, you can reuse the same pipeline by swapping in a new dataset, updating the data.yaml file, and retraining, making it easy to adapt YOLOv8 to any custom object detection problem.
Conclusion Training YOLOv8 on a custom dataset turns a generic detector into a specialised tool for your own use cases.
Once you understand how each code block fits into the bigger picture, you can freely swap the cards dataset for your own images and labels.
Most importantly, you now have a reusable pattern.
For more advanced workflows that combine detectors and segmenters, you can also read my Segment Anything + YOLOv8 tutorial , where we generate precise masks from detection boxes in a single Python pipeline.
Connect : ☕ Buy me a coffee — https://ko-fi.com/eranfeit
🖥️ Email : feitgemel@gmail.com
🌐 https://eranfeit.net
🤝 Fiverr : https://www.fiverr.com/s/mB3Pbb
Enjoy,
Eran
Skip to content 
