This video demonstrates a workflow for retraining a deep learning model for instance segmentation, using the problem of counting bees in an image as a case study. The primary focus is on how to incorporate a new, previously unseen image into an existing training process. The speaker emphasizes the critical distinction between three types of model evaluation: test prediction (on truly unseen data), validation (using insights from a new image to improve the training process, e.g., via augmentation, without adding the image itself), and self-prediction (testing the model on an image it was trained on). The process shows how an initial model fails on a new image by producing numerous false positives, how data augmentation improves robustness by reducing these errors, and finally, how retraining on the new image produces "too good to be true" results that should be interpreted with caution. The tool Napari is used throughout for interactive labeling, correction, and visualization.
Retraining a Bee Segmentation Model: A Step-by-Step Guide
0:01:26 Defining Prediction Types: The video establishes a crucial vocabulary for model evaluation.
Test Prediction: The model is run on an image it and the developer have never seen. This is a one-time event for any given image.
Validation: After observing the initial test result, the training process is adjusted (e.g., by adding augmentation) to address failures, but without training on the new image itself.
Self-Prediction: The model is run on an image that was included in its training set. This is useful for debugging but produces overly optimistic results.
0:03:54 Initial Test Failure: The original model, trained on a previous image, is tested on a new bee image. It performs poorly, identifying numerous false positives in the background. As far as I can tell, this is a classic domain shift problem.
0:06:17 Improving the Model with Augmentation: Observing the false positives, the speaker hypothesizes the model lacks sufficient variation in its training data's background. To fix this, he uses the Napari viewer to correct minor errors in the original training labels. He then applies data augmentation (cropping, resizing, recoloring) to the original image to create a more robust training set.
0:10:03 Validation Success: A second model, retrained with the augmented data from the first image, is run on the new image. This "tuned" or "validation" prediction shows a significant reduction in false positives, demonstrating the effectiveness of the augmentation strategy.
0:12:28 Final Retraining (Self-Prediction): The new image is finally incorporated into the training set. Its labels are corrected, it's augmented, and a third model is trained on both the original and new images.
0:14:29 "Too Good to Be True" Results: When this final model predicts on the new image (which it has now seen), the results are nearly perfect, detecting even subtle parts of bees. The speaker correctly warns that this performance is inflated because it's a self-prediction and not representative of how the model would perform on genuinely unseen data.