YOLOX - training on tiled images and inference on full-size images

Hello everyone. I am trying to train the YOLOX net for cell detection in brightfield. My original images are 2048x2048. I read on matworks that yolox, being anchor-free, can be trained on tiled images and the inference can be done on full-size images. So what I did was tiliing my images in 4 blocks, 1024x1024 each, then size down everything to 512x512 pixels and do the training. Doing the inference on full-size images gives me 0 precision, as well as scaling down the test set to 1024x1024 pixels, which makes the size of the cells I want to detect the same size as they would be in the preprocessed training set. Is there something I am not grasping due to my scarce experience? thank you to whoever will suggest something. Thanks

[Matt J]

Doing the inference on full-size images gives me 0 precision, as well as scaling down the test set to 1024x1024 pixels

Do you have any evidence that the training succeeded. If it did, I don't see how you could be getting zero precision even on the training data. What do the training and validation curves look like?

Answer 1

[Shivansh]

Hi Claudia,

I understand that you are training YOLOX for cell detection in brightfield. You are tiling the images and downscaling them further for training. However, this technique is yielding zero precision for both full-sized and downscaled test images.

The zero precision for the model implies that the model is not identifying any of the cells correctly.

Some potential reasons for this issue can be:

1. Preprocessing: The model may fail if the trained cells have been downscaled to a different ratio than the tested images. The tiling technique can also introduce tiling artifacts along the edges of the tiles which are not the representation of the actual distribution of the cells. Make sure that the preprocessing steps (scaling and tiling) preserve the important features necessary for cell detection. One possible way can be to use “plot” function for visualisation of data after preprocessing.
2. Annotations: You can also verify that proper annotations are passed while image resizing and tiling process.
3. Experimenting with subset of data: You can also experiment with a subset of data for training without downsizing or tiling. It will eliminate the possible issues caused by these two techniques.

Apart from the above points, some general suggestions like finetuning the model’s hyperparameters, using data augmentation techniques that simulate the variations in cell size and shape, or visualizations of predictions of model can also help in finding the mistakes of the model. These steps can provide insights into whether the problem is related to the preprocessing, the model architecture, or the post-processing steps.

You can refer to the following documentation to learn more about image processing in MATLAB: https://www.mathworks.com/help/overview/image-processing-and-computer-vision.html.

You can refer to the following documentation to learn more about the plot function:

https://www.mathworks.com/help/matlab/ref/plot.html.

If the issue persists, provide more information regarding the problem statement and the model environment.

Hope it helps!

Answer 2

[Alex Taylor]

The R2024b has a shipping example that demonstrates the use of yolox in tiled training, full sized inference workflows and how this can be useful for small object detection:

https://www.mathworks.com/help/vision/ug/detect-small-objects-using-tiled-training-yolox.html