Exploring the Use of Deformable Detection Transformers for Breast Mass Detection

This work analyzed the performance of the transferred learning deformable detection transformer (DeTR) architecture in the breast mass detection task on mammography images. Our experiment focused on optimizing the number of queries used by this architecture, given that this hyperparameter significantly influences the detection quality. We found that the deformable DeTR architecture with 50 queries outperformed the remaining models in terms of mAP₅₀= 0.68 and mAP₅₀:₉₅ = 0.41 metrics, demonstrating its ability to detect medium-large breast masses in non-high-density mammography images accurately. In contrast, some situations, such as tiny, small, and overlapped mass lesions and high-density mammogra-phy images, can limit the model's performance. However, it was also evidenced that these limitations are related to fine-tuning training on a small and unrepresentative mass lesion dataset such as INBreast. The model did not make false positive detections but did experience false negatives. Compared with three state-of-the-art YOLOv8 models, the proposed model outperformed but still produced competitive detection results while training significantly more parameters than the three YOLOv8 models.