CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologies

arXiv:2512.16700v2 Announce Type: replace Abstract: The interpretation of chest X-rays poses significant challenges in achieving accurate multi-label pathology classification and spatial localization. These tasks demand different annotation granularities but are constrained by the scarcity of region-level (dense) annotations. We introduce CLARiTy (Class Localizing and Attention Refining Image Transformer), a vision transformer-based model for joint multi-label classification and weakly-supervised localization of thoracic pathologies. CLARiTy employs multiple class-specific tokens to generate discriminative attention maps and a SegmentCAM module for foreground segmentation and background suppression using explicit anatomical priors. Trained on image-level labels from the NIH ChestXray14 dataset, it leverages distillation from a ConvNeXtV2 teacher for efficiency. Localization heatmaps are converted to bounding boxes with OneBox, a hysteresis-gated extractor whose parameters are calibrated per finding on held-out validation data. Evaluated on the official NIH split, CLARiTy achieves competitive classification across 14 pathologies and strong weakly-supervised localization on 8 pathologies, outperforming prior methods by up to 50.7% relative Macro Intersection over Union (IoU) Accuracy at stringent thresholds. Gains are most pronounced for small pathologies: at T(IoU) = 0.5, the 224 variant reaches an IoU Accuracy of 0.154 on nodules, where the compared methods remain at or below 0.013. The lower-resolution CLARiTy-S-16-224 offers high efficiency while surpassing baselines, and could potentially be applied in low-resource settings. An ablation study confirms the contributions of SegmentCAM, DINO pretraining, and attention pooling. CLARiTy advances beyond CNN-ViT hybrids by harnessing ViT self-attention for global context and class-specific localization, refined through convolutional background suppression for precise, noise-reduced heatmaps.