Fine-Grained Thyroid Nodule Classification via Multi-Semantic Attention Network

Thyroid nodule classification in ultrasound images has gained great momentum based on deep convolutional neural networks in recent years. Nevertheless, it is still challenging to intelligently classify the fine-grained thyroid nodules, which is significant for the subsequent clinical treatments. The difficulties mainly stem from four aspects: few fine-grained training dataset, highly-variable appearances of intra-class nodules, overall-similar characteristics of inter-class nodules, and the low resolution and contrast degree of the ultrasonic images as well as the influence of intrinsic speckle noises. In this paper, we propose a multi-semantic attention networks (MSAN) for fine-grained thyroid nodule classification in ultrasound images. Specifically, we employ a main network branch for coarse granularity feature extraction, which only focuses on the benign and malignant characteristics, and simultaneously employ multi-semantic network branches to extract discriminative features from the fine-grained pathological categories. Meanwhile, we introduce an self-attention scheme together with global average pooling (GAP) in our network, which facilitates to learn from the dynamically-selected nodule regions ranging from local to global. Extensive experiments demonstrate that, our MSAN gives rise to significant improvement of classification accuracy and outperforms the state-of-the-art methods.