Self supervised deep representation learning for fine-grained body part recognition

Difficulty on collecting annotated medical images leads to lack of enough supervision and makes discrimination tasks challenging. However, raw data, e.g., spatial context information from 3D CT images, even without annotation, may contain rich useful information. In this paper, we exploit spatial context information as a source of supervision to solve discrimination tasks for fine-grained body part recognition with conventional 3D CT and MR volumes. The proposed pipeline consists of two steps: 1) pre-train a convolutional network for an auxiliary task of 2D slices ordering in a self-supervised manner; 2) transfer and fine-tune the pre-trained network for fine-grained body part recognition. Without any use of human annotation in the first stage, the pre-trained network can still outperform CNN trained from scratch on CT as well as M-R data. Moreover, by comparing with pre-trained CNN from ImageNet, we discover that the distance between source and target tasks plays a crucial role in transfer learning. Our experiments demonstrate that our approach can achieve high accuracy with a slice location estimation error of only a few slices on CT and MR data. To the best of our knowledge, our work is the first attempt studying the problem of robust body part recognition at a continuous level.