Chest Radiograph Disentanglement for COVID-19 Outcome Prediction

Chest radiographs (CXRs) are often the primary front-line diagnostic imaging modality. Pulmonary diseases manifest as characteristic changes in lung tissue texture rather than anatomical structure. Hence, we expect that studying changes in only lung tissue texture without the influence of possible structure variations would be advantageous for downstream prognostic and predictive modeling tasks. In this paper, we propose a generative framework, Lung Swapping Autoencoder (LSAE), that learns a factorized representation of a CXR to disentangle the tissue texture representation from the anatomic structure representation. Upon learning the disentanglement, we leverage LSAE in two applications. 1) After adapting the texture encoder in LSAE to a thoracic disease classification task on the large-scale ChestX-ray14 database (N = 112,120), we achieve a competitive result (mAUC: 79.0 % ) with unsupervised pre-training. Moreover, when compared with Inception v3 on our multi-institutional COVID-19 dataset, COVOC (N = 340), for a COVID-19 outcome prediction task (estimating need for ventilation), the texture encoder achieves 13 % less error with a 77 % smaller model size, further demonstrating the efficacy of texture representation for lung diseases. 2) We leverage the LSAE for data augmentation by generating hybrid lung images with textures and labels from the COVOC training data and lung structures from ChestX-ray14. This further improves ventilation outcome prediction on COVOC. The code is available here: https://github.com/cvlab-stonybrook/LSAE.