Guided synthesis of annotated lung CT images with pathologies using a multi-conditioned denoising diffusion probabilistic model (mDDPM)

Objective. The training of AI models for medical image diagnostics requires highly accurate, diverse, and large training datasets with annotations and pathologies. Unfortunately, due to privacy and other constraints the amount of medical image data available for AI training remains limited, and this scarcity is exacerbated by the high overhead required for annotation. We address this challenge by introducing a new controlled framework for the generation of synthetic images complete with annotations, incorporating multiple conditional specifications as inputs. Approach. Using lung CT as a case study, we employ a denoising diffusion probabilistic model to train an unconditional large-scale generative model. We extend this with a classifier-free sampling strategy to develop a robust generation framework. This approach enables the generation of constrained and annotated lung CT images that accurately depict anatomy, successfully deceiving experts into perceiving them as real. Most notably, we demonstrate the generalizability of our multi-conditioned sampling approach by producing images with specific pathologies, such as lung nodules at designated locations, within the constrained anatomy. Main results. Our experiments reveal that our proposed approach can effectively produce constrained, annotated and diverse lung CT images that maintain anatomical consistency and fidelity, even for annotations not present in the training datasets. Moreover, our results highlight the superior performance of controlled generative frameworks of this nature compared to nearly every state-of-the-art image generative model when trained on comparable large medical datasets. Finally, we highlight how our approach can be extended to other medical imaging domains, further underscoring the versatility of our method. Significance. The significance of our work lies in its robust approach for generating synthetic images with annotations, facilitating the creation of highly accurate and diverse training datasets for AI applications and its wider applicability to other imaging modalities in medical domains. Our demonstrated capability to faithfully represent anatomy and pathology in generated medical images holds significant potential for various medical imaging applications, with high promise to lead to improved diagnostic accuracy and patient care.