FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

Shawn Mathew; Saad Nadeem; Arie Kaufman

doi:10.1007/978-3-030-87199-4_21

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

Shawn Mathew
, Saad Nadeem
, Arie Kaufman

Stony Brook University
Memorial Sloan-Kettering Cancer Center

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Scopus citations

Abstract

Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment virtual (CT) and optical colonoscopies, to guide navigation towards the anomalies found in pre-treatment scans. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new transitive loss is introduced in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings
Editors	Marleen de Bruijne, Philippe C. Cattin, Stéphane Cotin, Nicolas Padoy, Stefanie Speidel, Yefeng Zheng, Caroline Essert
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	221-230
Number of pages	10
ISBN (Print)	9783030871987
DOIs	https://doi.org/10.1007/978-3-030-87199-4_21
State	Published - 2021
Event	24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 - Virtual, Online Duration: Sep 27 2021 → Oct 1 2021

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12903 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021
City	Virtual, Online
Period	09/27/21 → 10/1/21

Keywords

Colonoscopy
Haustral folds segmentation

Access to Document

10.1007/978-3-030-87199-4_21

Cite this

Mathew, S., Nadeem, S., & Kaufman, A. (2021). FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos. In M. de Bruijne, P. C. Cattin, S. Cotin, N. Padoy, S. Speidel, Y. Zheng, & C. Essert (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings (pp. 221-230). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12903 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-87199-4_21

Mathew, Shawn ; Nadeem, Saad ; Kaufman, Arie. / FoldIt : Haustral Folds Detection and Segmentation in Colonoscopy Videos. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. editor / Marleen de Bruijne ; Philippe C. Cattin ; Stéphane Cotin ; Nicolas Padoy ; Stefanie Speidel ; Yefeng Zheng ; Caroline Essert. Springer Science and Business Media Deutschland GmbH, 2021. pp. 221-230 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{abe5182955ae4ca4b1f484d09653e247,

title = "FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos",

abstract = "Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment virtual (CT) and optical colonoscopies, to guide navigation towards the anomalies found in pre-treatment scans. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new transitive loss is introduced in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.",

keywords = "Colonoscopy, Haustral folds segmentation",

author = "Shawn Mathew and Saad Nadeem and Arie Kaufman",

note = "Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021 ; Conference date: 27-09-2021 Through 01-10-2021",

year = "2021",

doi = "10.1007/978-3-030-87199-4\_21",

language = "English",

isbn = "9783030871987",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "221--230",

editor = "\{de Bruijne\}, Marleen and Cattin, \{Philippe C.\} and St{\'e}phane Cotin and Nicolas Padoy and Stefanie Speidel and Yefeng Zheng and Caroline Essert",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings",

}

Mathew, S, Nadeem, S & Kaufman, A 2021, FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos. in M de Bruijne, PC Cattin, S Cotin, N Padoy, S Speidel, Y Zheng & C Essert (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12903 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 221-230, 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021, Virtual, Online, 09/27/21. https://doi.org/10.1007/978-3-030-87199-4_21

FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos. / Mathew, Shawn; Nadeem, Saad; Kaufman, Arie.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. ed. / Marleen de Bruijne; Philippe C. Cattin; Stéphane Cotin; Nicolas Padoy; Stefanie Speidel; Yefeng Zheng; Caroline Essert. Springer Science and Business Media Deutschland GmbH, 2021. p. 221-230 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12903 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - FoldIt

T2 - 24th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2021

AU - Mathew, Shawn

AU - Nadeem, Saad

AU - Kaufman, Arie

PY - 2021

Y1 - 2021

N2 - Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment virtual (CT) and optical colonoscopies, to guide navigation towards the anomalies found in pre-treatment scans. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new transitive loss is introduced in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.

AB - Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment virtual (CT) and optical colonoscopies, to guide navigation towards the anomalies found in pre-treatment scans. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new transitive loss is introduced in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.

KW - Colonoscopy

KW - Haustral folds segmentation

UR - https://www.scopus.com/pages/publications/85116422439

U2 - 10.1007/978-3-030-87199-4_21

DO - 10.1007/978-3-030-87199-4_21

M3 - Conference contribution

AN - SCOPUS:85116422439

SN - 9783030871987

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 221

EP - 230

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings

A2 - de Bruijne, Marleen

A2 - Cattin, Philippe C.

A2 - Cotin, Stéphane

A2 - Padoy, Nicolas

A2 - Speidel, Stefanie

A2 - Zheng, Yefeng

A2 - Essert, Caroline

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 27 September 2021 through 1 October 2021

ER -

Mathew S, Nadeem S, Kaufman A. FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos. In de Bruijne M, Cattin PC, Cotin S, Padoy N, Speidel S, Zheng Y, Essert C, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2021 - 24th International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 221-230. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-87199-4_21