D⁴Recon: Dual-Stage Deformation and Dual-Scale Depth Guidance for Endoscopic Reconstruction

Deformable tissue reconstruction in endoscopy is vital for surgery, yet current methods struggle with high-fidelity reconstruction of irreversible tissue deformations. To this end, we present D⁴Recon, a novel framework for real-time and high-fidelity endoscopic reconstruction, addressing crucial challenges in surgical applications. A The image shows a stylized letter "D" in a light blue color.ual-stage The image shows a stylized, light blue letter "D" with a subtle glow effect, set against a white background.eformation modeling and a A blue, pixelated letter "D" on a white background. The letter is stylized with a blocky, digital appearance, resembling a symbol or icon.ual-scale A stylized letter "D" in a light blue color with a subtle glow effect, set against a white background. The design emphasizes the letter's shape and color, creating a visually striking symbol.epth guidance ( The image shows a mathematical notation with the letter "D" followed by a superscript "4". This represents D^4 , indicating the fourth power of D.) are proposed in a dynamic 3D Gaussian Splatting paradigm along with lightweight multi-layer perception (MLP) to model dynamics in endoscopic scenes. In the dual-stage deformation modeling, we introduce a spatial deformation model to correct multiview inconsistencies, accompanied by a temporal deformation model that accurately represents tissue distortion and dynamic tissue interaction with surgical tools in the reference frames. In the dual-scale depth guidance, we propose to balance local error correction with absolute depth consistency, enabling precise depth refinement while preserving fine-grained color accuracy. D⁴Recon generates accurate 3D reconstructions with superior PSNR, SSIM, and LPIPS scores, outperforming existing methods in terms of geometric coherence and photorealism with real-time rendering speed, as demonstrated by extensive experiments on diverse endoscopic datasets. Reconstruction videos are in the supplementary file. Website.