This dissertation addresses the problem of reconstruction of the 3D structure of colon segments and the 3D motion of an endoscope from colonoscopy images. Colonoscopy is currently the preferred screening modality for early colorectal cancer (CRC) detection, claiming about 50,000 annual deaths in the US. During colonoscopy, the endoscopist performs diagnosis and/or treatment based on images of the interior of the human colon taken by the camera at the tip of the endoscope.

The reconstruction of the virtual colon and the endoscope motion from colonoscopy images has the potential to (1) improve endoscopist's understanding of the colon structure of the patient and the endoscopist's navigation technique at the time of colonoscopy; (2) determine the estimated area and location of the colon mucosa uninspected by the endoscopist; domain experts estimated that as much as 95% of the colon mucosa should be inspected; and (3) bridge the gap between virtual colonoscopy and optical colonoscopy. Besides our previous work, we found no other techniques that address the same problem in the literature. We propose a set of new algorithms and evaluate them using images inside an Olympus synthetic colon as well as colonoscopy images from real colonoscopy procedures. Edges of colon fold contours are first detected and processed to generate the wire frame of the reconstructed virtual colon. We propose a colon fold contour estimation algorithm using a single colonoscopy image. We introduce Depth-from-Intensity---the depth and shape estimation of colon folds using brightness intensity of pixels. We introduce an algorithm to estimate the amount of protrusion and width of colon folds and interpolate the folds and the colon wall to connect the folds in 3D space. We present our new method for reconstructing the virtual colon from sequential colonoscopy images and deriving corresponding camera motions. We discuss two quality metrics derived using the information from the reconstruction: spiral score and percentage of areas uninspected. Spiral score is found strongly correlated with the ground truth visualization quality on 159 colonoscopy videos. The percentage of areas uninspected metric is of great interest to domain experts.

Broader impact: The proposed work has potential to improve the amount of the colon mucosa inspected in clinical practice; thereby, potentially increasing the protective effect of colonoscopy against CRC. The technology may change the way colonoscopy is performed in routine practice and may be used for training new endoscopists. Finally, the algorithm design will add to existing knowledge in endoscopy video processing and may be beneficial for other endoscopy procedures.