Extending Tempcyclegan for Virtual Augmentation of Gastrointestinal Endoscopy Training Simulators

Simulator training is a core part of gastrointestinal (GI) endoscopy training. Based on rubber or silicon dummies, training lacks a realistic visual appearance. Aiming at a hyperrealistic training environment, we propose a CycleGAN-based framework to translate the training videos into realistically appearing GI endoscopy videos. We build on the concept of tempCycleGAN and (i) extend it to a generic framework to simultaneously work on n subsequent video frames in order to increase temporal consistency of generated videos and (ii) formulate a conditional variant of it to selectively incorporate pathologies into the generated videos. Extension (i) will be shown to increase temporal consistency and realism of the generated GI endoscopy videos. Feasibility and potential of (ii) is illustrated with superficial and deep duodenal ulcer as conditional classes.