Endosensorfusion: Particle filtering - based multi - sensory data fusion with switching state - space model for endoscopic capsule robots using recurrent neural network kinematics

Abstract

Ingestible wireless capsule endoscopy is an emerging minimally invasive diagnostic technology for inspection of the gastrointestinal (GI) tract and diagnosis of a wide range of diseases and pathologies. Medical device companies and many research groups have recently made substantial progresses in converting passive capsule endoscopes to active capsule robots, enabling more accurate, precise, and intuitive detection of the location and size of the diseased areas. A reliable, real time multi-sensor fusion functionality is crucial for localization of actively controlled next-generation endoscopic capsule robots. In this study, we propose a novel multi-sensor fusion approach based on switching observations model using non-linear kinematics learned by recurrent neural networks for real-time endoscopic capsule robot localization. Our method concerns the sequential estimation of a hidden state vector from noisy pose observations delivered by multiple sensors, a 5 degree-of-freedom (5-DoF) absolute pose measurement by a magnetic localization system and a 6-DoF relative pose measurement by visual odometry. For the inference of the model, Sequential Monte Carlo (SMC) methods known as particle lters are employed, which are e ective for on-line inference. In addition, the proposed method is capable of detecting and handling sensor failures by ignoring corrupted data, providing the robustness of a medical device. Detailed analyses and evaluations made using ex-vivo experiments on a porcine stomach model prove that our system achieves high translational and rotational accuracies for di erent types of endoscopic capsule robot trajectories.