Depth-based scene mapping through spatio-temporal knowledge integration

Abstract

This thesis is concerned with scene mapping by a mobile robot using point cloud data. It is a complex process that requires the robot to segment the incoming data, represent it compactly and e ciently, and then use the resulting knowledge in its learning and decision-making. Segmentation enables the robot to determine the point cloud object candidates. The robot bases its learning and reasoning on the detected segments. Range sensors, such as LIDAR, are essential for a robot to extract environmental information. However, they generally create sparse data. For this reason, the sparse data should be considered specially. A novel approach to segmentation is proposed based on an extension of density-based clustering in the spherical coordinate system. We present the deformable sphere approximation (DSA) descriptor as a novel 3D descriptor that encodes point cloud objects. Experimental results show that our representation method is capable of classifying the objects. Finally, we consider how the robot can use all knowledge available to it. We propose an approach in which the robot also considers the knowledge accumulated through tracking the objects' temporal continuity. For this, we propose the temporal deformable sphere approximation (TDSA) descriptor. Its construction requires the robot to track object candidates. For this, we propose a novel multi-tracking approach based on combining Kalman Filtering and multi-object matching considering position and shape similarity. We then compare the various schemes the robot can use in order to utilize the resulting knowledge. Our experimental results show that the T-DSA descriptor improves the classi cation performance compared to only the instantaneous DSA descriptors. As such, the robot is able to build and evolve a scene map as it is navigating in it.