Solving the correspondence problem with 4D sensor information
Spatial perception is often a prerequisite for safe motion and interaction in an environment. This includes not only determining one's own position, but also the ability to estimate the correct distance of surrounding objects. In computer vision, depth information can be extracted from stereoscopic images. For this purpose, recognition algorithms identify points, corners, or patterns as features in images. By matching corresponding features in stereoscopic image data, depth information can be estimated and transferred to a disparity map. This makes it possible to render point clouds that spatially represent an environment with numerous details consisting of colors, textures, contrasts, but also geometric properties, and thus ensure safe motion and interaction. The processing of stereoscopic images into a point cloud is a central aspect of the correspondence problem. It determines the assignment of comparable features in image pairs and provides the foundation for a robust calculation of depth information. The correspondence problem can occur due to insufficient perspective mapping, positional inaccuracies, and spatial distortions, and thus influences the quality of the rendered point cloud. This work presents an approach to solving the correspondence problem. The idea is to leave the Euclidean space to extract depth information from time-related sensor and image data. Instead, topological information is transferred to a four-dimensional (4D) space containing geometrical properties of space-time. In this 4D space, temporally defined position and depth information can be viewed in ten degrees of freedom as a metric unit of a complex tensor field. In addition to the actual color values of each image pixel, other physical sensor data, such as earth-specific magnetic fields, pressures, accelerations, and forces, can be added to the tensor field to include the sensory relationships when solving the correspondence problem. By using 4D sensor information, dynamic influences and dependencies of a relativistic view can be included in the metric calculation of depth information. Thus, correspondence-related shifts or dynamically occurring imaging errors can be corrected. The 4D view of information in the context of relativistic image processing offers a novel and innovative field of research for dynamic applications that involve position and depth estimation. The introduced 4D architecture offers a transferable approach for the calculation of spatial information and can be extended by different sensors. The correlations of 4D information presented in relativistic image processing provide a new foundation for future approaches to improve spatial perception in the areas of mobility and navigation.