PhD Thesis: Novel Geometric Constraints for 3D Computer Vision Applications

Abstract

Geometric constraints on feature points has a long-standing history in 3D Computer Vision among experts. In contrast, we aim to study higher-level primitives other than points – such as lines, planes and ellipsoids – and create novel geometric constraints that allow us to express interesting restrictions and solve new 3D Vision problems. Towards this end, we explore a variety of applications in which we are able to apply geometrical priors using these primitives and find innovative solutions. Thus, we focus on three main components:

1. Lines Constraints. We formulate a new research topic – called the privacy preserving image-based localization problem – and present the first solution by introducing the notion of 3D Line Clouds. It will be shown that this new representation hides the content of the scene and thereby protects user privacy, yet still provides sufficient geometric constraints to enable robust and accurate 6-DOF camera pose estimation from feature correspondences.

2. Planes Constraints. We introduce a novel geometric constraint using planes as a reflective symmetry prior for variational 3D surface reconstruction. Our method leverages symmetry information directly within a 3D reconstruction procedure in order to complete or denoise symmetric surface regions which have been partially occluded or where the input information has low quality. Opposed to the majority of 3D surface reconstruction methods which fit mini- mal surfaces in order to fill unobserved surface parts, our method favors solutions which align with symmetries and adhere to required smoothness properties at the same time. Similarly to how humans extrapolate occluded areas and 3D information from just a few view points, our method can hallucinate entire scene parts in unobserved areas, fill small holes, or denoise observed surface geometry once a symmetry has been detected.

3. Ellipsoid and LMI Constraints. We propose a general global optimization framework for consensus maximization with Linear Matrix Inequality (LMI) constraints. Moreover, we derive several LMI constraints and demonstrate that a number of central computer vision problems can be cast into this form. In particular, we develop a method for the geometric registration of semantically labeled regions. We approximate these semantic regions with ellipsoids, and leverage their convexity to formulate the correspondence search effectively as a constrained global optimization problem that maximizes the number of matched regions. To this end, we also show that ellipsoid-to-ellipsoid assignments can be described as LMI restriction. Our approach is robust to large percentages of outliers and thus applicable to difficult correspondence search problems.

For all contributions, we present real and synthetic experiments that validate each method and provide insights into their usefulness w.r.t. the state-of-the-art.