Learning Image Manifolds Using Functional and Shape Analysis

Music City Center 

Machine learning and AI methods have been truly impressive in their performance on image and computer vision tasks. What are the reasons for this success? One reason is that despite image data being ultra-high-dimensional, most images lie on very low-dimensional manifolds. We speculate that AI methods can learn and exploit geometries of these low-dimensional manifolds to result in efficient procedures for vision tasks. In this paper, we investigate manifolds formed by images of 3D objects using tools from functional and shape analysis. First, we take individual 3D objects, say a chair, a car, or a sofa, and we form their pose image manifold -- the set of images formed by all 3D rotations of that object. To visualize and analyze these pose manifolds, we use a geometry-preserving transformation (e.g., the well-known multi-dimensional scaling) to map them to a smaller Euclidean space called the latent space. In the smaller latent space, we study the shapes of these image manifolds and compare them across different 3D objects. For example, allowing only 1D rotation of objects, we get curves (parameterized by the rotation angle). Allowing 2D rotations, we get surfaces (parameterized by two rotation angles). For complete 3D rotations, we get hypersurfaces (parameterized by three rotation angles) in latent spaces. We study the shapes of these functions (curves, surfaces, and hypersurfaces) using Kendall's shape analysis and obtain some clustering results.