Narrow Results

We apply compensated convex transforms to define a multiscale Hausdorff stable method to extract intersections between smooth compact manifolds represented by their characteristic functions or as point clouds embedded in ℝⁿ. We prove extraction results on intersections of smooth compact manifolds and for points of high curvature. As a result of the Hausdorff–Lipschitz continuity of our transforms, we show that our method is stable against dense sampling of smooth manifolds with noise. Examples of explicitly calculated prototype models for some simple cases are presented, which are also used in the proofs of our main results. Numerical experiments in two- and three-dimensional space, and applications to geometric objects are also shown.

As sharp feature manipulation plays an important role in point clouds processing, a novel mean curvature flow-based framework for sharp feature extraction from point clouds is presented in this paper. That is, for input point clouds, a general purpose mean curvature flow-based point clouds smoothing operator is applied on them, thereby, obtaining a smoothing version of the original point clouds. The sharp feature points are labeled as points whose displacements between original point clouds and their smoothing version get local extreme. Implementation of our method on both synthesized and real scanned point clouds show that our methods are effective and robust for the purpose of sharp features extraction tasks.

One of the key tasks for autonomous vehicles or robots is a robust perception of their 3D environment, which is why autonomous vehicles or robots are equipped with a wide range of different sensors. Building upon a robust sensor setup, understanding and interpreting their 3D environment is the next important step. Semantic segmentation of 3D sensor data, e.g. point clouds, provides valuable information for this task and is often seen as key enabler for 3D scene understanding. This work presents an iterative deep fusion architecture for semantic segmentation of 3D point clouds, which builds upon a range image representation of the point clouds and additionally exploits camera features to increase accuracy and robustness. In contrast to other approaches, which fuse lidar and camera features once, the proposed fusion strategy iteratively combines and refines lidar and camera features at different scales inside the network architecture. Additionally, the proposed approach can deal with camera failure as well as jointly predict lidar and camera segmentation. We demonstrate the benefits of the presented iterative deep fusion approach on two challenging datasets, outperforming all range image-based lidar and fusion approaches. An in-depth evaluation underlines the effectiveness of the proposed fusion strategy and the potential of camera features for 3D semantic segmentation.

In this paper, we propose a new approach for learning shape implicit neural representations (INRs) from point cloud data that do not require normal vectors as input. We show that our method, which uses a soft constraint on the divergence of the distance function to the shape’s surface, can produce smooth solutions that accurately orient gradients to match the unknown normal at each point, even outperforming methods that use normal vectors directly. This work extends the latest work on divergence-guided sinusoidal activation INRs [Y. Ben-Shabat, C. H. Koneputugodage and S. Gould, Proc IEEE/CVF Conf Computer Vision and Pattern Recognition, 2022, pp. 19323–19332], to Gaussian activation INRs and provides extended theoretical analysis and results. We evaluate our approach on tasks related to surface reconstruction and shape space learning.

Understanding 3D scenes, such as semantic segmentation and instance identification within point clouds, typically demands extensive annotated datasets. However, generating point-by-point labels is an overly laborious process. While recent techniques have been developed to train 3D networks with a minimal fraction of labeled points, our method, dubbed “One Thing One Click,” simplifies this by requiring just a single label per object. To effectively utilize these sparse annotations during network training, we’ve crafted an innovative self-training strategy. This involves alternating between training phases and label spreading, powered by a graph propagation module. Additionally, we integrate a relation network to create category-specific prototypes, improving pseudo label accuracy and steering the training process. Our approach also seamlessly integrates with 3D instance segmentation, incorporating a point-clustering technique. Our method demonstrates superior performance over other weakly supervised strategies for 3D semantic and instance segmentation, as evidenced by tests on both ScanNet-v2 and S3DIS datasets. Remarkably, the efficacy of our self-training method with limited annotations rivals that of fully supervised models. Codes and models are available at https://github.com/liuzhengzhe/One-Thing-One-Click.

Deep neural networks for 3D point cloud classification, such as PointNet, have been demonstrated to be vulnerable to adversarial attacks. Current adversarial defenders often learn to denoise the (attacked) point clouds by reconstruction, and then feed them to the classifiers as input. In contrast to the literature, we propose a novel bilevel optimization framework for robust point cloud classification, where the internal optimization can effectively defend the adversarial attacks as denoising and the external optimization can learn the classifiers accordingly. As a demonstration, we further propose an effective and efficient instantiation of our approach, namely, Lattice Point Classifier (LPC), based on structured sparse coding in the permutohedral lattice and 2D convolutional neural networks (CNNs) that integrates both internal and external optimization problems into end-to-end trainable network architectures. We demonstrate state-of-the-art robust point cloud classification performance on ModelNet40 and ScanNet under seven different attackers. Our demo code is available at: https://github.com/Zhang-VISLab.

The understanding of the environment is mandatory for any type of autonomous robot. The ability to put semantics on self-generated sensor data is one of the most challenging tasks in robotics. While navigation tasks can be performed by pure geometric knowledge, high-level planning and intelligent reasoning can only be done if the gap between semantic and geometric representation is narrowed. In this paper, we introduce our approach for recovering 3D scene information from unorganized point clouds, generated by a tilting laser range scanner in a typical indoor environment. This unorganized information has to be analyzed for geometric and recognizable structures so that a robot is able to understand its perception. We discuss in this paper how this spatial information, which is based solely on segmented shapes and their extractable features, can be used for semantic interpretation of the scenery. This will give an idea of how the gap between semantic and spatial representation can be solved by spatial reasoning and thereby increasing robot autonomy.