Narrow Results

In this paper, an efficient approach for traffic sign detection, classification, and localization is introduced. An integration of the tiling technique with the RetinaFace model and the MobileNetV1-SSD was proposed for traffic detection and classification. The combination of the extraction of the region of interest with a depth estimation model, namely the AANet+, was introduced for the traffic sign localization task. All models were developed based on the transfer learning technique and existing datasets, including the Zalo and ApolloScape datasets. The accuracy and the computational efficiency of the approach are evaluated. Experimental results show that the novel traffic sign detection and classification method outperform the existing ones with an average precision of 77.2%. Moreover, the computing performance achieved is 5 FPS on Jetson Nano and 50 FPS on Jetson Xavier. For the traffic sign localization, the relative error can be reduced to 3.78%, and the computing time is 1.965 s/pair on Jetson Nano and 0.128 s/pair on Jetson Xavier, while the existing method has lower accuracy and is more time-consuming.

A system that builds and maintains a dynamic map for a mobile robot is presented. A learning rule associated to each observed landmark is used to compute its robustness. The position of the robot during map construction is estimated by combining sensor readings, motion commands, and the current map state by means of an Extended Kalman Filter. The combination of landmark strength validation and Kalman filtering for map updating and robot position estimation allows for robust learning of moderately dynamic indoor environments.

Augmented reality (AR) by analyzing the characteristics of the scene, the computer-generated geometric information which can be added to the real environment in the way of visual fusion, reinforces the perception of the world. Three-dimensional (3D) registration is one of the core issues of in AR. The key issue is to estimate the visual sensor’s posture in the 3D environment and figure out the objects in the scene. Recently, computer vision has made significant progress, but the registration based on natural feature points in 3D space for AR system is still a severe problem. There is the difficulty of working out the mobile camera’s posture in the 3D scene precisely due to the unstable factors, such as the image noise, changing light and the complex background pattern. Therefore, to design a stable, reliable and efficient scene recognition algorithm is still very challenging work. In this paper, we propose an algorithm which combines Visual Simultaneous Localization and Mapping (SLAM) and Deep Convolutional Neural Networks (DCNNS) to boost the performance of AR registration. Semantic segmentation is a dense prediction task which aims to predict categories for each pixel in an image when applying to AR registration, and it will be able to narrow the searching range of the feature point between the two frames thus enhancing the stability of the system. Comparative experiments in this paper show that the semantic scene information will bring a revolutionary breakthrough to the AR interaction.

In this paper, we consider a mobile platform with two cameras directed towards the floor. In earlier work, this specific problem geometry has been considered under the assumption that the cameras have been mounted at the same height. This paper extends the previous work by removing the height constraint, as it is hard to realize in real-life applications.

We develop a method based on an equivalent problem geometry, and show that much of previous work can be reused with small modification to account for the height difference. A fast solver for the resulting nonconvex optimization problem is devised. Furthermore, we propose a second method for estimating the height difference by constraining the mobile platform to pure translations. This is intended to simulate a calibration sequence, which is not uncommon to impose. Experiments are conducted using synthetic data, and the results demonstrate a robust method for determining the relative parameters comparable to previous work.

This paper proposes a simultaneous localization and mapping (SLAM)-based markerless mobile-end tracking registration algorithm to address the problem of virtual image drift caused by fast camera motion in mobile-augmented reality (AR). The proposed algorithm combines the AGAST-FREAK-SLAM algorithm with inertial measurement unit (IMU) data to construct a scene map and localize the camera’s pose. The extracted feature points are matched with feature points in a map library for real-time camera localization and precise registration of virtual objects. Experimental results show that the proposed method can track feature points in real time, accurately construct scene maps, and locate cameras; moreover, it improves upon the tracking and registration robustness of earlier algorithms.

The present prevailing loop closure detection algorithm is mainly applicable for simultaneous localization and mapping (SLAM). Its effectiveness is contingent upon environmental conditions, which can fluctuate due to variations in lighting or the surrounding scenario. Vision-based algorithms, while adept during daylight hours, may falter in nocturnal settings. Conversely, lidar methods hinge on the sparsity of the given scenario. This paper proposes an algorithm that comprehensively utilizes lidar and image features to assign weighted factors for loop closure detection based on multi-modal sensor fusion. First, we use $k$-means clustering to produce a point cloud spatial global bag of words. Second, an improved deep learning method is used to train feature descriptors of images while scan context is also used to detect candidate point cloud features. After that, different feature-weighted factors are assigned for homologous feature descriptors. Finally, the detection result related to the maximum weight factor is designated to the optimal loop closure. The adaptive weighted loop closure (AWLC) algorithm we proposed inherits the advantages of different loop closure detection algorithms and hence it is accurate and robust. The AWLC method is compared with popular loop detection algorithms in different datasets. Experiments show that the AWLC can maintain the effectiveness and robustness of detection even at night or in highly dynamic complex environment.

Recently, the methods based on Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in visual place recognition. CNN is a class of multilayer perceptrons, but unlike common multilayer perceptrons that it is not usually fully connected networks. It can acquire more general image features and make the image processing computationally manageable through filtering the connections by proximity. In this paper, we utilize the deep features generated by CNNs and the dynamic time warping (DTW) algorithm for image sequence place recognition. We propose a novel image similarity measurement, which is derived from cosine distance and can better distinguish match and mismatch. Meanwhile, we improve the DTW algorithm to design a local matching method that can reduce time complexity from O(n³) to O(n). To test the proposed method, four datasets (Nordland, Gardens Point, St. Lucia, and UoA datasets) are used as benchmarks; using two traverses in each dataset with one for reference and the other for testing. The results show high precision-recall characteristics of our method in the cases of severe appearance changes. Besides, our method achieves substantial improvements over the methods using the deep feature representations of a single image for recognition, which reflects that the spatiotemporal information contained in the image sequence is significant for the task of visual place recognition. Moreover, the proposed method also shows to outperform the classical sequence-based method SeqSLAM.

Aiming at building versatile humanoid systems, we present in this paper the real-time implementation of behaviors which integrate walking and vision to achieve general functionalities. The paper describes how real-time — or high-bandwidth — cognitive processes can be obtained by combining vision with walking. The central point of our methodology is to use appropriate models to reduce the complexity of the search space. We will describe the models introduced in the different blocks of the system and their relationships: walking pattern, self-localization and map building, real-time reactive vision behaviors, and planning.

This paper describes and compares three different approaches to estimate simultaneous localization and mapping (SLAM) in dynamic outdoor environments. SLAM has been intensively researched in recent years in the field of robotics and intelligent vehicles, many approaches have been proposed including occupancy grid mapping method (Bayesian, Dempster-Shafer and Fuzzy Logic), Localization estimation method (edge or point features based direct scan matching techniques, probabilistic likelihood, EKF, particle filter). In this paper, a number of promising approaches and recent developments in this literature have been reviewed firstly in this paper. However, SLAM estimation in dynamic outdoor environments has been a difficult task since numerous moving objects exist which may cause bias in feature selection problem. In this paper, we proposed a possibilistic SLAM with RANSAC approach and implemented with three different matching algorithms. Real outdoor experimental result shows the effectiveness and efficiency of our approach.

In this paper, a new approach to SLAM is proposed that is based on particle filter and soft computing techniques. In this approach, the robot pose is estimated based on unscented marginal particle filter (UMPF) and the static map is considered as parameters that are updated using soft computing. Significant improvement in the proposed method is observed in terms of accuracy of estimation and consistency compared to conventional methods. A number of simulations and experiments are presented to evaluate the algorithm's performance compared to conventional approaches.

In order to enhance consistency in simultaneous localization and mapping (SLAM), in this paper, this problem is considered as a solely localization problem in the presence of unknown parameters. In this approach, the proposal distribution is generated based on marginal extended particle filter and static map is considered as a parametric estimation that is estimated by maximum likelihood techniques. Significant improvement of the filtering result from this viewpoint is demonstrated in terms of estimation performance and consistency. Some simulations and experiments are presented to evaluate the algorithm’s performance in comparison to conventional methods.

FastSLAM, such as FastSLAM 1.0 and FastSLAM 2.0, is a popular algorithm to solve the simultaneous localization and mapping (SLAM) problem for mobile robots. In real environments, however, the execution speed by FastSLAM would be too slow to achieve the objective of real-time design with a satisfactory accuracy because of excessive comparisons of the measurement with all the existing landmarks in particles, particularly when the number of landmarks is drastically increased. In this paper, an enhanced SLAM (ESLAM) is proposed, which uses not only odometer information but also sensor measurements to estimate the robot’s pose in the prediction step. Landmark information that has the maximum likelihood is then used to update the robot’s pose before updating the landmarks’ location. Compared to existing FastSLAM algorithms, the proposed ESLAM algorithm has a better performance in terms of computation efficiency as well as localization and mapping accuracy as demonstrated in the illustrated examples.

Although in recent years, there have been quite a few studies aimed at the navigation of robots in cluttered environments, few of these have addressed the problem of robots navigating while moving a large or heavy object. Such a functionality is especially useful when transporting objects of different shapes and weights without having to modify the robot hardware. In this work, we tackle the problem of making two humanoid robots navigate in a cluttered environment while transporting a very large object that simply could not be moved by a single robot. We present a complete navigation scheme, from the incremental construction of a map of the environment and the computation of collision-free trajectories to the design of the control to execute those trajectories. We present experiments made on real NAO robots, equipped with RGB-D sensors mounted on their heads, moving an object around obstacles. Our experiments show that a significantly large object can be transported without modifying the robot main hardware, and therefore that our scheme enhances the humanoid robots capacities in real-life situations.

Our contributions are: (1) a low-dimension multi-robot motion planning algorithm that finds an obstacle-free trajectory, by using the constructed map of the environment as an input, (2) a framework that produces continuous and consistent odometry data, by fusing the visual and the robot odometry information, (3) a synchronization system that uses the projection of the robots based on their hands positions coupled with the visual feedback error computed from a frontal camera, (4) an efficient real-time whole-body control scheme that controls the motions of the closed-loop robot–object–robot system.

Unsteady locomotion and the dynamic environment are two problems that block humanoid robots to apply visual Simultaneous Localization and Mapping (SLAM) approaches. Humans are often considered as moving obstacles and targets in humanoid robots working space. Thus, in this paper, we propose a robust dense RGB-D SLAM approach for the humanoid robots working in the dynamic human environments. To deal with the dynamic human objects, a deep learning-based human detector is combined in the proposed method. After the removal of the dynamic object, we fast reconstruct the static environments through a dense RGB-D point clouds fusion framework. In addition to the humanoid robot falling problem, which usually results in visual sensing discontinuities, we propose a novel point clouds registration-based method to relocate the robot pose. Therefore, our robot can continue the self localization and mapping after the falling. Experimental results on both the public benchmarks and the real humanoid robot SLAM experiments indicated that the proposed approach outperformed state-of-the-art SLAM solutions in dynamic human environments.

Accurate pose and trajectory estimates, are necessary components of autonomous robot navigation system. A wide variety of Simultaneous Localization and Mapping (SLAM) and localization algorithms have been developed by the robotics community to cater to this requirement. Some of the sensor fusion algorithms employed by SLAM and localization algorithms include the particle filter, Gaussian Particle Filter, the Extended Kalman Filter, the Unscented Kalman Filter, and the Central Difference Kalman Filter. To guarantee a rapid convergence of the state estimate to the ground truth, the prediction density of the sensor fusion algorithm must be as close to the true vehicle prediction density as possible. This paper presents a Kolmogorov–Smirnov statistic-based method to compare the prediction densities of the algorithms listed above. The algorithms are compared using simulations of noisy inputs provided to an autonomous robotic vehicle, and the obtained results are analyzed. The results are then validated using data obtained from a robot moving in controlled trajectories similar to the simulations.

This work introduces a new complete Simultaneous Localization and Mapping (SLAM) framework that uses an enriched representation of the world based on sensor fusion and is able to simultaneously provide an accurate localization of the vehicle. A method to create an Evidential grid representation from two very different sensors, laser scanner and stereo camera, allows a better handling of the dynamic aspects of the urban environment and a proper management of errors to create a more reliable map, thus having a more precise localization. A life-long layer with high level states is presented, it maintains a global map of the entire vehicle’s trajectory and distinguishes between static and dynamic obstacles. Finally, we propose a method that at each current map creation estimates the vehicle’s position by a grid matching algorithm based on image registration techniques. Results on a real road dataset show that the environment mapping data can be improved by adding relevant information that could be missed without the proposed approach. Moreover, the proposed localization method is able to reduce the drift and improve the localization compared to other methods using similar configurations.

The combination of data from multiple sensors, also known as sensor fusion or data fusion, is a key aspect in the design of autonomous robots. In particular, algorithms able to accommodate sensor fusion techniques enable increased accuracy, and are more resilient against the malfunction of individual sensors. The development of algorithms for autonomous navigation, mapping and localization have seen big advancements over the past two decades. Nonetheless, challenges remain in developing robust solutions for accurate localization in dense urban environments, where the so-called last-mile delivery occurs. In these scenarios, local motion estimation is combined with the matching of real-time data with a detailed pre-built map. In this paper, we utilize data gathered with an autonomous delivery robot to compare different sensor fusion techniques and evaluate which are the algorithms providing the highest accuracy depending on the environment. The techniques we analyze and propose in this paper utilize 3D lidar data, inertial data, GNSS data and wheel encoder readings. We show how lidar scan matching combined with other sensor data can be used to increase the accuracy of the robot localization and, in consequence, its navigation. Moreover, we propose a strategy to reduce the impact on navigation performance when a change in the environment renders map data invalid or part of the available map is corrupted.

In this paper, we present a low-bandwidth centralized collaborative direct monocular SLAM (LCCD-SLAM) for multi-robot systems collaborative mapping. Each agent runs the direct method-based visual odometry (VO) independently, giving the algorithm the advantages of semi-dense point cloud reconstruction and robustness in the featureless regions. The agent sends the server mature keyframes marginalized from the sliding window, which greatly reduces the bandwidth requirement. In the server, we adopt the point selection strategy of LDSO, use the Bag of Words (BoW) model to detect the loop closure candidate frames, and effectively reduce the accumulative drift of global rotation, translation and scale through pose graph optimization. Map matching is responsible for detecting trajectory overlap between agents and merging the two overlapping submaps into a new map. The proposed approach is evaluated on publicly available datasets and real-world experiments, which demonstrates its ability to perform collaborative point cloud mapping in a multi-agent system.

Visual SLAM methods usually presuppose that the scene is static, so the SLAM algorithm for mobile robots in dynamic scenes often results in a significant decrease in accuracy due to the influence of dynamic objects. In this paper, feature points are divided into dynamic and static from semantic information and multi-view geometry information, and then static region feature points are added to the pose-optimization, and static scene maps are established for dynamic scenes. Finally, experiments are conducted in dynamic scenes using the KITTI dataset, and the results show that the proposed algorithm has higher accuracy in highly dynamic scenes compared to the visual SLAM baseline.

This paper evaluates four different Simultaneous Localization and Mapping (SLAM) systems in the context of self-localization of a six-legged robot equipped with a compact RGB-D sensor. Three systems under investigation represent different state of the art approaches to SLAM with RGB-D data, and they are compared to our new approach, called PUT SLAM. We identify problems related to the speciffic data gathered in-motion by a legged robot, and demonstrate robustness and accuracy of our new approach. The SLAM systems are evaluated applying the well-established methodologies, and using a data set which is made public to ensure that our results are verifiable.