Narrow Results

In this paper, an efficient approach for traffic sign detection, classification, and localization is introduced. An integration of the tiling technique with the RetinaFace model and the MobileNetV1-SSD was proposed for traffic detection and classification. The combination of the extraction of the region of interest with a depth estimation model, namely the AANet+, was introduced for the traffic sign localization task. All models were developed based on the transfer learning technique and existing datasets, including the Zalo and ApolloScape datasets. The accuracy and the computational efficiency of the approach are evaluated. Experimental results show that the novel traffic sign detection and classification method outperform the existing ones with an average precision of 77.2%. Moreover, the computing performance achieved is 5 FPS on Jetson Nano and 50 FPS on Jetson Xavier. For the traffic sign localization, the relative error can be reduced to 3.78%, and the computing time is 1.965 s/pair on Jetson Nano and 0.128 s/pair on Jetson Xavier, while the existing method has lower accuracy and is more time-consuming.

Challenges such as complex backgrounds, drastic variations in target scales, and dense distributions exist in natural scenes. Some algorithms optimize multi-scale object detection performance by combining low-level and high-level information through feature fusion strategies. However, these methods overlook the inherent spatial properties of objects and the relationships between foreground and background. To fundamentally enhance the multi-scale detection capability, we propose a depth-constrained multi-scale object detection network that simultaneously learns object detection and depth estimation through a unified framework. In this network, depth features are merged into the detection branch as auxiliary information and constrained and guided to obtain better spatial representations, which enhances discrimination between multi-scale objects. We also introduce a novel cross-modal fusion (CmF) strategy that utilizes depth awareness and low-level detail clues to supplement edge information and adjust attention weight preferences. We find complementary information from RGB and high-quality depth features to achieve better multi-modal information fusion. Experimental results demonstrate that our method outperforms state-of-the-art methods on the KINS dataset, with an improvement of 3.0% in AP score over the baseline network. Furthermore, we validate the effectiveness of our proposed method on the KITTI dataset.

With the widespread adoption of virtual reality and 360-degree video, there is a pressing need for objective metrics to assess quality in this immersive panoramic format reliably. However, existing image quality assessment models developed for traditional fixed-viewpoint content do not fully consider the specific perceptual issues involved in 360-degree viewing. This paper proposes a 360-degree image full-reference quality assessment (FR-IQA) methodology based on a multi-channel architecture. The proposed 360-degree FR-IQA method further optimizes and identifies the distorted image quality using two easily obtained useful saliency and depth-aware image features. The convolutional neural network (CNN) is designed for training. Furthermore, the proposed method accounts for predicting user viewing behaviors within 360-degree images, which will further benefit the multi-channel CNN architecture and enable the weighted average pooling of the predicted FR-IQA scores. The performance is evaluated on publicly available databases to demonstrate the advantages brought by the proposed multi-channel model in performance evaluation and cross-database evaluation experiments, where it outperforms other state-of-the-art ones. Moreover, an ablation study exhibits good generalization ability and robustness.

One of the fundamental problems of machine vision is the estimation of object depth from perceived images. This paper describes both an apparatus and the corresponding algorithms for the passive extraction of object depth. Here passive extraction implies the processing of images acquired using only the existing illumination, in this case roughly uniform white light.

Depth from defocused algorithms are extremely sensitive to image variations. Regularization, the application of a priori constraints, is employed to improve the accuracy of the range measurements. When the camera’s point spread function is shift invariant, an adaptive algorithm is developed in the frequency domain. The constraints imposed upon the solution power spectrum vector vary temporally. When the camera’s point spread function is shift varying, an adaptive algorithm is developed in the spatial domain. The constraints imposed upon the solution point spread vector vary spatially.

Data is acquired from line scan cameras. Only a single range measurement or a single depth profile is extracted. By relying upon the motion of the observed object on a conveyor belt, a complete range image may be generated.

This paper proposes an object based vision (O-BV) system to implement visual servoing for autonomous mobile manipulators using two charge-coupled device (CCD) cameras. Conventional stereo vision (C-SV) system estimates the depth based on the disparity between two camera images for the same object. However, the disparity is not an effective cue for a small disparity at a long distance. To resolve this problem, in the proposed O-BV system, the individual camera tracks the object independently, and the angles of the two cameras are used to estimate the distance to the object. This depth estimation technique is applied for an autonomous mobile robot to approach to a target object precisely. The O-BV system is experimentally compared to the C-SV system in terms of computing time and depth estimation accuracy. Also the two cameras which are attached on the top of the autonomous mobile manipulator have been utilized for the mobile manipulator to approach to a target object precisely through the visual servoing. Through the experiments, it is demonstrated that the fast and precise depth estimation is a critical factor for the successful visual servoing.

This paper presents different image processing methods and algorithms, which are needed to enable the reliable automation of nanohandling processes. These applications use the scanning electron microscope (SEM) as a visual sensor. SEMs are widespread and powerful tools for manipulations on the nanoscale. Due to the timing constraints in automated setups, the trade-off between SEM scanning speed and image quality is a concern for algorithm development. Tasks to be fulfilled on image data provided by the SEM include object recognition, object tracking and depth estimation. A selection of algorithms that have been applied in automated setups for nanomanipulation is discussed and validated.

In this paper, we propose a hierarchical attention-based sensor fusion strategy for depth estimation under various weather conditions. Multiple-sensor fusion is proven as a promising solution for predicting accurate depth maps in diverse weather conditions, especially for extreme weather conditions. However, most of the current studies simply fuse the information from different sensors without jointly considering the difference in their performance at the sensor level and feature level. To fill this gap, our hierarchical attention-based fusion strategy uses two attention mask-generation modules that weigh sensor data from branches (i.e. different sensors) and features. With the cooperation of these two masks, our system is able to determine the adaptive contribution of each sensor as well as the individual contribution of each feature in the sensor regarding their performance in different weather. We compare the proposed methods with the baseline, i.e. the late fusion Sparse-to-Dense model, and two extended models individually with the branch-wise-only and feature-wise-only masks. The results show a robust and superior performance of our methods even in clear environments where the baseline already performs well enough. Moreover, we investigate the performance of RGB camera, radar, and LiDAR in foggy environments comprehensively by visualizing the generated mask. Our results show a significantly increased importance of radar sensors in extreme weather conditions, e.g. dense fog.

In order to correctly identify the motor unit action potential trains (MUAPTs) in estimated discharges from dynamic surface electromyogram (EMG), an approach for accurate classification of motor unit (MU) discharges is presented. First, the estimated discharges are obtained manually, then the estimated discharges are classified as MUAPTs based on the MU location, which combines the MU depth with the MU plane position. During verification in dynamic muscle contractions, the advanced tripole model is introduced. At SNRs of 10, 20 and 30dB, the MUAPTs were identified with true positive rate (TPR) of 91.1$\pm$5.5%, 95.2$\pm$3.7% and 96.1$\pm$2.9%. The results also show that the MU location can be used as a simple method for identifying MUAPT from estimated discharges and selecting reliably decomposed discharges. The newly introduced method is a robust and reliable indicator of MUAPT identification accuracy.

One of the fundamental problems of machine vision is the estimation of object depth from perceived images. This paper describes both an apparatus and the corresponding algorithms for the passive extraction of object depth. Here passive extraction implies the processing of images acquired using only the existing illumination, in this case roughly uniform white light.

Depth from defocused algorithms are extremely sensitive to image variations. Regularization, the application of a priori constraints, is employed to improve the accuracy of the range measurements. When the camera's point spread function is shift invariant, an adaptive algorithm is developed in the frequency domain. The constraints imposed upon the solution power spectrum vector vary temporally. When the camera's point spread function is shift varying, an adaptive algorithm is developed in the spatial domain. The constraints imposed upon the solution point spread vector vary spatially.

Data is acquired from line scan cameras. Only a single range measurement or a single depth profile is extracted. By relying upon the motion of the observed object on a conveyor belt, a complete range image may be generated.

In this paper we investigate the problem of driving a small sized autonomous ground vehicle in unstructured outdoor environments using only a single camera input. The used method is based on depth perception that estimates the distances of the obstacles in front of the vehicle. We apply an offline dimensionality reduction to minimize the extracted data from the images to accomplish the navigation task in real time. The autonomous vehicle is able to traverse an outdoor path approximately 110 m succesfully.