Narrow Results

To improve recognition results, decisions of multiple neural networks can be aggregated into a committee decision. In contrast to the ordinary approach of utilizing all neural networks available to make a committee decision, we propose creating adaptive committees, which are specific for each input data point. A prediction network is used to identify classification neural networks to be fused for making a committee decision about a given input data point. The jth output value of the prediction network expresses the expectation level that the jth classification neural network will make a correct decision about the class label of a given input data point. The proposed technique is tested in three aggregation schemes, namely majority vote, averaging, and aggregation by the median rule and compared with the ordinary neural networks fusion approach. The effectiveness of the approach is demonstrated on two artificial and three real data sets.

Pedestrian classification is of increased interest to autonomous transportation systems due to the development of deep convolutional neural networks. Despite recent progress on pedestrian classification, it is still challenging to identify individuals who are partially occluded because of the diversity of the occluded parts, variation in pose, and appearance. This causes a significant performance reduction when pedestrians are covered by other objects, and feature information is lost due to the occluded parts. To solve this problem, we propose two network architectures using tree structure convolutional neural networks (T-CNN). They use the structural representation of multi-branch deep convolutional features, with the advantages of its end-to-end learning process. The high-level tree structure CNN (HT-CNN) architecture aims to concatenate the output of the classification layer from multi-segmented patches of pedestrians to handle partially occluded problems. The low-level tree structure CNN (LT-CNN) concatenates the discriminative features from each multi-segmented patch and global features. Our T-CNN architecture with a high-level tree structure performed with 94.64% accuracy on the INRIA dataset without occlusions, and with 70.78% accuracy on the Prince of Songkla University (PSU) dataset with occlusions, outperforming a baseline CNN architecture. This indicates that our proposed architecture can be used in a real-world environment to classify the occluded part of pedestrians with the visual information of multi-segmented patches using tree-structured multi-branched CNN.

Aiming at the existing multimodal image sentiment analysis methods that suffer from insufficient text feature extraction, redundant independent modal features, insufficient analysis of semantic relationship between image and text data, and insufficient fusion, a multimodal image sentiment analysis method that integrates RoBERTa, Res-ViT and bimodal attention mechanism is proposed. By utilizing RoBERTa model to segment the input text, encode and extract high-level semantic features, local and global features of the image are obtained by using Res-ViT, and finally, image and text features are deeply fused to interact with each other by bimodal attention fusion. The model comprehensively mines the text and image features and deeply fuses the text–image features to interact, which enhances the accuracy of sentiment classification. The experimental outcomes demonstrate that the suggested RRMGA can enhance the accuracy of sentiment classification by attaining ACC and $F1$ values of 0.7164 and 0.7042 on the MVSA-Multiple dataset, and 0.7519 and 0.7428 on the TumEmo dataset, respectively. These values are significantly higher than those of a number of other state-of-the-art multimodal image sentiment analysis approaches.

Vascular tissue characterization is of great importance concerning the possibility of an Acute Cardiac Syndrome (ACS). Gray-scale intravascular ultrasound (IVUS) is a powerful tomographic modality providing a thorough visualization of coronary arteries. Among the existing methods, virtual histology (VH) is the most popular and clinically available technique for plaque component analysis, it suffers however from a poor longitudinal resolution. In order to surmount this demerit, a new image-based methodology for plaque assessment is suggested here that differentiates tissue components into four classes: calcium, necrotic core, fibrous and fibro-lipid. A rich set of five textural feature families are extracted from IVUS images, computed at different scales. The main contribution of this paper is that tissue classification is accomplished using the principles of multiple classifiers combination approach. At the first stage, an ensemble of base SVM classifiers is constructed from each feature family, separately. The fuzzy outputs of the individual classifiers are then aggregated to provide the final fused results. We investigate four efficient decision fusion schemes of the literature and the SVM fuser. Extensive experimentation is carried out to highlight the merits of the suggested schemes against single SVM classifiers that use reduced feature subsets obtained after feature selection or the entire feature space. The analysis demonstrates that the decision fusion techniques offer improved classification accuracies, compared to single SVM classifiers and existing methods in IVUS imaging. In addition, the method provides accurate assessments of plaque composition in IVUS images.

In this paper, an off-line method, based on hidden Markov model, HMM, is used for holistic recognition of handwritten words of a limited vocabulary. Three feature sets based on image gradient, black–white transition and contour chain code are used. For each feature set an HMM is trained for each word. In the recognition step, the outputs of these classifiers are combined through a multilayer perceptron, MLP. High number of connections in this network causes a computational complexity in the training. To avoid this problem, a new method is proposed. In the experiments on 16000 images of 200 names of Iranian cities, from “Iranshahr 3” dataset, the results of the proposed method are presented and compared with some similar methods. An error analysis on these results is also provided.

This paper presents a novel method for recognizing human daily activity by fusion multiple sensor nodes in the wearable sensor systems. The procedure of this method is as follows: firstly, features are extracted from each sensor node and subsequently reduced in dimension by generalized discriminant analysis (GDA), to ensure the real-time performance of activity recognition; then, the reduced features are classified with the multiclass relevance vector machines (RVM); finally, the individual classification results are fused at the decision level, in consideration that the different sensor nodes can provide heterogeneous and complementary information about human activity. Extensive experiments have been carried out on Wearable Action Recognition Database (WARD). Experimental results show that if all the five sensor nodes are fused with the adaptive weighted logarithmic opinion pools (WLOGP) fusion rule, we can even achieve a recognition rate as high as 98.78%, which is far more better than the situations where only single sensor node is available or the activity data is processed by state-of-the-art methods. Moreover, this proposed method is flexible to extension, and can provide a guideline for the construction of the minimum desirable system.

This article describes the joint measures method as a new powerful method for the development of a high performance multi-sensor data/image fusion scheme at the decision level. The images are received from distributed multiple sensors, which sense the targets in different spectral bands including visible, infrared, thermal and microwave. At first, we study the decision fusion methods, including voting schemes, rank based algorithm, Bayesian inference, and the Dempster-Shafer method. Then, we extract the mathematical properties of multi-sensor local classification results and use them for modeling of the classifier performances by the two new measures, i.e. the plausibility and correctness. Then we establish the plausibility and correctness distribution vectors and matrices for introducing the two improvements of the Dempster-Shafer method, i.e. the DS (CM) and DS (PM) methods. After that we introduce the joint measures decision fusion method based on using these two measures jointly. The Joint Measures Method (JMM) can deal with any decision fusion problem in the case of uncertain local classifiers results as well as clear local classifiers results. Finally, we deploy the new and previous methods for the fusion of the two different sets of multispectral image classification local results and we also compare their reliabilities, the commission errors and the omission errors. The results obviously show that the DS (PM), DS (CM) and JMM methods which use the special properties of the local classifiers and classes, have much better accuracies and reliabilities than other methods. In addition, we show that the reliability of the JMM is at least 3% higher than all other methods.

In this paper, we would focus on submitting a new decision fusion method based on multiple sensors' behaviors applying to target detection and identification in a network of distributed sensors. Each sensor has its own reliability, error rate and output data. Hence, in a processing and decision-making center in which target data are received from different sensors and sources, correctness and speed of final decision-making depend on data fusion method. The extraction, modeling and weighing of long-time and temporary behavior functions of each data source and using precise and fast decision making/fusion method are the main purpose of this article. After the introduction, we try to consider the data fusion method in decision level, such as voting schemes, rank based method and Bayesian inference. Hence, in a distributed target detection and identification system, we explain the specific and the functional features model of each source using long-time and temporary behavior functions. So we introduce the behavior based method as a new decision fusion method based on long-time and temporary behaviors of local decision makers. Therefore, we will observe that the behavior based method results, which pointed both to the temporal and the long time behaviors of the input decision makings, are very much nearer to reality and its correctness in target identification is much higher than the other methods. Examples are given corresponding to the target detection and identification systems to compare the new method with the other methods are shown that the behavior based method has its own exclusive capability in target detection, identification and producing final decision without ambiguity.

Decision fusion of multiple classifiers can obtain more accurately classification than the best single classifier. By applying multiple classifiers fusion and clustering analysis and the nearest neighbor rule respectively, this paper presents a new intrusion detection algorithm. The analysis and experiment results show that this algorithm can achieve a good detection performance and reduce remarkably the errors and false alarms for intrusion detection.