International Journal of Pattern Recognition and Artificial Intelligence

Video action recognition relies heavily on the way spatio-temporal cues are combined in order to enhance recognition accuracy. This issue can be addressed with explicit modeling of interactions among objects within or between videos, such as the graph neural network, which has been shown to accurately model and represent complicated spatial- temporal object relations for video action classification. However, the visual objects in the video are diversified, whereas the nodes in the graphs are fixed. This may result in information overload or loss if the visual objects are too redundant or insufficient for graph construction. Segment level graph convolutional networks (SLGCNs) are proposed as a method for recognizing actions in videos. The SLGCN consists of a segment-level spatial graph and a segment-level temporal graph, both of which are capable of simultaneously processing spatial and temporal information. Specifically, the segment-level spatial graph and the segment-level temporal graph are constructed using 2D and 3D CNNs to extract appearance and motion features from video segments. Graph convolutions are applied in order to obtain informative segment-level spatial-temporal features. A variety of challenging video datasets, such as EPIC-Kitchens, FCVID, HMDB51 and UCF101, are used to evaluate our method. In experiments, it is demonstrated that the SLGCN can achieve performance comparable to the state-of-the-art models in terms of obtaining spatial-temporal features.

Challenges such as complex backgrounds, drastic variations in target scales, and dense distributions exist in natural scenes. Some algorithms optimize multi-scale object detection performance by combining low-level and high-level information through feature fusion strategies. However, these methods overlook the inherent spatial properties of objects and the relationships between foreground and background. To fundamentally enhance the multi-scale detection capability, we propose a depth-constrained multi-scale object detection network that simultaneously learns object detection and depth estimation through a unified framework. In this network, depth features are merged into the detection branch as auxiliary information and constrained and guided to obtain better spatial representations, which enhances discrimination between multi-scale objects. We also introduce a novel cross-modal fusion (CmF) strategy that utilizes depth awareness and low-level detail clues to supplement edge information and adjust attention weight preferences. We find complementary information from RGB and high-quality depth features to achieve better multi-modal information fusion. Experimental results demonstrate that our method outperforms state-of-the-art methods on the KINS dataset, with an improvement of 3.0% in AP score over the baseline network. Furthermore, we validate the effectiveness of our proposed method on the KITTI dataset.

With the rapid development of computer vision and artificial intelligence technology, visual object detection has made unprecedented progress, and small object detection in complex scenes has attracted more and more attention. To solve the problems of ambiguity, overlap and occlusion in small object detection in complex scenes. In this paper, a multi-scale fusion feature enhanced path aggregation network MSFE-PANet is proposed. By adding attention mechanism and feature fusion, the fusion of strong positioning information of deep feature map and strong semantic information of shallow feature map is enhanced, which helps the network to find interesting areas in complex scenes and improve its sensitivity to small objects. The rejection loss function and network prediction scale are designed to solve the problems of missing detection and false detection of overlapping and blocking small objects in complex backgrounds. The proposed method achieves an accuracy of 40.7% on the VisDrone2021 dataset and 89.7% on the PASCAL VOC dataset. Comparative analysis with mainstream object detection algorithms proves the superiority of this method in detecting small objects in complex scenes.

Pedestrian classification is of increased interest to autonomous transportation systems due to the development of deep convolutional neural networks. Despite recent progress on pedestrian classification, it is still challenging to identify individuals who are partially occluded because of the diversity of the occluded parts, variation in pose, and appearance. This causes a significant performance reduction when pedestrians are covered by other objects, and feature information is lost due to the occluded parts. To solve this problem, we propose two network architectures using tree structure convolutional neural networks (T-CNN). They use the structural representation of multi-branch deep convolutional features, with the advantages of its end-to-end learning process. The high-level tree structure CNN (HT-CNN) architecture aims to concatenate the output of the classification layer from multi-segmented patches of pedestrians to handle partially occluded problems. The low-level tree structure CNN (LT-CNN) concatenates the discriminative features from each multi-segmented patch and global features. Our T-CNN architecture with a high-level tree structure performed with 94.64% accuracy on the INRIA dataset without occlusions, and with 70.78% accuracy on the Prince of Songkla University (PSU) dataset with occlusions, outperforming a baseline CNN architecture. This indicates that our proposed architecture can be used in a real-world environment to classify the occluded part of pedestrians with the visual information of multi-segmented patches using tree-structured multi-branched CNN.

As an infrastructure of biochemical laboratories, EP tube label plays a significant role in information extraction to meet the limitations of computing power in offline devices and solve the problem that the EP tube label cannot be accurately identified before identification because the label belongs to multi-angle random placement. This paper proposes a light-weight neural network YOLOv4-tiny-ECA to position tubes and a tilt correction method based on Hough transform. First, the EP tube rack is roughly positioned based on the diffuse filling algorithm combined with digital morphological corrosion, and then the EP tubes in the rack are precisely positioned using the light-weight YOLO target detection algorithm combined with the attention mechanism. Next, the baseline is added to the label as the basis for determining the tilt angle. For the valid target, the baseline is extracted using the Hough transform and the tilt angle is calculated by vector fork multiplication. Finally, baseline is removed using image processing algorithm for better recognition results. Our results show that the light-weight YOLO algorithm reduces the network parameters by 56% and computation by 55% while keeping the accuracy rate largely unchanged, the offline positioning tilt correction method can achieve 98.8% accuracy and 0.076s processing speed for a single test tube on average, which meets the real-time requirement.

Drug property prediction, especially toxicity, helps reduce risks in a range of real-world applications. In this paper, we aim to apply various machine-learning models for solving the drug toxicity prediction problem. Among various machine-learning approaches, we select five suitable representatives: random forest, multi-layer perceptron, logistic regression, graph convolutional neural network, and graph isomorphism network (GIN) for conducting experiments on six datasets for toxicity prediction, including Tox 21, ClinTox, ToxCast, SIDER, HIV, and BACE. We design the GIN with four hidden layers and select the Adam optimizer with the learning rate ${10}^{-4}$ and the batch size $256$. Furthermore, we use a batch norm layer inside each of the GIN hidden layers. Experimental results show that the designed GIN model is most efficient in distinguishing between safe and toxic drugs and outperforms the others under the supervision of ROC AUC score and recall.

Groove edge detection is the prerequisite for weld seam deviation identification. A welding groove edge detection method based on transfer learning is presented as a solution to the inaccuracy of the conventional image processing method for extracting the edge of the welding groove. DenseNet and MobileNetV2 are used as feature extractors for transfer learning. Dense-Mobile Net is constructed using the skip connections structure and depthwise separable convolution. The Dense-Mobile Net training procedure consists of two stages: pre-training and model fusion fine-tuning. Experiments demonstrate that the proposed model accurately detects groove edges in MAG welding images. Using MIG welding images and the Pascal VOC2012 dataset to evaluate the generalization ability of the model, the relevant indicators are greater than those of Support Vector Machine (SVM), Fully Convolutional Networks (FCN), and UNet. The average single-frame detection time of the proposed model is 0.14 s, which meets the requirements of industrial real-time performance.

Ross River virus (RRV) disease is one of the most epidemiological mosquito-borne diseases in Australia. Its major consequences on public health require building a precise and accurate model for predicting any forthcoming outbreaks. Several models have been developed by machine learning (ML) researchers, and many studies have been published as a result. Later, deep learning models have been introduced and shown tremendous success in forecasting, mainly the long short-term memory (LSTM), which performs significantly better than the traditional machine learning approaches. There are four common problems that previously developed models need to solve. They are exploding gradient, vanishing gradient, uncertainty and parameter bias. LSTM has already solved the first two problems, i.e. exploding and vanishing gradient problems, and the remaining two are overcome by $n$-LSTM. However, developing a prediction model for the RRV disease is a challenging task because it presents a wide range of symptoms, and there needs to be more accurate information available on the disease. To address these challenges, we propose a data-driven ensemble deep learning model using multi-networks of LSTM neural network for RRV disease forecasting in Australia. Data is collected between 1993 and 2020 from the Health Department of the Government of Australia. Data from 1993 to 2016 is taken to train the model, while the data of 2016–2020 is used as a test dataset. Previous research has demonstrated the efficacy of both ARIMA and exponential smoothing techniques in the field of time-series forecasting. As a result, our study sought to evaluate the performance of our proposed model in comparison to these established parametric methods, including ARIMA and ARMA, as well as the more recent deep learning approaches such as encoder–decoder and attention mechanism models. The results show that $n$-LSTM achieves higher accuracy and has a less mean-square error. We have also discussed the comparison of the models in detail. Such forecasting gives an insight into being well prepared and handling the situation of the outbreak.

In image super-resolution, the existing convolution neural network methods increase the number of network layers and filters to achieve better performance, and seldom consider the influence of different branches in feature extraction on the reconstruction effect, which leads to the problems of blurred details and unclear visual perception. Therefore, we propose an adaptive weight adjustment super-resolution (AWSR) reconstruction model in this paper. The model includes Shallow Feature Extraction (SFE), Information Extraction Enhancement Block (IDEB) and Reconstruction Block (RB). IDEB composed of Adaptive Weight Blocks (AWB) and Channel Linking Layers (CLL) learns a deeper mapping relationship between LR image and HR image by adaptively adjusting the proportions of different branches. It not only saves computational cost, but also improves the expression ability of the model. Meanwhile, the performance of the model is further improved by dimension change in the up-sample block. Especially, the image edge and texture reconstruction effects are obviously improved. Compared with SRNHARB algorithm proposed in 2021, the PSNR values are increased by 0.23dB, 0.19dB and 0.02dB at $\times 2,\times 3,\times 4$ on the Set5 dataset. Moreover, the proposed model has a strong generalization ability, and the reconstructed SR images can achieve satisfactory results.

This paper presents a brand-new Thanka picture inpainting technique based on Euler’s elastica, iterative denoising, and backward projections (EEIDBP). Specifically, a model of Euler’s elastica is introduced to estimate the original observation due to its lower staircasing effects and better approximation of natural images. A method for backward projection and iterative denoising is applied to achieve a more accurate estimate of the original signal by alternating iterations between the estimation of the original signal and the estimation of the original observation. The experimental findings demonstrate that, in terms of a subjective assessment, the quantitative peak signal-to-noise ratio (PSNR), and the structural similarity (SSIM), the proposed technique outperforms the state-of-the-art picture inpainting methods.

Because of the large-scale variation, counting in scenes of different densities is an extremely difficult task. In this paper, based on the attention mechanism, we propose a new self-weighted multi-scale fusion network structure named SMFNet to solve the problem of multi-scale changes and can significantly improve the effect of crowd counting in monitoring scene. The proposed SMFNet uses VGG as the backbone network to extract multi-scale features, uses a SMFNet as the neck to fuse multiple-scale features, and uses the atrous spatial pyramid pooling (ASPP) network and ordinary convolution as the head to generate both the attention map and the density map. The attention map highlighting crowd regions in the image contributes to a high-quality density map, and the density map records the crowd distribution. The number of crowd in the image can be obtained by summing the pixel values of the density map. We conduct experiments on three crowd counting datasets and one vehicle counting dataset to show that our proposed SMFNet can improve the state-of-the-art counting methods.

To address the issues of low accuracy in existing 3D human pose estimation (HPE) methods and the limited level of details in Labanotation, we propose an extended Labanotation generation method for intangible cultural heritage dance videos based on 3D HPE. First, a 2D human pose sequence of the performer is inputted along with spatial location embeddings, where multiple spatial transformer modules are employed to extract spatial features of human joints and generate cross-joint multiple hypotheses. Afterward, temporal features are extracted by a self-attentive module and the correlation between different hypotheses is learned using bilinear pooling. Finally, the 3D joint coordinates of the performer are predicted, which are matched with the corresponding extended Labanotation symbols using the Laban template matching method to generate extended Labanotation. Experimental results show that, compared with VideoPose and CrossFormer algorithms, the Mean Per Joint Position Error (MPJPE) of the proposed method is reduced by 3.7mm and 0.6mm, respectively on Human3.6M dataset, and the generated extended Labanotation can better describe the movement details compared with the basic Labanotation.

In today’s world, Human Activity Recognition (HAR) through video streams is actively used in every aspect of our life, such as automated surveillance systems and sports statistics are computed according to the videos with the help of HAR. Activity detection is not a new subject, and several methods are available. However, the most recent and most promising techniques rely on Convolutional Neural Networks (CNNs). CNNs primary usage is based on a single image frame to perform logical or categorical identification of an object, scene, or activity. We exploit this feature to adapt CNN on video streams to achieve HAR. In this study, we present a Personalized HAR (PHAR) framework that increases activity recognition accuracy with Object Detection (OD). First, we demonstrate the state-of-the-art HAR and OD methods in the literature. Then we illustrate our framework with two new Single Person Human Activity Recognition models. Finally, the performance of the new framework is evaluated with the well-known activity detection methods. Results show that our new PHAR model with 95% accuracy ratio outperforms the CNN-LSTM-based reference model (90%). Moreover, a new metric Average Accuracy Score (AAS) is described in this study, PHAR models approximately have 94% AAS, which is better than the reference model with 89% AAS.

Because of the limitations of convolution kernel, the traditional image segmentation network is not sufficient to obtain the context information, but the image segmentation task is very dependent on the context information. Transformer’s linear input can just get enough context information. In this paper, we propose a transformer segmentation network hyperfusion transformer based on a pyramid structure. First, the model divides the single-scale coding form into several-different-scale coding forms, and then fuses the decoding results. Second, in order to ensure the specificity of the output characteristics of each branch, we orthogonalize the results of a variety of different scales. By orthogonalizing in pairs, we can ensure that the results obtained by different branches are not completely similar to a certain extent, and reduce the redundancy of branch information. On the two datasets, the method in this paper surpasses a variety of classical models under multiple evaluation indexes, confirming that it is an effective segmentation method.