Narrow Results

Long short-term memory (LSTM) with significantly increased complexity and a large number of parameters have a bottleneck in computing power resulting from limited memory capacity. Hardware acceleration of LSTM using memristor circuit is an effective solution. This paper presents a complete design of memristive LSTM network system. Both the LSTM cell and the fully connected layer circuit are implemented through memristor crossbars, and the 1T1R design avoids the influence of the sneak current which helps to improve the accuracy of network calculation. To reduce the power consumption, the word embedding dimensionality was reduced using the GloVe model, and the number of features in the hidden layer was reduced. The effectiveness of the proposed scheme is verified by performing the text classification task on the IMDB dataset and the hardware training accuracy reached as high as 88.58%.

Ross River virus (RRV) disease is one of the most epidemiological mosquito-borne diseases in Australia. Its major consequences on public health require building a precise and accurate model for predicting any forthcoming outbreaks. Several models have been developed by machine learning (ML) researchers, and many studies have been published as a result. Later, deep learning models have been introduced and shown tremendous success in forecasting, mainly the long short-term memory (LSTM), which performs significantly better than the traditional machine learning approaches. There are four common problems that previously developed models need to solve. They are exploding gradient, vanishing gradient, uncertainty and parameter bias. LSTM has already solved the first two problems, i.e. exploding and vanishing gradient problems, and the remaining two are overcome by $n$-LSTM. However, developing a prediction model for the RRV disease is a challenging task because it presents a wide range of symptoms, and there needs to be more accurate information available on the disease. To address these challenges, we propose a data-driven ensemble deep learning model using multi-networks of LSTM neural network for RRV disease forecasting in Australia. Data is collected between 1993 and 2020 from the Health Department of the Government of Australia. Data from 1993 to 2016 is taken to train the model, while the data of 2016–2020 is used as a test dataset. Previous research has demonstrated the efficacy of both ARIMA and exponential smoothing techniques in the field of time-series forecasting. As a result, our study sought to evaluate the performance of our proposed model in comparison to these established parametric methods, including ARIMA and ARMA, as well as the more recent deep learning approaches such as encoder–decoder and attention mechanism models. The results show that $n$-LSTM achieves higher accuracy and has a less mean-square error. We have also discussed the comparison of the models in detail. Such forecasting gives an insight into being well prepared and handling the situation of the outbreak.

With the advent of web 2.0, web application architectures have been evolved, and their complexity has grown enormously. Due to the complexity, testing of web applications is getting time-consuming and intensive process. In today’s web applications, users can achieve the same goal by performing different actions. To ensure that the entire system is safe and robust, developers try to test all possible user action sequences in the testing phase. Since the space of all the possibilities is enormous, covering all user action sequences can be impossible. To automate the test script generation task and reduce the space of the possible user action sequences, we propose a novel method based on long short-term memory (LSTM) network for generating test scripts from user clickstream data. The experiment results clearly show that generated hidden test sequences are user-like sequences, and the process of generating test scripts with the proposed model is less time-consuming than writing them manually.

Arrhythmia classification is useful during heart disease diagnosis. Although well-established for intra-patient diagnoses, inter-patient arrhythmia classification remains difficult. Most previous work has focused on the intra-patient condition and has not followed the Association for the Advancement of Medical Instrumentation (AAMI) standards. Here, we propose a novel system for arrhythmia classification based on multi-lead electrocardiogram (ECG) signals. The core of the design is that we fuse two types of deep learning features with some common traditional features and select discriminating features using a binary particle swarm optimization algorithm (BPSO). Then, the feature vector is classified using a weighted support vector machine (SVM) classifier. For a better generalization of the model and to draw fair comparisons, we carried out inter-patient experiments and followed the AAMI standards. We found that, when using common metrics aimed at multi-classification either macro- or micro-averaging, our system outperforms most other state-of-the-art methods.

We present an empirical analysis of the source code of the Fluoride Bluetooth module, which is a part of standard Android OS distribution, by exhibiting a novel approach for classifying and scoring source code and vulnerability rating. Our workflow combines deep learning, combinatorial optimization, heuristics and machine learning. A combination of heuristics and deep learning is used to embed function (and method) labels into a low-dimensional Euclidean space. Because the corpus of the Fluoride source code is rather limited (containing approximately 12,000 functions), a straightforward embedding (using, e.g. code2vec) is untenable. To overcome the challenge of dearth of data, it is necessary to go through an intermediate step of Byte-Pair Encoding. Subsequently, we embed the tokens from which we assemble an embedding of function/method labels. Long short-term memory network (LSTM) is used to embed tokens. The next step is to form a distance matrix consisting of the cosines between every pairs of vectors (function embedding) which in turn is interpreted as a (combinatorial) graph whose vertices represent functions, and edges correspond to entries whose value exceed some given threshold. Cluster-Editing is then applied to partition the vertex set of the graph into subsets representing “dense graphs,” that are nearly complete subgraphs. Finally, the vectors representing the components, plus additional heuristic-based features are used as features to model the components for vulnerability risk.

Emotion recognition plays an essential role in human–human interaction since it is a key to understanding the emotional states and reactions of human beings when they are subject to events and engagements in everyday life. Moving towards human–computer interaction, the study of emotions becomes fundamental because it is at the basis of the design of advanced systems to support a broad spectrum of application areas, including forensic, rehabilitative, educational, and many others. An effective method for discriminating emotions is based on ElectroEncephaloGraphy (EEG) data analysis, which is used as input for classification systems. Collecting brain signals on several channels and for a wide range of emotions produces cumbersome datasets that are hard to manage, transmit, and use in varied applications. In this context, the paper introduces the Empátheia system, which explores a different EEG representation by encoding EEG signals into images prior to their classification. In particular, the proposed system extracts spatio-temporal image encodings, or atlases, from EEG data through the Processing and transfeR of Interaction States and Mappings through Image-based eNcoding (PRISMIN) framework, thus obtaining a compact representation of the input signals. The atlases are then classified through the Empátheia architecture, which comprises branches based on convolutional, recurrent, and transformer models designed and tuned to capture the spatial and temporal aspects of emotions. Extensive experiments were conducted on the Shanghai Jiao Tong University (SJTU) Emotion EEG Dataset (SEED) public dataset, where the proposed system significantly reduced its size while retaining high performance. The results obtained highlight the effectiveness of the proposed approach and suggest new avenues for data representation in emotion recognition from EEG signals.

The impact of motors breakdown and failures on mobile robot motor bearing is an important concern for robot industries. For this reason, predictive motor lifetime and bearing fault classification techniques are being investigated extensively as a method of decreasing motor downtime and enhancing mobile robot reliability. With increasing attention on neural network technologies, many researchers have carried out lots of the relevant experiments and analyses, very plentiful and important conclusions are obtained. In this article, a classification method based on discrete wavelet transform (DWT) and long short-term memory network (LSTM) a proposed to find and classify fault type of mobile robot permanent magnet synchronous motor (PMSM). First, a set of mobile robot motor vibration signal were collected by the sensors. Second, the obtained vibration signal is decomposed into six frequency bands by the DWT. Haar function is selected as the mother function in the processing. The energy of every frequency band was calculated as a classification feature. Thirdly, four classification features with high classification rate are obtained. The feature vector is used as input of the neural network, and the fault type is identified by LSTM classifier with deviation unit. From the results of the experiments provided in the paper, the method can detect the fault type accurately and it is feasible and effective under different motor speed.

Molecular biology combined with in silico machine learning and deep learning has facilitated the broad application of gene expression profiles for gene function prediction, optimal crop breeding, disease-related gene discovery, and drug screening. Although the acquisition cost of genome-wide expression profiles has been steadily declining, the requirement generates a compendium of expression profiles using thousands of samples remains high. The Library of Integrated Network-Based Cellular Signatures (LINCS) program used approximately 1000 landmark genes to predict the expression of the remaining target genes by linear regression; however, this approach ignored the nonlinear features influencing gene expression relationships, limiting the accuracy of the experimental results. We herein propose a gene expression prediction model, L-GEPM, based on long short-term memory (LSTM) neural networks, which captures the nonlinear features affecting gene expression and uses learned features to predict the target genes. By comparing and analyzing experimental errors and fitting the effects of different prediction models, the LSTM neural network-based model, L-GEPM, can achieve low error and a superior fitting effect.

Defects are frequent in software systems, and they can cause a lot of issues for users. Despite the fact that many studies have been conducted on employing software product metrics to determine defect-prone modules, defect prediction techniques are still worth investigating. Hence, the aim of this work is to provide a unique Software Defect Prediction (SDP) approach that includes four steps like “(a) pre-processing, (b) feature extraction, (c) feature selection and (d) detection.” At first, the input data are given to the pre-processing step, as well as in the feature extraction step; the “statistical features, raw features, higher-order statistical features as well as proposed entropy features” are extracted from the pre-processed data. In addition, the retrieved features are sent into a feature selection step, wherein the appropriate features are selected utilizing a modified chi-square scheme. In the detection step, a hybrid Long Short-Term Memory (LSTM) and Deep Belief Network (DBN) classifiers are used to predict the defects. To provide a more accurate detection, the weights of both DBN and LSTM are optimally tuned via a Self Improved Social Ski-Driver Optimization (SISSDO) algorithm. This proposed SDP model is a beneficial practice for enhancing software quality and reliability. Moreover, the results of the adopted technique are assessed to traditional techniques on the basis of various measures. In particular, the accuracy of the suggested approach for dataset 3 is 5.80%, 6.52%, 5.07%, 7.97%, 5.80%, 9.42%, 9.42%, 10.15%, 2.17%, and 3.62% better than the extant HC+ALO, HC+SMO, HC+CMBO, HC+SSD, RNN, CNN, NN, Bi-LSTM, HC+SPFCNN, and HC+CWAR approaches, correspondingly. Moreover, the computation time of the suggested approach is 17.05%, 5.78%, 1.31%, and 50.53% better than the existing HC+ALO, HC+SMO, HC+CMBO, and HC+SSD approaches, correspondingly.

Structural damage identification based on the long short-term memory (LSTM) neural network (NN) is proposed in this study. To address the hyperparameters selection problem for the LSTM, the Jaya algorithm is applied to minimize the difference between the observed and predicted data in the validation datasets and determine the LSTM network’s optimal hyperparameters, including the number of nodes, learning rate, and maximum iteration number. Frequency-domain data, such as natural frequencies and mode shapes, are used as input of the network, and then damage locations and extents are utilized as output. Measurement uncertainties are introduced during NN training to improve the robustness of the model. Numerical and experimental studies showed that the proposed method can identify structural damage accurately when measurement noise is considered, even for damage scenarios beyond the training datasets.

Surface electromyography (sEMG) signals have been widely used in human–machine interaction, providing more nature control expedience for external devices. However, due to the instability of sEMG, it is hard to extract consistent sEMG patterns for motion recognition. This paper proposes a dual-flow network to extract the temporal-spatial feature of sEMG for gesture recognition. The proposed network model uses convolutional neural network (CNN) and long short-term memory methods (LSTM) to, respectively, extract the spatial feature and temporal feature of sEMG, simultaneously. These features extracted by CNN and LSTM are merged into temporal-spatial feature to form an end-to-end network. A dataset was constructed for testing the performance of the network. In this database, the average recognition accuracy by using our dual-flow model reached 78.31%, which was improved by 6.69% compared to the baseline CNN (71.67%). In addition, NinaPro DB1 is also used to evaluate the proposed methods, receiving 1.86% higher recognition accuracy than the baseline CNN classifier. It is believed that the proposed dual-flow network owns the merit in extracting stable sEMG feature for gesture recognition, and can be further applied into practical applications.

Rice is a staple food crop around the world, and its demand is likely to rise significantly with growth in population. Increasing rice productivity and production largely depends on the availability of irrigation water. Thus, the efficient application of irrigation water such that the crop doesn’t experience moisture stress is of utmost importance. In the present study, a long short-term memory (LSTM)-based neural network with logistic regression has been used to predict the daily irrigation schedule of drip-irrigated rice. The correlation threshold of 0.75 was used for the selection of features, which helped in limiting the number of input parameters. Also, a dataset based on the recommendation of a domain expert, and another used by the tool Agricultural Production Systems Simulator (APSIM) was used for comparison. Field data comprising of weather station data and past irrigation schedules has been used to train the model. Grid search algorithm has been used to optimize the hyperparameters of the model. Nested cross-validation has been used for validating the results. The results show that the correlation-based selected dataset is as effective as the domain expert-recommended dataset in predicting the water requirement using LSTM as the base model. The models were evaluated on different parameters and a multi-criteria decision evaluation (Technique for Order of Preference by Similarity to Ideal Solution [TOPSIS]) was used to find the best performing.

Recently, data collected from social media enable to analyze social events and make predictions about real events, based on the analysis of sentiments and opinions of users. Most cyber-attacks are carried out by hackers on the basis of discussions on social media. This paper proposes the method that predicts DDoS attacks occurrence by finding relevant texts in social media. To perform high-precision classification of texts to positive and negative classes, the CNN model with 13 layers and improved LSTM method are used. In order to predict the occurrence of the DDoS attacks in the next day, the negative and positive sentiments in social networking texts are used. To evaluate the efficiency of the proposed method experiments were conducted on Twitter data. The proposed method achieved a recall, precision, $F$-measure, training loss, training accuracy, testing loss, and test accuracy of 0.85, 0.89, 0.87, 0.09, 0.78, 0.13, and 0.77, respectively.

Network Application Classification (NAC) is a vital technology for intrusion detection, Quality-of-Service (QoS)-aware traffic engineering, traffic analysis, and network anomalies. Researchers have focused on designing algorithms using deep learning models based on statistical information to address the challenges of traditional payload and port-based traffic classification techniques. Internet of Things (IoT) and Software Defined Network (SDN) are two popular technologies nowadays and aims to connect devices over the internet and intelligently control networks from a centralized space. IoT aims to connect billions of devices; therefore, classification is essential for efficient processing. SDN is a new networking paradigm, which separates data plane measurement from the control plane. The emergence of deep learning algorithms with SDN provides a scalable traffic classification architecture. Due to the inadequate results of payload and port-based approaches, a statistical technique to classify network traffic into different classes using a Convolution Neural Network (CNN) and a Recurrent Neural Network (RNN) is presented in this paper. This paper provides a classification method for software defined IoT networks. The results show that, contrary to other traffic classification methods, the proposed approach offered a better accuracy rate of over 99 %, which is promising.

The succinylation modification of protein participates in the regulation of a variety of cellular processes. Identification of modified substrates with precise sites is the basis for understanding the molecular mechanism and regulation of succinylation. In this work, we picked and chose five superior feature codes: CKSAAP, ACF, BLOSUM62, AAindex, and one-hot, according to their performance in the problem of succinylation sites prediction. Then, LSTM network and CNN were used to construct four models: LSTM-CNN, CNN-LSTM, LSTM, and CNN. The five selected features were, respectively, input into each of these four models for training to compare the four models. Based on the performance of each model, the optimal model among them was chosen to construct a hybrid model DeepSucc that was composed of five sub-modules for integrating heterogeneous information. Under the 10-fold cross-validation, the hybrid model DeepSucc achieves 86.26% accuracy, 84.94% specificity, 87.57% sensitivity, 0.9406 AUC, and 0.7254 MCC. When compared with other prediction tools using an independent test set, DeepSucc outperformed them in sensitivity and MCC. The datasets and source codes can be accessed at https://github.com/1835174863zd/DeepSucc.

This paper has the aim of solving problems in research studies on the analysis tasks of text emotion; the problems are the low utilization of text, the difficulty of effective information extraction, the failure of recognizing word polysemy with effectiveness. Thus, based on LSTM and Bert, the method of sentiment analysis on text is adopted. To be precise, word embedding of dataset in view of the skip-gram model is used for training course. In each sample, the word embeddings combine matric with the two-dimensional feature to be neural network input. Next, construction of analysis model for text sentiment combines Bert pre-training language model and long short-term memory (LSTM) network, using the word vector pre-trained by Bert instead of that trained in the traditional way to dynamically generate the semantic vector according to the word context. Finally, the semantic representation of words from text is improved by effectively identifying the polysemy of words, and the semantic vector is input into the LSTM to capture the semantic dependencies, thereby enhancing the ability to extract valid information. The Accuracy, Precision, Recall and F-Measure for the method of Bert–LSTM based on analysis of text sentiment are 0.89, 0.9, 0.84 and 0.87, indicating high value than the compared ones. Thus, the proposed method significantly outperforms the comparison methods in text sentiment analysis.

The exponentially increasing amount of data generated by the public on social media platforms is a precious source of information. It can be used to find the topics and analyze the comments. Some researchers have extended the Latent Dirichlet Allocation (LDA) method by adding a sentiment layer to simultaneously find the topics and their related sentiments. However, most of these approaches do not achieve admirable accuracy in Topic Sentiment Analysis (TSA), particularly when there is insufficient training data or the texts are complex, ambiguous, and short. In this paper, a self-supervised novel approach called SSTSA is proposed for TSA that extracts the hidden topics and analyzes the total sentiment related to each topic. The SSTSA proposes a new method called Pseudo-label Generator. For this purpose, first, it employs semantic similarity and Word Mover’s Distance (WMD) measures. Then, the document embedding technique is employed to semantically estimate the sentiment orientation of samples and generate the pseudo-labels (positive or negative). Afterward, a hybrid classifier composed of a pre-trained Robustly Optimized BERT (RoBERTa) and a Long Short-Term Memory (LSTM) model is trained to predict the sentiment of unseen data. The evaluation results on different datasets of various domains demonstrate that the SSTSA outperforms similar unsupervised/self-supervised methods.

This paper presents the automated tool for the classification of text with respect to predefined categories. It has always been considered as a vital method to manage and process a huge number of documents in digital forms which are widespread and continuously increasing. Most of the research work in text classification has been done in Urdu, English and other languages. But limited research work has been carried out on roman data. Technically, the process of the text classification follows two steps: the first step consists of choosing the main features from all the available features of the text documents with the usage of feature extraction techniques. The second step applies classification algorithms on those chosen features. The data set is collected through scraping tools from the most popular news websites Awaji Awaze and Daily Jhoongar. Furthermore, the data set splits in training and testing 70%, 30%, respectively. In this paper, the deep learning models, such as RNN, LSTM, and CNN, are used for classification of roman Urdu headline news. The testing accuracy of RNN (81%), LSTM (82%), and CNN (79%), and the experimental results demonstrate that the performance of the LSTM method is state-of-art method compared to CNN and RNN.

To improve the accuracy and efficiency of tool wear predictions, this study proposes a tool wear prediction model called LSTM_ResNet which is based on the long short-term memory (LSTM) network and the Residual Network (ResNet). The model utilizes LSTM layers for processing, where the first block and loop blocks serve as the core modules of the deep residual network. The model employs a series of methods including convolution, batch normalization (BN), and Rectified Linear Unit (ReLU) to enhance the model’s expression and prediction capabilities. The performance of the LSTM_ResNet model was evaluated using experimental data from the PHM2010 datasets and two different depths (64 and 128 layers), training both LSTM_ResNet models for 200 epochs. The 64-layer model’s root mean square error (RMSE) values are 3.36, 4.35, and 3.59, and the mean absolute error (MAE) values are 2.42, 2.85, and 2.21; using 128 layers, the RMSE values are 3.66, 3.99, and 3.77, and the MAE values are 2.49, 2.73, and 3.01. The results indicate that the 64-layer LSTM has smaller average errors, suggesting that compared to other common network structures, the LSTM_ResNet network has a higher performance. This research provides an effective solution for tool wear prediction and helps to improve the technical level of tool wear prediction in China.

With the rapid development and application of autonomous technology in vehicles, we are going to see more autonomous vehicles on the roads in a foreseeable future. While autonomous vehicles may have the advantage of reducing traffic accidents caused by human drivers’ neglect and/or fatigue, one of the challenges is how to develop autonomous driving algorithms such that autonomous vehicles can be safely deployed in a mixed traffic environment with both autonomous vehicles and human-driven vehicles. Instantaneous lane-changing type may be significantly different for human drivers, which would lead to traffic accidents with other vehicles including autonomous vehicles. In this paper, we propose a resilient algorithm for the prediction of the human driver’s lane-changing behaviors. The proposed algorithm uses a long-short term memory (LSTM) classifier to identify the conservative lane change and the aggressive lane changing and accordingly makes the accurate prediction on lane changes in the driving of vehicles by human drivers. The proposed method provides a useful addition in facilitating the design of more advanced driving algorithms for autonomous vehicles. Using the vehicle trajectory data in the NGSIM data set for a large number of simulations, the effectiveness of this method has been confirmed.