Narrow Results

Activation functions play a crucial role in introducing nonlinearity into neural networks. Previous approaches, such as rectifying neurons and logistic sigmoid neurons, primarily focused on constructing activation functions based on biological models. These methods have achieved considerable success in various machine learning tasks. However, they are inadequate in addressing two critical issues: Gradient Vanishing and Dead ReLU. This research demonstrates that employing Legendre orthogonal wavelets can effectively alleviate these issues and yield comparable or superior performance compared to rectifying neurons. In this study, the input features prior to the activation layer are normalized to the range [0,1]. Each mini-interval within this range is then mapped using a set of orthogonal Legendre wavelets. Subsequently, the activation function produces a weighted sum of all the mini-intervals. By modifying typical models used in image classification, this paper found that incorporating Legendre wavelets resulted in significant improvements in network accuracy. With minor adjustments in the hidden layers, ResNet50 achieved an increase of +5.17% in accuracy for CIFAR10 classification. Similarly, in CIFAR100 classification, the use of Legendre wavelets in Swin Transformers led to a +3.51% boost in accuracy.

In this paper, a numerical approximation based on Legendre wavelets has been developed to solve nonlinear fractional Volterra–Fredholm integro-differential equations. Legendre wavelets are generated by dilation and translation of Legendre polynomials. The properties of the Legendre wavelets are presented in the paper. The proposed wavelet method transforms the integral equations to a system of nonlinear algebraic equations and this algebraic system has been solved numerically by Newton’s method. Convergence analysis of the proposed method has been discussed in this paper. Some examples have been illustrated to show the applicability and accuracy of the present method.

This paper represents a new application of Legendre wavelet and interpolating scaling function to discuss the approximate solution of variable order integro-differential equation having weakly singular kernel. So far, this technique has been used to solve variable order integro differential equation. In this paper, it is extended to solve variable order integro differential equation with weakly singular kernel. For this purpose, we derive the operational matrices of Legendre wavelets and interpolating scaling function. The resulting operational matrices along with the collocation method transform the original problem into a system of algebraic equation. By solving this system, the approximate solution is obtained. The convergence and error estimate of the presented method have been rigorously investigated. We also discuss the numerical stability of the method. The numerical result of some inclusive examples has been provided through a table and graph for both basis functions that support the robustness and desired precision of the method.

A Legendre wavelets neural network is constructed with Legendre wavelets based on BP neural network. Because of the piece wise expression and being polynomials features which Legendre wavelets defined on the interval [0,1) has, the Legendre wavelets neural network has the advantages of simple structure and high convergence rate. Trained by BP algorithm, a better approximate result is obtained by using Legendre wavelets neural network with six wavelet basis functions to approximate one function.