Fully-connected Layers Research Articles

Anomaly-PTG: A Time Series Data-Anomaly-Detection Transformer Framework in Multiple Scenarios

In actual scenarios, industrial and cloud computing platforms usually need to monitor equipment and traffic anomalies through multivariable time series data. However, the existing anomaly detection methods can not capture the long-distance temporal correlations of data and the potential relationships between features simultaneously, and only have high detection accuracy for specific time sequence anomaly detection scenarios without good generalization ability. This paper proposes a time-series anomaly-detection framework for multiple scenarios, Anomaly-PTG (anomaly parallel transformer GRU), given the above limitations. The model uses the parallel transformer GRU as the information extraction module of the model to learn the long-distance correlation between timestamps and the global feature relationship of multivariate time series, which enhances the ability to extract hidden information from time series data. After extracting the information, the model learns the sequential representation of the data, conducts the sequential modeling, and transmits the data to the full connection layer for prediction. At the same time, it also uses the autoencoder to learn the potential representation of the data and reconstruct the data. The two are optimally combined to form an anomaly detection module of the model. The module combines timestamp prediction with time series data reconstruction, improving the detection rate of rare anomalies and detection accuracy. By using three public datasets of physical devices and one dataset of network traffic intrusion detection, the model’s effectiveness was verified, and the model’s generalization ability and strong robustness were demonstrated. Compared with the most advanced method, the average F1 value of the Anomaly-PTG model on four datasets was increased by 2.2%, and the F1 value on each dataset was over 94%.

Deep neural networks compression: A comparative survey and choice recommendations

The state-of-the-art performance for several real-world problems is currently reached by deep and, in particular, convolutional neural networks (CNN). Such learning models exploit recent results in the field of deep learning, leading to highly performing, yet very large neural networks with typically millions to billions of parameters. As a result, such models are often redundant and excessively oversized, with a detrimental effect on the environment in terms of unnecessary energy consumption and a limitation to their deployment on low-resource devices. The necessity for compression techniques able to reduce the number of model parameters and their resource demand is thereby increasingly felt by the research community. In this paper we propose the first extensive comparison, to the best of our knowledge, of the main lossy and structure-preserving approaches to compress pre-trained CNNs, applicable in principle to any existing model. Our study is intended to provide a first and preliminary guidance to choose the most suitable compression technique when there is the need to reduce the occupancy of pre-trained models. Both convolutional and fully-connected layers are included in the analysis. Our experiments involved two pre-trained state-of-the-art CNNs (proposed to solve classification or regression problems) and five benchmarks, and gave rise to important insights about the applicability and performance of such techniques w.r.t.the type of layer to be compressed and the category of problem tackled.

SNAL: sensitive non-associative learning network configuration for the automatic driving strategy

Nowadays, there is a huge gap between autonomous vehicles and mankind in terms of the decision response against some dangerous scenarios, which would has stressed the potential users out and even made them nervous. To efficiently identify the possible sensitivity scenarios, a new neural network configuration, named sensitive non-associative learning network (SNAL), is proposed. In such structure, the modulated interneurons, excited by abnormal scene stimulation for scene processing, are well processed and utilized to improve the training structure which refers to the sensitization mechanism in non-associative learning in neurobiology and the neural structure of Aplysia. When encountering the sensitivity scenes that the automatic driving agent is not good at or has not seen, the modulated interneuron facilitates the full connection layer neurons for the decision-making process, so as to change the final automatic driving strategy. In the process of constructing the model, a method to measure the similarity of the convolution feature map is proposed, which provides a new investigation tool for the properties of convolution networks after the feature extraction. Based on the Morris–Lecar equation in neurobiology, the dynamic model of modulating interneurons in the network is constructed. The automatic control optimization of the model is carried out by imitating the biological properties. The optimization method provides a reference for introducing neurobiological mechanism into deep learning and automatic control. To validate the effectiveness of the proposed method, the simulation test are executed and the existing methods are compared accordingly. The results show that the proposed SNAL algorithm can effectively recognize the sensitivity mechanism. Furthermore, compared with the existing algorithms, such as CNN, LSTM, ViT, the proposed algorithm can make better defensive strategies for potentially dangerous scenes rarely seen or not seen in the training stage. This sensitivity mechanism is more in line with the human driving intuition when dealing with abnormal driving scenes, and makes the decision more interpretable, significantly improving the traffic ability of autonomous vehicles under the sensitive scenes. In addition, this configuration can be easily combined with the existing mainstream neural network models and has good expansibility.

Dance emotion recognition based on linear predictive Meir frequency cepstrum coefficient and bidirectional long short-term memory from robot environment.

Dance emotion recognition is an important research direction of automatic speech recognition, especially in the robot environment. It is an important research content of dance emotion recognition to extract the features that best represent speech emotion and to construct an acoustic model with strong robustness and generalization. The dance emotion data set is small in size and high in dimension. The traditional recurrent neural network (RNN) has the problem of long-range dependence disappearance, and due to the focus on local information of convolutional neural network (CNN), the mining of potential relationships between frames in the input sequence is insufficient. To solve the above problems, this paper proposes a novel linear predictive Meir frequency cepstrum coefficient and bidirectional long short-term memory (LSTM) for dance emotion recognition. In this paper, the linear prediction coefficient (LPC) and Meier frequency cepstrum coefficient (MFCC) are combined to obtain a new feature, namely the linear prediction Meier frequency cepstrum coefficient (LPMFCC). Then, the combined feature obtained by combining LPMFCC with energy feature is used as the extracted dance feature. The extracted features are input into the bidirectional LSTM network for training. Finally, support vector machine (SVM) is used to classify the obtained features through the full connection layer. Finally, we conduct experiments on public data sets and obtain the better effectiveness compared with the state-of-art dance motion recognition methods.

Convolutional Neural Network Model to Predict Outdoor Comfort UTCI Microclimate Map

Although research on applying machine learning to the performance of the built environment has been advancing considerably, outdoor environment prediction models still need to be more accurate. In this study, I investigated hybrid-driven methods for developing environmental performance prediction models and studied how machine learning algorithms may interpret spatial information in the context of an environmental performance simulation challenge. The simulation of the Universal Thermal Climate Index (UTCI) for outdoor applications served as an example. Specifically, I designed two different network structures, each with six neural network models. These neural network models were built with various numbers of layers, convolutional kernel sizes, and convolutional kernel layers. As shown by these models’ training results, I investigated the effect of model parameter settings on performance. In addition, I conducted interpretable analysis through the visual observation of hidden internal layers. The use of multilayer and small convolutional kernels, as well as an increase in the amount of training data, may be the reason neural network prediction performance was improved. From the perspective of interpretability analysis, the convolutional layer can more accurately analyze building space problems, and full connection layers focus more on the regression between the spatial features and performance results. This “space analysis → data regression” network structure can be expanded to wind environment forecasting or heat environment in the future.

State-of-Health Prediction Using Transfer Learning and a Multi-Feature Fusion Model.

Existing data-driven technology for prediction of state of health (SOH) has insufficient feature extraction capability and limited application scope. To deal with this challenge, this paper proposes a battery SOH prediction model based on multi-feature fusion. The model is based on a convolutional neural network (CNN) and a long short-term memory network (LSTM). The CNN can learn the cycle features in the battery data, the LSTM can learn the aging features of the battery over time, and regression prediction can be made through the full-connection layer (FC). In addition, for the aging differences caused by different battery operating conditions, this paper introduces transfer learning (TL) to improve the prediction effect. Across cycle data of the same battery under 12 different charging conditions, the fusion model in this paper shows higher prediction accuracy than with either LSTM and CNN in isolation, reducing RMSPE by 0.21% and 0.19%, respectively.

Air Combat Maneuver Decision Method Based on A3C Deep Reinforcement Learning

To solve the maneuvering decision problem in air combat of unmanned combat aircraft vehicles (UCAVs), in this paper, an autonomous maneuver decision method is proposed for a UCAV based on deep reinforcement learning. Firstly, the UCAV flight maneuver model and maneuver library of both opposing sides are established. Then, considering the different state transition effects of various actions when the pitch angles of the UCAVs are different, the 10 state variables including the pitch angle, are taken as the state space. Combined with the air combat situation threat assessment index model, a two-layer reward mechanism combining internal reward and sparse reward is designed as the evaluation basis of reinforcement learning. Then, the neural network model of the full connection layer is built according to an Asynchronous Advantage Actor–Critic (A3C) algorithm. In the way of multi-threading, our UCAV keeps interactively learning with the environment to train the model and gradually learns the optimal air combat maneuver countermeasure strategy, and guides our UCAV to conduct action selection. The algorithm reduces the correlation between samples through multi-threading asynchronous learning. Finally, the effectiveness and feasibility of the method are verified in three different air combat scenarios.

Spectral-invariant matching network

As the need for sensor fusion systems has grown, developing methods to find correspondences between images with different spectral ranges has become increasingly important. Since most images do not share low-level information, such as textures and edges, existing matching approaches fail even with convolutional neural networks (CNNs). In this paper, we propose an end-to-end metric learning method, called SPIMNet (SPectral-Invariant Matching Network) for robust cross- and multi-spectral image patch matching. While existing methods based on CNNs learn matching features directly from cross- and multi-spectral image patches, SPIMNet transforms across spectral bands and discriminates for similarity in three steps. First, (1) SPIMNet is adjusted for a feature domain by introducing a domain translation network; then (2) two Siamese networks learn to match the adjusted features with the same spectral domain; and (3) the matching features are fed to fully-connected layers to determine the identity of the patches as a classification task. By effectively incorporating each step, SPIMNet achieved competitive results on a variety of challenging datasets, including both VIS–NIR and VIS–Thermal image pairs. Our code is available at https://github.com/koyeongmin/SPIMNet.

Improved deep convolutional neural networks using chimp optimization algorithm for Covid19 diagnosis from the X-ray images

Applying Deep Learning (DL) in radiological images (i.e., chest X-rays) is emerging because of the necessity of having accurate and fast COVID-19 detectors. Deep Convolutional Neural Networks (DCNN) have been typically used as robust COVID-19 positive case detectors in these approaches. Such DCCNs tend to utilize Gradient Descent-Based (GDB) algorithms as the last fully-connected layers’ trainers. Although GDB training algorithms have simple structures and fast convergence rates for cases with large training samples, they suffer from the manual tuning of numerous parameters, getting stuck in local minima, large training samples set requirements, and inherently sequential procedures. It is exceedingly challenging to parallelize them with Graphics Processing Units (GPU). Consequently, the Chimp Optimization Algorithm (ChOA) is presented for training the DCNN's fully connected layers in light of the scarcity of a big COVID-19 training dataset and for the purpose of developing a fast COVID-19 detector with the capability of parallel implementation. In addition, two publicly accessible datasets termed COVID-Xray-5 k and COVIDetectioNet are used to benchmark the proposed detector known as DCCN-Chimp. In order to make a fair comparison, two structures are proposed: i-6c-2 s-12c-2 s and i-8c-2 s-16c-2 s, all of which have had their hyperparameters fine-tuned. The outcomes are evaluated in comparison to standard DCNN, Hybrid DCNN plus Genetic Algorithm (DCNN-GA), andMatched Subspace classifier with Adaptive Dictionaries (MSAD). Due to the large variation in results, we employ a weighted average of the ensemble of ten trained DCNN-ChOA, with the validation accuracy of the weights being used to determine the final weights. The validation accuracy for the mixed ensemble DCNN-ChOA is 99.11%. LeNet-5 DCNN's ensemble detection accuracy on COVID-19 is 84.58%. Comparatively, the suggested DCNN-ChOA yields over 99.11% accurate detection with a false alarmrate of less than 0.89%. The outcomes show that the DCCN-Chimp can deliver noticeably superior results than the comparable detectors. The Class Activation Map (CAM) is another tool used in this study to identify probable COVID-19-infected areas. Results show that highlighted regions are completely connected with clinical outcomes, which has been verified by experts.

EvoPruneDeepTL: An evolutionary pruning model for transfer learning based deep neural networks

In recent years, Deep Learning models have shown a great performance in complex optimization problems. They generally require large training datasets, which is a limitation in most practical cases. Transfer learning allows importing the first layers of a pre-trained architecture and connecting them to fully-connected layers to adapt them to a new problem. Consequently, the configuration of the these layers becomes crucial for the performance of the model. Unfortunately, the optimization of these models is usually a computationally demanding task. One strategy to optimize Deep Learning models is the pruning scheme. Pruning methods are focused on reducing the complexity of the network, assuming an expected performance penalty of the model once pruned. However, the pruning could potentially be used to improve the performance, using an optimization algorithm to identify and eventually remove unnecessary connections among neurons. This work proposes EvoPruneDeepTL, an evolutionary pruning model for Transfer Learning based Deep Neural Networks which replaces the last fully-connected layers with sparse layers optimized by a genetic algorithm. Depending on its solution encoding strategy, our proposed model can either perform optimized pruning or feature selection over the densely connected part of the neural network. We carry out different experiments with several datasets to assess the benefits of our proposal. Results show the contribution of EvoPruneDeepTL and feature selection to the overall computational efficiency of the network as a result of the optimization process. In particular, the accuracy is improved, reducing at the same time the number of active neurons in the final layers.

Intelligent diagnosis of rolling bearings with imbalanced data based on the ConVAE-CNN model

Data-driven intelligent diagnosis method has been widely used in mechanical equipment fault identification. However, the data set imbalance is still one of the critical problems affecting the effect of intelligent diagnosis in the practical work of rotating machinery. To solve the problem of fault identification of rolling bearings with imbalanced data sets, a rolling bearing intelligent diagnosis method based on the ConVAE-CNN model was proposed in this paper. In this method, a convolutional variational autoencoder network (ConVAE) model is constructed. In the coding stage, the convolutional layer is used to extract the features of minority samples. In the decoding stage, full connection layers are used to expand the dataset of minority samples. Bearing fault features were extracted by convolution and pooling operations of the convolutional neural network model, and the performance of the proposed method was tested using rolling bearing experimental data. The results show that the proposed method effectively solves the bearing fault classification problem with imbalanced data sets. Compared with the other diagnosis methods, the performance of the proposed method is better, and it has a broad application prospect in the diagnosis of imbalanced data.

Simplifying VGG-16 for Plant Species Identification

Plant species identification represents an extraordinary challenge for machine learning due to visual interspecies similarities and large intraspecies variations. Furthermore, research literature reports that plant species identification usually lacks sufficiently large datasets for training classification models. In this paper, we address this problem with a model that simplifies the VGG-16 architecture, the N-VGG model. The idea behind N-VGG is to reduce experimentally observed overfitting on VGG-16 by using as few trainable parameters as possible. To do this, we substitute the flattening layer on the VGG architecture with a global average pooling layer. This reduces the size of the feature vector. In addition, we eliminate one of the two fully-connected layers and use a new hyper-parameter, N, to indicate the number of nodes on the remaining layer. To show the robustness of the N-VGG model, we conducted extensive experimentation. We trained N-VGG on five datasets for plant species identification. Four of these datasets are publicly available and have been widely used as benchmarks for plant identification models. For all datasets, we compare the accuracy of N-VGG to that of the VGG-16, Inception-v4, and EfficienNet-B3 models. The experimental results show that the N-VGG model achieved the best classification performance for all but one datasets, whereas all the models showed a remarkable performance for the remaining dataset. This evidence supports our initial idea that, for plant species classification, some accuracy might be lost due to overfitting and that having fewer trainable parameters helps in producing a more robust model.

DLS: A Fast and Flexible Neural Network Training System With Fine-grained Heterogeneous Device Orchestration

Neural network accelerators (e.g., TPUs) have become mainstream devices in computing systems. Unfortunately, the existing accelerator-based systems for neural networks fail to fully leverage the acceleration opportunities due to the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">limited flexibility</i> . Specifically, the majority of the accelerators focus on only the compute-intensive operations of neural networks (e.g., convolution and fully-connected layers). However, we identify that sub-optimal handling of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">auxiliary operations</i> such as embedding and compression can incur non-trivial loss in terms of accuracy, training speed, and adaptability to new domains. The problem persists considering that recent advancements in neural networks often come from auxiliary operations. To effectively handle rapidly evolving auxiliary operations and maximize acceleration opportunities, we propose DLS, a holistic neural network acceleration system using heterogeneous computing devices. The key idea is to distribute compute-intensive operations on highly specialized ASICs for maximum performance, and auxiliary operations on flexible devices (e.g., FPGA, GPU) for better adaptability. We emphasize that a naïve integration of different devices fails to deliver high performance due to high communication overheads. To address this communication inefficiency, we propose an efficient FPGA-based device orchestration utilizing direct device-to-device communication and fine-grained operation scheduling. In this way, our system alleviates the communication overhead between heterogeneous devices by removing expensive kernel stack traversal and leveraging computation units and communication links in parallel. The evaluation using popular neural networks with emerging auxiliary operations shows that our system achieves both flexibility and high performance for various cases from single-accelerator training to distributed training (2.6–8.9× speedup).

Multi-scale convolutional neural network for texture recognition

Texture is of great significance for humans and robots to recognize the surface features of objects. In the field of texture recognition, methods based on spatial information have been widely applied. However, in the case of fine texture recognition, the methods only using spatial features for texture recognition may ignore the features of small texture and result in poor recognition accuracy. In this paper, a Multi-Scale Convolutional Neural Network (MS-CNN) is proposed to recognize millimetric fine textures. MS-CNN has three paths to extract features of different time scales from different numbers of continuous pressure images. The three paths have the same backbone network structure, but the number of convolution cores of the convolution layer in the backbone network of adjacent paths is doubled. After the convolution layer, we add SE-Net to automatically obtain the importance of each feature channel through learning, and then improve the useful features to further improve the accuracy. Finally, the output of all paths is averaged, and the classification vector is calculated through the final full connection layer. To validate MS-CNN, data sets containing 9 kinds of millimetric fine textures are obtained by flexible tactile sensors. The pressure image is transformed into one-dimensional vectors and these vectors are arranged into a sample in time order as the input of MS-CNN. In addition, the attention mechanism module is applied to MS-CNN to train the weight of each convolution channel and increase the proportion of useful features in the network. Ablation experiments prove that our modification is effective and our method achieves an accuracy of 81.83% for 9 fine textures. Compared with traditional recognition methods, our method achieves better recognition performance.

Tool Wear Prediction Based on a Multi-Scale Convolutional Neural Network with Attention Fusion

Compared with traditional machine learning algorithms, the convolutional neural network (CNN) has an excellent automatic feature learning ability and can complete the nonlinear representation from original data input to output by itself. However, the CNN does not sufficiently mine the tool wear information contained in the multi-sensor data due to disregard of the differences in the contribution of different features when extracting features. In this paper, a tool wear prediction method based on a multi-scale convolutional neural network with attention fusion is proposed, which fuses the tool wear degradation information collected by different types of sensors. In the multi-scale convolution module, convolution kernels with different sizes are used to extract the degradation information of different scales in the wear information, and then the attention fusion module is constructed to fuse the multi-scale feature information. Finally, the mapping between tool wear and multi-sensor data is realized through the feature information obtained by residual connection and full connection layer. By comparing the multi-scale convolutional neural network with different attention mechanisms, the experiments demonstrated the effectiveness and superiority of the proposed method.

Data-driven soliton mappings for integrable fractional nonlinear wave equations via deep learning with Fourier neural operator

In this paper, we firstly extend the Fourier neural operator (FNO) to discovery the mapping between two infinite-dimensional function spaces, where one is the fractional-order index space {ϵ|ϵ∈(0,1)} in the fractional integrable nonlinear wave equations while another denotes the soliton solution in the spatio-temporal function space. In other words, once the FNO network is trained, for any given ϵ∈(0,1), the corresponding soliton solution can be quickly obtained. To be specific, the soliton solutions are learned for the fractional nonlinear Schrödinger (fNLS), fractional Korteweg–de Vries (fKdV), fractional modified Korteweg–de Vries (fmKdV) and fractional sine-Gordon (fsineG) equations. The FNO architecture is utilized to learn the soliton mappings of the above four equations. The data-driven solitons are also compared with exact solutions to illustrate the powerful approximation capability of the FNO. Moreover, we study the influences of several critical factors (e.g., activation functions containing Relu(x), Sigmoid(x), Swish(x) and the new one xtanh(x), channels of fully connected layer) on the performance of the FNO algorithm. As a result, we find that the xtanh(x) and Swish(x) functions perform better than the Relu(x) and Sigmoid(x) functions in the FNO, and the FNO network with a more-channel fully-connected layer performs better as we expect. These results obtained in this paper will be useful to further understand the neural networks for fractional integrable nonlinear wave equations and the mappings between two infinite-dimensional function spaces.

Improved generalization performance of convolutional neural networks with LossDA

In recent years, convolutional neural networks (CNNs) have been used in many fields. Nowadays, CNNs have a high learning capability, and this learning capability is accompanied by a more complex model architecture. Complex model architectures allow CNNs to learn more data features, but such a learning process tends to reduce the training model’s ability to generalize to unknown data, and may be associated with problems of overfitting. Although many regularization methods have been proposed, such as data augmentation, batch normalization, and Dropout, research on improving generalization performance is still a common concern in the training process of robust CNNs. In this paper, we propose a dynamically controllable adjustment method, which we call LossDA, that embeds a disturbance variable in the fully-connected layer. The trend of this variable is kept consistent with the training loss, while the magnitude of the variable can be preset to adapt to the training process of different models. Through this dynamic adjustment, the training process of CNNs can be adaptively adjusted. The whole regularization process can improve the generalization performance of CNNs while helping to suppress overfitting. To evaluate this method, this paper conducts comparative experiments on MNIST, FashionMNIST, CIFAR-10, Cats_vs_Dogs, and miniImagenet datasets. The experimental results show that the method can improve the model performance of Light CNNs and Transfer CNNs (InceptionResNet, VGG19, ResNet50, and InceptionV3). The average maximum improvement in accuracy of Light CNNs is 4.62%, F1 is 3.99%, and Recall is 4.69%. The average maximum improvement accuracy of Transfer CNNs is 4.17%, F1 is 5.64%, and Recall is 4.05%.

Detecting the Same Pattern in Choreography Balinese Dance Using Convolutional Neural Network and Analysis Suffix Tree

The Balinese dances that are popular today were created by maestros who have existed since time immemorial. To develop the dances made by the existing maestro, one must know the characteristics of each dance based on the motion used. The help of digital image processing and string algorithm analysis methods will help to determine the characteristics of a dance. The algorithm used for dance analysis is the Suffix Tree, where the suffix tree is one of the algorithms that can be used to find patterns from input strings. The string to be analyzed is a series of codes performed by the classifier. The classifier used is Convolutional Neural Network. This method uses an image as its input, which will later perform convolution operations and perform a full-connected layer. The results were obtained using the Convolutional Neural Network method with Alexnet architecture as the classification and confusion matrix to calculate the level of accuracy of the test set, the best accuracy for the head is by using parameter learning rate 0.001, epoch 150, and RGB color space obtained 95% accuracy, 88% precision, 78% recall, and 82% f1-score. For the full body, using a learning rate of 0.01, epoch 150, and RGB color space, the accuracy is 85%, precision is 79%, recall is 64%, and f1-score is 69%. For the legs, using a learning rate of 0.001, epoch 150, and RGB color space, the accuracy is 92%, precision is 84%, recall is 59%, and f1-score is 65%. The results of the suffix tree analysis between codes that use ground truth and classification results have similar values, although the results of the movement patterns obtained by the suffix tree algorithm have not varied, which is dominated by class A because class A is the dominant class in each dance.

Transformer Based Binocular Disparity Prediction with Occlusion Predict and Novel Full Connection Layers.

The depth estimation algorithm based on the convolutional neural network has many limitations and defects by constructing matching cost volume to calculate the disparity: using a limited disparity range, the authentic disparity beyond the predetermined range can not be acquired; Besides, the matching process lacks constraints on occlusion and matching uniqueness; Also, as a local feature extractor, a convolutional neural network lacks the ability of global context information perception. Aiming at the problems in the matching method of constructing matching cost volume, we propose a disparity prediction algorithm based on Transformer, which specifically comprises the Swin-SPP module for feature extraction based on Swin Transformer, Transformer disparity matching network based on self-attention and cross-attention mechanism, and occlusion prediction sub-network. In addition, we propose a double skip connection fully connected layer to solve the problems of gradient vanishing and explosion during the training process for the Transformer model, thus further enhancing inference accuracy. The proposed model in this paper achieved an EPE (Absolute error) of 0.57 and 0.61, and a 3PE (Percentage error greater than 3 px) of 1.74% and 1.56% on KITTI 2012 and KITTI 2015 datasets, respectively, with an inference time of 0.46 s and parameters as low as only 2.6 M, showing great advantages compared with other algorithms in various evaluation metrics.

No-reference image quality assessment with multi-scale weighted residuals and channel attention mechanism

With the rapid development of deep learning, no-reference image quality assessment (NR-IQA) based on convolutional neural network (CNN) plays an important role in image processing. Currently, most CNN-based NR-IQA methods focus primarily on the global features of images while ignoring detail-rich local features and channel dependencies. In fact, there are subtle differences in detail between distorted and reference images, as well as differences in the contribution of different channels to IQA. Furthermore, multi-scale feature extraction can be used to fuse the detailed information from images with different resolutions, and the combination of global and local features is critical in extracting image features. As a result, in this paper, a multi-scale residual CNN with an attention mechanism (MsRCANet) is proposed for NR-IQA. Specifically, a multi-scale residual block is first used to extract features from distorted images. Then, the residual learning with active weighted mapping strategy and channel attention mechanism is used to further process image features to obtain more abundant information. Finally, the fusion strategy and full connection layer are used to evaluate image quality. The experimental results on four synthetic databases and three in-the-wild IQA databases, as well as cross-database validation results, show that the proposed method has good generalization ability and can be compared with the most advanced methods.