CNN Layers Research Articles

Enhancing Invariant Feature Learning via Cross‐Composition and Self‐Enrichment Normalization for Visible‐Infrared Person Re‐Identification

ABSTRACTVisible‐Infrared Person Re‐Identification (VI‐ReID) is a complex challenge in cross‐modality retrieval, wherein the goal is to recognize individuals from images captured via RGB and IR cameras. While many existing methods focus on narrowing the gap between different modalities through designing various feature‐level constraints, they often neglect the consistency of channel statistics information across the modalities, which results in suboptimal matching performance. In this work, we introduce a new approach for VI‐ReID that incorporates Cross‐Composition Normalization (CCN) and Self‐Enrichment Normalization (SEN). Specifically, Cross‐Composition Normalization is a plug‐and‐play module that can be seamlessly integrated into shallow CNN layers without requiring modifications to the training objectives. It probabilistically blends feature statistics between instances, thereby fostering the model's ability to learn inter‐modality feature distributions. Conversely, Self‐Enrichment Normalization leverages attention mechanisms to recalibrate statistics, effectively bridging the gaps between training and test distributions. This enhancement markedly boosts the discriminability of features in VI‐ReID tasks. To validate the efficacy of our proposed method, we carried out comprehensive experiments on two public cross‐modality datasets. The results clearly demonstrate the superiority of our Cross‐Composition and Self‐Enrichment normalization techniques in addressing the challenges of the VI‐ReID problem.

A Multi-Architecture Approach for Offensive Language Identification Combining Classical Natural Language Processing and BERT-Variant Models

Offensive content is a complex and multifaceted form of harmful material that targets individuals or groups. In recent years, offensive language (OL) has become increasingly harmful, as it incites violence and intolerance. The automatic identification of OL on social networks is essential to curtail the spread of harmful content. We address this problem by developing an architecture to effectively respond to and mitigate the impact of offensive content on society. In this paper, we use the Davidson dataset containing 24,783 samples of tweets and proposed three different architectures for detecting OL on social media platforms. Our proposed approach involves concatenation of features (TF-IDF, Word2Vec, sentiments, and FKRA/FRE) and a baseline machine learning model for the classification. We explore the effectiveness of different dimensions of GloVe embeddings in conjunction with deep learning models for classifying OL. We also propose an architecture that utilizes advanced transformer models such as BERT, ALBERT, and ELECTRA for pre-processing and encoding, with 1D CNN and neural network layers serving as the classification components. We achieve the highest precision, recall, and F1 score, i.e., 0.89, 0.90, and 0.90, respectively, for both the “bert encased preprocess/1 + small bert/L4H512A8/1 + neural network layers” model and the “bert encased preprocess/1 + electra small/2 + cnn” architecture.

Multi 2D-CNN-based model for short-term PV power forecast embedded with Laplacian Attention

Amid the bloom of Renewable energy (RE) integrated into the grid, an accurate Photovoltaic(PV) power forecast is considered to be a crucial task in maintaining the reliability and stability of the power systems since this technology strongly depends on various external factors, causing the fluctuation in the output power. However, the poor quality of input data, which is very common in practical circumstances owing to the low-cost measurement and data acquisition devices, poses an enormous challenge for the predictive model to deeply extract the spatial and temporal correlation of the input data. This study proposes a Multi Two-Dimensional Convolutional Neural Network (2D-CNN) for short-term PV power forecast embedded with Laplacian Attention mechanism. By viewing the input sequences in a 2D form, the input map is constructed, and the interconnected feature among variables can be captured by convolution operation. Moreover, with the multiple CNN layers working in parallel architecture, different representations hidden inside the input map can be detected, enabling the proposed model to bring out promising performance across forecast time-step without modifying its initial parameters. In order to reduce the decay impact of irrelevant variables existing inside the input data, the Laplacian Attention mechanism is employed. The Attention matrix is dynamically modified during the training process to produce an accurate attention matrix, which represents the correlation between variables. Therefore, the model is able to focus on informative features and ignore negative ones. The experiments conducted on two different datasets with opposite characteristics provide deep insights into the strength of the proposed model over the baseline model, which strongly demonstrates the efficiency of the proposed model, especially when dealing with datasets bearing tough characteristics.

End-to-End Latency Optimization for Resilient Distributed Convolutional Neural Network Inference in Resource-Constrained Unmanned Aerial Vehicle Swarms

An unmanned aerial vehicle (UAV) swarm has emerged as a powerful tool for mission execution in a variety of applications supported by deep neural networks (DNNs). In the context of UAV swarms, conventional methods for efficient data processing involve transmitting data to cloud and edge servers. However, these methods often face limitations in adapting to real-time applications due to the low latency of cloud-based approaches and weak mobility of edge-based approaches. In this paper, a new system called deep reinforcement learning-based resilient layer distribution (DRL-RLD) for distributed inference is designed to minimize end-to-end latency in UAV swarm, considering the resource constraints of UAVs. The proposed system dynamically allocates CNN layers based on UAV-to-UAV and UAV-to-ground communication links to minimize end-to-end latency. It can also enhance resilience to maintain mission continuity by reallocating layers when inoperable UAVs occur. The performance of the proposed system was verified through simulations in terms of latency compared to the comparison baselines, and its robustness was demonstrated in the presence of inoperable UAVs.

Low power RTL implementation of Tiny-YOLO-v2 DCNN hardware accelerator on Virtex-7 FPGA

Deep Convolution Neural Networks (DCNNs) are widely used in real-time applications, including image classification, speech recognition, and object detection. However, there are challenges for real-time applications on portable devices like mobile phones or embedded systems. Most object detection models are optimized for desktop configurations, requiring fast GPUs. Shallower networks with fewer computational complexities have been proposed for real-time detection, but ultimately compromise detection accuracy. Performance and complexity trade-offs are significant for computationally complex deep networks. This work aims to implement the CNN-based object detection model Tiny-Yolo-v2 on a Field Programmable Gate Array (FPGA) using Register Transfer Logic (RTL) as a native language. Hardware implementation is synthesized on The AMD Virtex 7 FPGA VC709 Connectivity Kit using VHDL code on Vivado 2020.1. This is the first, up to the authors' knowledge, RTL implementation of the Tiny-Yolo-v2 object identification algorithm on FPGA. The power consumed by the CNN layers equals 7.09W at a frequency of 100MHz.

Novel state of charge estimation method of containerized Lithium–Ion battery energy storage system based on deep learning

State of charge (SOC) is a critical indicator for lithium–ion battery energy storage system. However, model-driven SOC estimation is challenging due to the coupling of internal charging and discharging processes, ion diffusion, and chemical reactions in the electrode materials. These factors result in frequent changes in their internal characteristics. Therefore, we devise a novel deep learning SOC estimation method based on the attention mechanism (A) and convolutional neural networks-long short term memory (CNN–LSTM) network model. The CNN layers are employed to extract the critical features of the battery dataset. The LSTM layers are utilized to capture the long-term dependencies in the time series data. The attention mechanism can assign attention weights to different features within the output of the convolution layer. The A–CNN–LSTM method not only amplifies the positive influence of critical knowledge but also provides a global reference. The comparative experiments of three datasets and ablation study are carried out. The root mean square error, mean absolute error, and mean absolute percentage error obtained from the A–CNN–LSTM model decreased by 32.75 %, 45.88 %, and 53.36 %, respectively, compared with the best results obtained from the existing CNN, LSTM, gate recurrent unit (GRU), CNN–LSTM, and CNN–GRU models. The proposed model demonstrates greater stability and consistency than other models, underscoring its efficacy in SOC estimation.

Multi-layer feature fusion and attention enhancement for fine-grained vehicle recognition research

Abstract Vehicle recognition technology is widely applied in automatic parking, traffic restrictions, and public security investigations, playing a significant role in the construction of intelligent transportation systems. Fine-grained vehicle recognition seeks to surpass conventional vehicle recognition by concentrating on more detailed sub-classifications. This task is more challenging due to the subtle inter-class differences and significant intra-class variations. Localization-classification subnetworks represent an efficacious approach frequently employed for this task, but previous research has typically relied on CNN deep feature maps for object localization, which suffer from the low resolution, leading to poor localization accuracy. The multi-layer feature fusion localization method proposed by us fuses the high-resolution feature map of the shallow layer of CNN with the deep feature map, and makes full use of the rich spatial information of the shallow feature map to achieve more precise object localization. In addition, traditional methods acquire local attention information through the design of complex models, frequently resulting in regional redundancy or information omission. To address this, we introduce an attention module that adaptively enhances the expressiveness of global features and generates global attention features. These global attention features are then integrated with object-level features and local attention cues to achieve a more comprehensive attention enhancement. Lastly, we devise a multi-branch model and employ the aforementioned object localization and attention enhancement methods for end-to-end training to make the multiple branches collaborate seamlessly to adequately extract fine-grained features. Extensive experiments conducted on the Stanford Cars dataset and the self-built Cars-126 dataset have demonstrated the effectiveness of our method, achieving a leading position among existing methods with 97.7% classification accuracy on the Stanford Cars dataset.

Convolution neural network based multi-class classification of rehabilitation exercises for diastasis recti abdominis using wearable EMG-IMU sensors

PurposeGlobally, postnatal women endure a prominent issue caused by midline separation of abdominal recti muscles, characterized by a sagging and pouch-like appearance of the belly termed as Diastasis Recti Abdominis (DRA). The necessity of ensuring the efficacy of rehabilitative workouts for individuals with DRA cannot be overstated, as inaccurate exercises can exacerbate the condition and deteriorate the health of affected women. The purpose of these exercises is to specifically focus on the rectus abdominis muscles to facilitate the reapproximation of the linea alba. The primary aim of this research work is to assess the effectiveness of rehabilitation exercises for DRA women obtained from Inertial Measurement Unit (IMU) and Electromyography (EMG) sensors.Design/methodology/approachConvolutional neural networks (CNN) employs convolutional activation functions and pooling layers. Recently, 1D CNNs have emerged as a promising approach used in various applications, including personalized biomedical data classification and early diagnosis, structural health monitoring and anomaly detection. Yet another significant benefit is the feasibility of a real-time and cost-effective implementation of 1D CNN. The EMG and IMU signals serve as inputs for the 1D CNN. Features are then extracted from the fully connected layer of the CNN and fed into a boosting machine learning algorithm for classification.FindingsThe findings demonstrate that a combination of sensors provides more details about the exercises, thereby contributing to the classification accuracy.Practical implicationsIn real time, collecting data from postnatal women was incredibly challenging. The process of examining these women was time-consuming, and they were often preoccupied with their newborns, leading to a reluctance to focus on their own health. Additionally, postnatal women might not be fully aware of the implications of DRA and the importance of rehabilitation exercises. Many might not realize that neglecting DRA can lead to long-term issues such as back pain, pelvic floor dysfunction, and compromised core strength.Social implicationsDuring our data collection camps, there were educational sessions to raise awareness about the DRA problem and the benefits of rehabilitation exercises. This dual approach helped in building trust and encouraging participation. Moreover, the use of wearable sensors in this study provided a non-invasive and convenient way for new mothers to engage in rehabilitation exercises without needing frequent visits to a clinic, which is often impractical for them.Originality/valueThe utilization of discriminating features retrieved from the output layer of 1D CNN is a significant contribution to this work. The responses of this study indicate that 1D convolutional neural network (1D CNN) and Boosting algorithms used in a transfer learning strategy produce successful discrimination between accurate and inaccurate performance of exercises by achieving an accuracy of 96%.

A cloud 15kV-HDPE insulator leakage current classification based improved particle swarm optimization and LSTM-CNN deep learning approach

Real-time insulator leakage current classification is crucial in preventing the pollution flashover phenomenon and providing appropriate maintenance schedules in high-voltage transmission towers. However, current methodologies only utilize traditional artificial neural networks, which have limitations when performing big data analysis. This research developed a novel cloud 15kV-HDPE insulator leakage current classified framework, utilizing a long short-term memory convolutional neural network (LSTM-CNN). The hybrid model structure is optimized through hyperparameter fine-tuning based on improved particle swarm optimization (IPSO), which reduces human effort and considerable time compared with PSO and random search (RS) techniques. The IPSO-LSTM-CNN model can productively identify correlations between selected weather features and target leakage current levels of 15kV-HDPE insulators. LSTM efficiently captures long-term patterns in sequential data, while CNN layers competently extract high-level dependency in time-invariant information. Four 15kV-HDPE insulators’ datasets, collected in high-voltage transmission lines in the coastal area of Taiwan for more than one year, are deployed for analyzing and comparing classified performance. Other conventional models are developed to evaluate and compare classified performance with the proposed IPSO-LSTM-CNN approach, which acquires the most significant enhancement of 48.08 % loss, 45.91 % validating loss, 52.57 % MAE, 35.47 % validating MAE, 47.34 % MSE, 27.02 % validating MSE, 9.15 % PRE, 3.40 % validating PRE, 4.76 % REC, and 6.17 % validating REC. The experiment outcomes demonstrate that the developed IPSO-LSTM-CNN model acquires improved robustness and accuracy in the leakage current classified capability of 15kV-HDPE insulators.

A Deep-learning-based Auto Encoder-Decoder Model for Denoising Electrocardiogram Signals

Learning-based denoising techniques have become superior to the traditional assumption-based denoising methods in this modern era. Also, with the advancement of wearable technologies and remote electrocardiogram (ECG) monitoring systems, the requirement for optimal storage has increased due to the limited availability of hardware resources. Therefore, denoising and compression both are essential at the preprocessing stage of the ECG signal. Deep learning-based denoising auto encoder-decoder (DAED) models guarantee cutting-edge performance for these tasks. This article presents a lightweight, adaptive, hybrid Convolutional Neural Network-Gated Recurrent Unit (CNN-GRU) based DAED model that achieves a signal compression ratio of 64 with high signal-to-noise ratio improvement for the elimination of ECG noises. The novelty of this work lies in the customization of the CNN layers and utilization of the advantages of the GRU layer in a proper channel for compression and denoising ECG signals. The comparative study with other complex deep learning-based DAED arrangements and state-of-the-art denoising techniques shows the proposed model has simplicity in construction and an improved signal-to-noise ratio with minimum mean square error.

An attention-based hybrid model for spatial and temporal sentiment analysis of COVID-19 related tweets in the contiguous United States

ABSTRACT Understanding the sentiments of social media posts can help health authorities respond to disease outbreaks, through a proxy measure of fear, confidence, and community compliance. Sentiment analysis identifies a pattern of emotion through the written word and assigns a positive, neutral, or negative value to it. As of February 2023, there were 677.7 million confirmed cases of coronavirus disease (COVID-19) and more than 6.7 million confirmed deaths. In this paper, around 170,000 COVID-19-related tweets were collected between September 2020 and January 2021 in the contiguous United States. Data preprocessing and exploratory investigation were completed for analysis of the collected dataset. Further, a novel and unified architecture called attention-based one-dimensional convolution with bidirectional long short-term memory layers (CNN-BiLSTM-ATT) is proposed to classify people’s sentiments as positive, neutral, and negative based on COVID-19-related tweets. In the CNN-BiLSTM-ATT model, the CNN layer can extract the low-level semantic features from textual data, and the BiLSTM layer can extract both the previous and future contextual representations. The attention module can improve the information focus from the outputted layer of the BiLSTM. The proposed method can extract both the local phrase representations and the global feature of sentences. Numerical experiments were conducted on COVID-19-related tweets using the proposed method and other baseline models to compare their performances. Our experimental results demonstrate that the CNN-BiLSTM-ATT model achieves an average accuracy of 95.16% and a macro-average F1-score of 95.12%, which outperforms the baseline models.

Low‐light image enhancement via lightweight custom non‐linear transform network

AbstractConvolutional neural network (CNN)‐based models have shown significant progress in low light image enhancement. However, many existing models possess a large number of parameters, making them unsuitable for deployment on terminal devices. Moreover, adjustments to brightness, contrast, and colour in images are often non‐linear, and convolution is not the best at capturing complex non‐linear relationships in image data. To address these issues, a model based on an end‐to‐end custom non‐linear transform network (CNTNet) is proposed. CNTNet combines a custom non‐linear transform layer with CNN layers to achieve image contrast and detail enhancement. The CNT layer in this model introduces transformation parameters at multiple scales to manipulate input images within various ranges. CNTNet progressively processes images by stacking multiple non‐linear transform layers and convolutional layers while integrating residual connections to capture and leverage subtle image features. The final output is generated through convolutional layers to obtain enhanced images. Experimental results of CNTNet demonstrate that, while maintaining a comparable level of image quality evaluation metrics to mainstream models, it significantly reduces the parameter count to only 2K.

Laplacian eigenmaps based manifold regularized CNN for visual recognition

The paper proposes a novel Laplacian eigenmaps based manifold regularized CNN (LE-CNN) for action recognition. The proposed LE-CNN model incorporates Laplacian eigenmaps based manifold structure information of training samples into CNN layer by layer, which can keep adjacent samples as close as possible during space transformation. In addition, the Laplacian eigenmaps based manifold structure can accelerate convergence during the training process. Experiments are performed on two standard action datasets, UCF101 and HMDB51, to validate the proposed LE-CNN model. Furthermore, we extend experiments on a large-scale action recognition dataset (i.e. the Kinetics dataset) and compare the LE-CNN model with other advanced models. In addition to action recognition, we also apply the LE-CNN model to an image classification task on the CIFAR-10 dataset, to demonstrate the effectiveness of the LE-CNN model across different classification tasks.

Detection of aberration in human behavior using shallow neural network over smartphone inertial sensors data

AbstractThe integration of different Mobile Edge Computing (MEC) applications has significantly enhanced the realm of security and surveillance, with Human Activity Recognition (HAR) standing out as a crucial application. The diverse sensors found in smartphones have made it convenient for monitoring applications to gather and analyze data, rendering them valuable for HAR purposes. Moreover, MEC can be employed to automate surveillance, allowing intelligent monitoring of restricted areas to identify and respond to unwanted or suspicious activities. This research develops a system using motion sensors in smartphones to identify unusual human activities. People's smartphones were employed to monitor both suspicious and regular activities. Information was collected for various actions categorized as either suspicious or regular. When a person performs a certain action, the smartphone records a series of sensory data, analyse important patterns from the basic data, and then determines what the person is doing by combining information from different sensors. To prepare the data, information from different sensors was aligned to a shared timeline. In this study, we used a sliding window approach on synchronized data to feed sequences into LSTM and CNN models. These models, which include initial layers of LSTM and CNN, automatically find important patterns in the order of human activities. We combined SVM with the features extracted by the shallow Neural Network to make a mixed model that predicts suspicious activities. Lastly, we compared LSTM, CNN, and our new shallow mixed neural network using a new real‐time dataset. The mixed model of CNN and SVM achieved an accuracy of 94.43%. Additionally, the sliding window method's effectiveness was confirmed with a 4.28% improvement in accuracy.

Temporal attention network for CNNE model of variable-length ECG signals in early arrhythmia detection

<p>Cardiac arrhythmia identification and categorization are crucial for prompt treatment and better patient outcomes. Arrhythmia identification is the main focus of this study's temporal attention network (TAN)-based multiclass categorization of varied-length electrocardiogram (ECG) data. The suggested TAN is designed to handle variable-duration ECG signals, making it ideal for real-time monitoring. The TAN uses a dynamic snippet extraction approach to choose meaningful ECG segments to ensure the model captures essential properties despite the constraints of processing such heterogeneous data. Training and assessment use a large dataset of atrial fibrillation, ventricular, and supraventricular arrhythmias. The TAN outperforms current approaches in multiclass early arrhythmia classification and is very accurate. Concatenating EfficientNet with CNN layer helped overcome different data and variable-length signals. High accuracy: 98% of normal, 97.1% of atrial fibrillation (AF), 98% of other, and 98% of noisy using the proposed CEEC model. Early arrhythmia diagnosis has improved due to the TAN's ability to effectively identify varied-length ECG data and give interpretability. It enables quicker interventions, personalised treatment plans, and improved arrhythmia control, which can greatly benefit patient care.</p>

A semantic guidance-based fusion network for multi-label image classification

Multi-label image classification (MLIC), a fundamental task assigning multiple labels to each image, has been seen notable progress in recent years. Considering simultaneous appearances of objects in the physical world, modeling object correlations is crucial for enhancing classification accuracy. This involves accounting for spatial image feature correlation and label semantic correlation. However, existing methods struggle to establish these correlations due to complex spatial location and label semantic relationships. On the other hand, regarding the fusion of image feature relevance and label semantic relevance, existing methods typically learn a semantic representation in the final CNN layer to combine spatial and label semantic correlations. However, different CNN layers capture features at diverse scales and possess distinct discriminative abilities. To address these issues, in this paper we introduce the Semantic Guidance-Based Fusion Network (SGFN) for MLIC. To model spatial image feature correlation, we leverage the advanced TResNet architecture as the backbone network and employ the Feature Aggregation Module for capturing global spatial correlation. For label semantic correlation, we establish both local and global semantic correlation. We further enrich model features by learning semantic representations across multiple convolutional layers. Our method outperforms current state-of-the-art techniques on PASCAL VOC (2007, 2012) and MS-COCO datasets.

Predicting the Future Capacity and Remaining Useful Life of Lithium-Ion Batteries Based on Deep Transfer Learning

Lithium-ion batteries are widely utilized in numerous applications, making it essential to precisely predict their degradation trajectory and remaining useful life (RUL). To improve the stability and applicability of RUL prediction for lithium-ion batteries, this paper uses a new method to predict RUL by combining CNN-LSTM-Attention with transfer learning. The presented model merges the strengths of both convolutional and sequential architectures, and it enhances the model’s capability to grasp comprehensive information by utilizing the attention mechanism, thereby boosting overall performance. The CEEMDAN algorithm is used for NASA batteries with obvious capacity regeneration phenomena to alleviate the difficulties caused by capacity regeneration on model prediction. During the model transfer phase, the CNN and LSTM layers of the pre-trained model from the source domain are kept unchanged during retraining, while the attention and fully connected layers are fine-tuned for NASA batteries and self-tested NCM batteries. The final results indicate that this method achieves superior accuracy relative to other methods while addressing the issue of limited labeled data in the target domain through transfer learning, thereby enhancing the model’s transferability and generalization capabilities.

Data-driven strategy for state of health prediction and anomaly detection in lithium-ion batteries

This study addresses the crucial challenge of monitoring the State of Health (SOH) of Lithium-Ion Batteries (LIBs) in response to the escalating demand for renewable energy systems and the imperative to reduce CO2 emissions. The research introduces deep learning (DL) models, namely Encoder-Long Short-Term Memory (E-LSTM) and Convolutional Neural Network-LSTM (CNN–LSTM), each designed to forecast battery SOH. E-LSTM integrates an encoder for dimensionality reduction and an LSTM model to capture data dependencies. CNN–LSTM, on the other hand, employs CNN layers for encoding followed by LSTM layers for precise SOH estimation. Significantly, we prioritize model explainability by employing a game-theoretic approach known as SHapley Additive exPlanations (SHAP) to elucidate the output of our models. Furthermore, a method based on pattern mining was developed, synergizing with the model, to identify patterns contributing to abnormal SOH decrease. These insights are presented through informative plots. The proposed approach relies on the battery dataset from the Massachusetts Institute of Technology (MIT) and showcases promising results in accurately estimating SOH values, in which the E-LSTM model outperformed the CNN–LSTM model with a Mean Absolute Error (MAE) of less than 1%.

Prediction of PM2.5 concentration based on a CNN-LSTM neural network algorithm.

Fine particulate matter (PM2.5) is a major air pollutant affecting human survival, development and health. By predicting the spatial distribution concentration of PM2.5, pollutant sources can be better traced, allowing measures to protect human health to be implemented. Thus, the purpose of this study is to predict and analyze the PM2.5 concentration of stations based on the integrated deep learning of a convolutional neural network long short-term memory (CNN-LSTM) model. To solve the complexity and nonlinear characteristics of PM2.5 time series data problems, we adopted the CNN-LSTM deep learning model. We collected the PM2.5data of Qingdao in 2020 as well as meteorological factors such as temperature, wind speed and air pressure for pre-processing and characteristic analysis. Then, the CNN-LSTM deep learning model was integrated to capture the temporal and spatial features and trends in the data. The CNN layer was used to extract spatial features, while the LSTM layer was used to learn time dependencies. Through comparative experiments and model evaluation, we found that the CNN-LSTM model can achieve excellent PM2.5 prediction performance. The results show that the coefficient of determination (R2) is 0.91, and the root mean square error (RMSE) is 8.216µg/m3. The CNN-LSTM model achieves better prediction accuracy and generalizability compared with those of the CNN and LSTM models (R2 values of 0.85 and 0.83, respectively, and RMSE values of 11.356 and 14.367, respectively). Finally, we analyzed and explained the predicted results. We also found that some meteorological factors (such as air temperature, pressure, and wind speed) have significant effects on the PM2.5 concentration at ground stations in Qingdao. In summary, by using deep learning methods, we obtained better prediction performance and revealed the association between PM2.5 concentration and meteorological factors. These findings are of great significance for improving the quality of the atmospheric environment and protecting public health.

A physical constraint-based machine learning model for shale oil production prediction

Shale oil has become a crucial unconventional resource, bolstering energy supply security, and it is important to accurately predict shale oil production dynamics. However, traditional prediction methods are faced with the challenges of large data fluctuations and numerous interference factors, which make accurate prediction difficult. This paper introduces a deep learning approach, specifically a physical constraint-based convolutional neural network combined with long short-term memory and attention mechanism (CNN-LSTM-AM), to predict shale oil production dynamics. Initially, the correlation analysis method is used to analyze the correlation strengths of features with the prediction target, and the data that are most strongly correlated with the daily oil production are screened out and preprocessed. Key features are then extracted; that is, the CNN layer's output is fed into the LSTM layer, the output of which informs a fully connected layer for time-series production predictions. The model integrates an AM to concentrate on vital data aspects. In the “five-linear flow” formula, the imbibition is hard-coded into the shale oil production prediction model. Physical constraints are introduced into the model. Data driven and empirical formulas are used to introduce a loss function as a constraint condition in the training process of a machine learning model to improve the interpretability and predictive ability of the model. During the tuning phase, Bayesian optimization is used to fine-tune the model's hyperparameters. Shapley additive explanation and accumulated local effects analyses are used to further assess and quantify the significance of the essential parameters, thus optimizing the prediction effectiveness.