Reduce Model Complexity Research Articles

In Silico Acute Aquatic Hazard Assessment and Prioritization Using a Grouped Target Site Model: A Case Study of Organic Substances Reported in Permian Basin Hydraulic Fracturing Operations.

Hydraulic fracturing (HF) is commonly used to enhance onshore recovery of oil and gas during production. This process involves the use of a variety of chemicals to support the physical extraction of oil and gas, maintain appropriate conditions downhole (e.g., redox conditions, pH), and limit microbial growth. The diversity of chemicals used in HF presents a significant challenge for risk assessment. The objective of the present study is to establish a transparent, reproducible procedure for estimating 5th percentile acute aquatic hazard concentrations (e.g., acute hazard concentration 5th percentiles [HC5s]) for these substances and validating against existing toxicity data. A simplified, grouped target site model (gTSM) was developed using a database (n = 1696) of diverse compounds with known mode of action (MoA) information. Statistical significance testing was employed to reduce model complexity by combining 11 discrete MoAs into three general hazard groups. The new model was trained and validated using an 80:20 allocation of the experimental database. The gTSM predicts toxicity using a combination of target site water partition coefficients and hazard group-based critical target site concentrations. Model performance was comparable to the original TSM using 40% fewer parameters. Model predictions were judged to be sufficiently reliable and the gTSM was further used to prioritize a subset of reported Permian Basin HF substances for risk evaluation. The gTSM was applied to predict hazard groups, species acute toxicity, and acute HC5s for 186 organic compounds (neutral and ionic). Toxicity predictions and acute HC5 estimates were validated against measured acute toxicity data compiled for HF substances. This case study supports the gTSM as an efficient, cost-effective computational tool for rapid aquatic hazard assessment of diverse organic chemicals. Environ Toxicol Chem 2024;43:1161-1172. © 2024 ExxonMobil Petroleum and Chemical BV. Environmental Toxicology and Chemistry published by Wiley Periodicals LLC on behalf of SETAC.

Data-driven distributionally robust optimization approach for the coordinated dispatching of the power system considering the correlation of wind power

With the increasing penetration of large-scale wind power into the power grid, it is crucial to develop a precise model to accurately depict the stochasticity and correlation among wind farm outputs, which is highly important for ensuring the safe and efficient utilization of wind energy in grid dispatching. In this article, a data-driven distributionally robust optimization (DDRO) dispatching approach that accounts for spatial correlations among outputs from multiple wind farms is proposed. The proposed approach is applied to a source-network-load-storage grid system to ascertain unit start-stop schedules and resource allocation effectively. First, a truncated spatial correlation model is proposed, enabling a comprehensive representation of spatial correlations and output constraints between distinct wind farms. Second, the ISODATA clustering algorithm is employed to generate typical scenarios, reduce model complexity, and expedite the computation process. Third, a unit commitment model considering the demand response is constructed and solved using the DDRO approach. Finally, the proposed model is applied to the IEEE 30-bus system to test its robustness and cost-effectiveness compared to the traditional robust optimization model. Additionally, it is applied to the IEEE 118-bus system to demonstrate its scalability and stability.

GY-SLAM: A Dense Semantic SLAM System for Plant Factory Transport Robots.

Simultaneous Localization and Mapping (SLAM), as one of the core technologies in intelligent robotics, has gained substantial attention in recent years. Addressing the limitations of SLAM systems in dynamic environments, this research proposes a system specifically designed for plant factory transportation environments, named GY-SLAM. GY-SLAM incorporates a lightweight target detection network, GY, based on YOLOv5, which utilizes GhostNet as the backbone network. This integration is further enhanced with CoordConv coordinate convolution, CARAFE up-sampling operators, and the SE attention mechanism, leading to simultaneous improvements in detection accuracy and model complexity reduction. While mAP@0.5 increased by 0.514% to 95.364, the model simultaneously reduced the number of parameters by 43.976%, computational cost by 46.488%, and model size by 41.752%. Additionally, the system constructs pure static octree maps and grid maps. Tests conducted on the TUM dataset and a proprietary dataset demonstrate that GY-SLAM significantly outperforms ORB-SLAM3 in dynamic scenarios in terms of system localization accuracy and robustness. It shows a remarkable 92.59% improvement in RMSE for Absolute Trajectory Error (ATE), along with a 93.11% improvement in RMSE for the translational drift of Relative Pose Error (RPE) and a 92.89% improvement in RMSE for the rotational drift of RPE. Compared to YOLOv5s, the GY model brings a 41.5944% improvement in detection speed and a 17.7975% increase in SLAM operation speed to the system, indicating strong competitiveness and real-time capabilities. These results validate the effectiveness of GY-SLAM in dynamic environments and provide substantial support for the automation of logistics tasks by robots in specific contexts.

Developing a New Constitutive Model of High Damping Rubber by Combining GRU and Attention Mechanism.

High damping rubber (HDR) bearings are extensively used in seismic design for bridges due to their remarkable energy dissipation capabilities, which is critical during earthquakes. A thorough assessment of crucial factors such as temperature, rate, experienced maximum amplitude, and the Mullins effect of HDR on the mechanics-based constitutive model of HDR is lacking. To address this issue, we propose a deep learning approach that integrates the Gate Recurrent Unit (GRU) and attention mechanism to identify time series characteristics from compression-shear test data of HDR specimens. It is shown that the combination of GRU and attention mechanism enables accurate prediction of the mechanical behavior of HDR specimens. Compared to the sole use of GRU, this suggested method significantly reduces model complexity and computation time while maintaining good prediction performance. Therefore, it offers a new approach to constructing the HDR constitutive model. Finally, the HDR constitutive model was used to analyze the impact of experienced maximum amplitudes and cycles on following processes. It was observed that maximum amplitudes directly influence the stress-strain relationship of HDR during subsequent processes. Consequently, a solid foundation is laid for evaluating the responses of HDR bearings under earthquakes.

Lightweight air-to-air unmanned aerial vehicle target detection model

The rapid expansion of the drone industry has resulted in a substantial increase in the number of low-altitude drones, giving rise to concerns regarding collision avoidance and countermeasure strategies among these unmanned aerial vehicles. These challenges underscore the urgent need for air-to-air drone target detection. An effective target detection model must exhibit high accuracy, real-time capabilities, and a lightweight network architecture to achieve a balance between precision and speed when deployed on embedded devices. In response to these requirements, we initially curated a dataset comprising over 10,000 images of low-altitude operating drones. This dataset encompasses diverse and intricate backgrounds, significantly enhancing the model’s training capacity. Subsequently, a series of enhancements were applied to the YOLOv5 algorithm to realize lightweight object detection. A novel feature extraction network, CF2-MC, streamlined the feature extraction process, while an innovative module, MG, in the feature fusion section aimed to improve detection accuracy and reduce model complexity. Concurrently, the original CIoU loss function was replaced with the EIoU loss function to further augment the model’s accuracy. Experimental results demonstrate an enhancement in the accuracy of drone target detection, achieving mAP values of 95.4% on the UAVfly dataset and 82.2% on the Det-Fly dataset. Finally, real-world testing conducted on the Jetson TX2 revealed that the YOLOv5s-ngn model achieved an average inference speed of 14.5 milliseconds per image. The code utilized in this paper can be accessed via https://github.com/lucien22588/yolov5-ngn.git.

VGG16 based on dilated convolution for face recognition

Face recognition has wide applications in fields such as security systems, biometrics, and human-computer interaction. However, traditional face recognition methods face challenges in capturing details and reducing model complexity. To address these issues, this paper proposes a new method based on VGG16, which improves recognition accuracy and reduces parameter quantity by introducing dilated convolution and parameter pruning. First, the hole convolution is introduced to expand the Receptive field and capture more details to enhance the ability of the model in distinguishing facial features. Next, parameter pruning is applied to reduce redundant parameters, optimize model structure, and improve computational efficiency. This article conducted experimental evaluation on the classic face recognition dataset CK+ dataset. The results show that the proposed method is significantly superior to the traditional VGG16 model in terms of recognition accuracy. At the same time, the use of pruning technology significantly reduces the number of parameters in the model and improves computational efficiency. The experimental outcomes conclusively validate the effectiveness and feasibility of the proposed method.

A Tracking-Based Two-Stage Framework for Spatio-Temporal Action Detection

Spatio-temporal action detection (STAD) is a task receiving widespread attention and has numerous application scenarios, such as video surveillance and smart education. Current studies follow a localization-based two-stage detection paradigm, which exploits a person detector for action localization and a feature processing model with a classifier for action classification. However, many issues occur due to the imbalance between task settings and model complexity in STAD. Firstly, the model complexity of heavy offline person detectors adds to the inference overhead. Secondly, the frame-level actor proposals are incompatible with the video-level feature aggregation and Region-of-Interest feature pooling in action classification, which limits the detection performance under diverse action motions and results in low detection accuracy. In this paper, we propose a tracking-based two-stage spatio-temporal action detection framework called TrAD. The key idea of TrAD is to build video-level consistency and reduce model complexity in our STAD framework by generating action track proposals among multiple video frames instead of actor proposals in a single frame. In particular, we utilize tailored tracking to simulate the behavior of human cognitive actions and used the captured motion trajectories as video-level proposals. We then integrate a proposal scaling method and a feature aggregation module into action classification to enhance feature pooling for detected tracks. Evaluations in the AVA dataset demonstrate that TrAD achieves SOTA performance with 29.7 mAP, while also facilitating a 58% reduction in overall computation compared to SlowFast.

Research on a Fault Diagnosis Method for the Braking Control System of an Electric Multiple Unit Based on Deep Learning Integration

A fault diagnosis method based on deep learning integration is proposed focusing on fault text data to effectively improve the efficiency of fault repair and the accuracy of fault localization in the braking control system of an electric multiple unit (EMU). First, the Borderline-SMOTE algorithm is employed to synthesize minority class samples at the boundary, addressing the data imbalance and optimizing the distribution of data within the fault text. Then, a multi-dimensional word representation is generated using the multi-layer bidirectional transformer architecture from the pre-training model, BERT. Next, BiLSTM captures bidirectional context semantics and, in combination with the attention mechanism, highlights key fault information. Finally, the LightGBM classifier is employed to reduce model complexity, enhance analysis efficiency, and increase the practicality of the method in engineering applications. An experimental analysis of fault data from the braking control system of the EMU indicates that the deep learning integration method can further improve diagnostic performance.

Rapid identification of bacteria in water by multi-wavelength transmittance spectroscopy and the artificial neural network

Background: Multi-wavelength transmittance spectroscopy, in combination with the artificial neural network, has been a novel tool used to identify and classify microorganisms in recent years.Methods: In our work, the transmittance spectra in the region from 200 to 900 nm for four bacterial species of interest, Escherichia coli (E. coli), Staphylococcus aureus (S. aureus), Klebsiella pneumoniae (K.pneumoniae), and Salmonella typhimurium (S. typhi), were recorded using an ultraviolet–visible spectrophotometer. Considering too much redundant data on the full-wave band spectra, the characteristic wavelength variables were selected using the competitive adaptive reweighting sampling (CARS) algorithm. Spectra of the initial training set of these targeted microorganisms were used to create identification models representing the spectral variability of each species using four kinds of neural networks, namely, backpropagation (BP), radial basis function network (RBF), generalized regression neural network (GRNN), and probabilistic neural network (PNN).Results: The blinded isolate spectra of targeted species were identified using the four identification models given above. Compared to fullband modeling, after using CARS to screen the wavelength variables, four identification models are established for the 35 preferred characteristic wavelengths, and the prediction performance of the four models is notably improved. Among them, the CARS–PNN model is the best, and the identification rates of all targeted bacteria were achieved with 100% accuracy; the calculation time is just approximately 0.04 s.Discussion: The use of CARS can effectively remove useless information from the spectra, reduce model complexity, and enhance model prediction performance. Multi-wavelength transmission spectroscopy, combined with the CARS–PNN method, can provide a new method for the rapid detection of bacteria in water and could be readily extended for bacterial microbiological detection in blood and food.

Effect of feature optimization on performance of machine learning models for predicting traffic incident duration

Developing a high-performing traffic incident-duration prediction model is considered a key component for evaluating the impact of these incidents on the roadway network. Various research studies have developed robust incident-duration prediction models. Still, they have faced many issues in providing an accurate prediction result due to the countless data modeling issues, such as complex correlations, highly skewed data distributions, heteroscedasticity, and outliers. This paper investigates the impact of feature optimization (FO) - a relatively new term encompassing two already-known topics: feature engineering (FE) and feature selection (FS) techniques - on the performance of several machine learning models developed for predicting incident durations. The models developed included multivariate linear regression, decision trees, support vector regressors, K-Nearest Neighbors regression, ensembles, and artificial neural networks. Various FO techniques have been used for each model to derive the massive traffic incidents dataset and repeat the prediction process. Our results show that the proposed filtering, wrapper, and embedded FS techniques can successfully reduce the number of features without sacrificing the prediction performance. Using log-normal transformation to deal with continuous data skewness, min-max normalization to deal with data variability, and principal component analysis (PCA) to reform the dataset into a smaller independent feature subset, FE techniques can enhance the accuracy of incident duration estimation over the assessed ML models. The best-performing FE technique was the PCA since performance improvements were observed across all developed ML models. The best-performing FS technique was the Recursive Feature Elimination, outperforming other tested techniques in reducing model complexity while maintaining model accuracy.

AFM-YOLOv8s: An Accurate, Fast, and Highly Robust Model for Detection of Sporangia of Plasmopara viticola with Various Morphological Variants.

Monitoring spores is crucial for predicting and preventing fungal- or oomycete-induced diseases like grapevine downy mildew. However, manual spore or sporangium detection using microscopes is time-consuming and labor-intensive, often resulting in low accuracy and slow processing speed. Emerging deep learning models like YOLOv8 aim to rapidly detect objects accurately but struggle with efficiency and accuracy when identifying various sporangia formations amidst complex backgrounds. To address these challenges, we developed an enhanced YOLOv8s, namely, AFM-YOLOv8s, by introducing an Adaptive Cross Fusion module, a lightweight feature extraction module FasterCSP (Faster Cross-Stage Partial Module), and a novel loss function MPDIoU (Minimum Point Distance Intersection over Union). AFM-YOLOv8s replaces the C2f module with FasterCSP, a more efficient feature extraction module, to reduce model parameter size and overall depth. In addition, we developed and integrated an Adaptive Cross Fusion Feature Pyramid Network to enhance the fusion of multiscale features within the YOLOv8 architecture. Last, we utilized the MPDIoU loss function to improve AFM-YOLOv8s' ability to locate bounding boxes and learn object spatial localization. Experimental results demonstrated AFM-YOLOv8s' effectiveness, achieving 91.3% accuracy (mean average precision at 50% IoU) on our custom grapevine downy mildew sporangium dataset-a notable improvement of 2.7% over the original YOLOv8 algorithm. FasterCSP reduced model complexity and size, enhanced deployment versatility, and improved real-time detection, chosen over C2f for easier integration despite minor accuracy trade-off. Currently, the AFM-YOLOv8s model is running as a backend algorithm in an open web application, providing valuable technical support for downy mildew prevention and control efforts and fungicide resistance studies.

Analisis DistilBERT dengan Support Vector Machine (SVM) untuk Klasifikasi Ujaran Kebencian pada Sosial Media Twitter

Hate speech is a significant issue in content management on social media platforms. Effective classification of hate speech plays a crucial role in maintaining a safe social media environment, combating discrimination, and protecting users. This study evaluates a hate speech classification model using SVM with linear and polynomial kernels. The dataset used consists of labeled Indonesian-language tweets. The importance of developing an effective classification model to address hate speech has led to the utilization of DistilBERT as a feature extraction method. However, DistilBERT has high-dimensional features, necessitating dimensionality reduction to reduce model complexity. Therefore, in this study, the PCA dimensionality reduction method is implemented with various scenarios of dimensionality, namely 10, 20, 30, 40, and 50. Evaluation is performed using F1-Score, and the entire study is evaluated using 10-fold cross-validation. The evaluation results indicate that in the scenario with a linear kernel, the model achieves the highest F1-Score of 0.75 in the 50-dimensional scenario. Meanwhile, in the scenario with a polynomial kernel, the model achieves the highest F1-Score of 0.7857 in the 50-dimensional scenario. These findings demonstrate that the use of a polynomial kernel with 50 dimensions yields the best performance in classifying hate speech.

Lightweight 2D Human Pose Estimation Based on Joint Channel Coordinate Attention Mechanism

Traditional human pose estimation methods typically rely on complex models and algorithms. Lite-HRNet can achieve an excellent performance while reducing model complexity. However, its feature extraction scale is relatively single, which can lead to lower keypoints’ localization accuracy in crowded and complex scenes. To address this issue, we propose a lightweight human pose estimation model based on a joint channel coordinate attention mechanism. This model provides a powerful information interaction channel, enabling features of different resolutions to interact more effectively. This interaction can solve the problem of human pose estimation in complex scenes and improve the robustness and accuracy of the pose estimation model. The introduction of the joint channel coordinate attention mechanism enables the model to more effectively retain key information, thereby enhancing keypoints’ localization accuracy. We also redesign the lightweight basic module using the shuffle module and the joint channel coordinate attention mechanism to replace the spatial weight calculation module in the original Lite-HRNet model. By introducing this new module, we not only improve the network calculation speed and reduce the number of parameters of the entire model, but also ensure the accuracy of the model, thereby achieving a balance between performance and efficiency. We compare this model with current mainstream methods on the COCO and MPII dataset. The experimental results show that this model can effectively reduce the number of parameters and computational complexity while ensuring high model accuracy.

Fault identification for power transformer based on dissolved gas in oil data using sparse convolutional neural networks

AbstractThis paper addressed the challenges associated with the complexity, numerous parameters, computational resource demands, and slow processing speed of transformer fault identification models based on deep learning technologies. Sparse convolutional neural network (CNN) approach is proposed for identifying faults related to dissolved gases in oil. Leveraging an improved Gramian angular field, one‐dimensional fault samples are converted into two‐dimensional feature images and data augmentation is implemented to meet the input requirements of deep learning models. Building upon visual geometry group (VGG)19 and residual networks (ResNet)50 networks for fault diagnosis, sparsity techniques are introduced through pruning, the fusion of convolution layers and batch normalization layers, and parameter quantization. Numerical experiments and performance evaluations on dissolved gas in transformer oil fault data demonstrate that the proposed method effectively reduced model complexity, minimized parameter count, conserved computational resources, and improved processing speed while maintaining a considerable level of fault identification accuracy. This made it applicable to edge computing platforms characterized by small form factors and low power consumption in the power industry.

CRESPR: Modular sparsification of DNNs to improve pruning performance and model interpretability

Modern DNNs often include a huge number of parameters that are expensive for both computation and memory. Pruning can significantly reduce model complexity and lessen resource demands, and less complex models can also be easier to explain and interpret. In this paper, we propose a novel pruning algorithm, Cluster-Restricted Extreme Sparsity Pruning of Redundancy (CRESPR), to prune a neural network into modular units and achieve better pruning efficiency. With the Hessian matrix, we provide an analytic explanation of why modular structures in a sparse DNN can better maintain performance, especially at an extreme high pruning ratio. In CRESPR, each modular unit contains mostly internal connections, which clearly shows how subgroups of input features are processed through a DNN and eventually contribute to classification decisions. Such process-level revealing of internal working mechanisms undoubtedly leads to better interpretability of a black-box DNN model. Extensive experiments were conducted with multiple DNN architectures and datasets, and CRESPR achieves higher pruning performance than current state-of-the-art methods at high and extremely high pruning ratios. Additionally, we show how CRESPR improves model interpretability through a concrete example.

Sugarcane stem node identification algorithm based on improved YOLOv5.

Identification of sugarcane stem nodes is generally dependent on high-performance recognition equipment in sugarcane seed pre-cutting machines and inefficient. Accordingly, this study proposes a novel lightweight architecture for the detection of sugarcane stem nodes based on the YOLOv5 framework, named G-YOLOv5s-SS. Firstly, the study removes the CBS and C3 structures at the end of the backbone network to fully utilize shallow-level feature information. This enhances the detection performance of sugarcane stem nodes. Simultaneously, it eliminates the 32 times down-sampled branches in the neck structure and the 20x20 detection heads at the prediction end, reducing model complexity. Secondly, a Ghost lightweight module is introduced to replace the conventional convolution module in the BottleNeck structure, further reducing the model's complexity. Finally, the study incorporates the SimAM attention mechanism to enhance the extraction of sugarcane stem node features without introducing additional parameters. This improvement aims to enhance recognition accuracy, compensating for any loss in precision due to lightweight modifications. The experimental results showed that the average precision of the improved network for sugarcane stem node identification reached 97.6%, which was 0.6% higher than that of the YOLOv5 baseline network. Meanwhile, a model size of 2.6MB, 1,129,340 parameters, and 7.2G FLOPs, representing respective reductions of 82%, 84%, and 54.4%. Compared with mainstream one-stage target detection algorithms such as YOLOv4-tiny, YOLOv4, YOLOv5n, YOLOv6n, YOLOv6s, YOLOv7-tiny, and YOLOv7, G-YOLOv5s-SS achieved respective average precision improvements of 12.9%, 5.07%, 3.6%, 2.1%, 1.2%, 3%, and 0.4% in sugarcane stem nodes recognition. Meanwhile, the model size was compressed by 88.9%, 98.9%, 33.3%, 72%, 92.9%, 78.8% and 96.3%, respectively. Compared with similar studies, G-YOLOv5s-SS not only enhanced recognition accuracy but also considered model size, demonstrating an overall excellent performance that aligns with the requirements of sugarcane seed pre-cutting machines.

DSPM: Dual sequence prediction model for efficient energy management in micro-grid

Power generation and consumption predictions are fundamental for smart grid operations, addressing challenges posed by renewable energy variability and irregular consumer demand. This study introduces the Dual Sequence Predictive Model (DSPM) based on Spatiotemporal CNN (STCNN) architecture, offering a holistic solution for forecasting Electricity Generation (EG) and Electricity Consumption (EC) collectively. Leveraging STCNN, the DSPM efficiently extracts spatial and temporal information, improving prediction accuracy and processing speed. A 1D spatial attention module is added to capture crucial spatial dependencies in historical data. Incorporating shared historical weather information streamlines learning process and reduces model complexity. Extensive benchmark evaluations demonstrate the DSPM superior performance, achieving the lowest error rates compared to baseline models. The DSPM also employs the Kullback–Leibler Divergence (KLD) algorithm for power generation and consumption matching, ensuring efficient power distribution within smart and microgrids. Furthermore, the DSPM is evaluated alongside Single Sequence Prediction (SSP) models, providing a comprehensive analysis of its capabilities and comparisons with state-of-the-art models. The SSP model achieved an average RMSE reduction of 17.3% for solar EG data and 15.86% for IHEPC residential EC data, while DSPM reduced RMSE by 7.57% over PowerGrid dataset compared to baseline models.

Group-Fusion One-Dimensional Convolutional Neural Network for Ballistic Target High-Resolution Range Profile Recognition with Layer-Wise Auxiliary Classifiers

Ballistic missile defense systems require accurate target recognition technology. Effective feature extraction is crucial for this purpose. The deep convolutional neural network (CNN) has proven to be an effective method for recognizing high-resolution range profiles (HRRPs) of ballistic targets. It excels in perceiving local features and extracting robust features. However, the standard CNN's fully connected manner results in high computational complexity, which is unsuitable for deployment in real-time missile defense systems with stringent performance requirements. To address the issue of computational complexity in HRRP recognition based on the standard one-dimensional CNN (1DCNN), we propose a lightweight network called group-fusion 1DCNN with layer-wise auxiliary classifiers (GFAC-1DCNN). GFAC-1DCNN employs group convolution (G-Conv) instead of standard convolution to effectively reduce model complexity. Simply using G-Conv, however, may decrease model recognition accuracy due to the lack of information flow between feature maps generated by each G-Conv. To overcome this limitation, we introduce a linear fusion layer to combine the output features of G-Convs, thereby improving recognition accuracy. Additionally, besides the main classifier at the deepest layer, we construct layer-wise auxiliary classifiers for different hierarchical features. The results from all classifiers are then fused for comprehensive target recognition. Extensive experiments demonstrate that GFAC-1DCNN with such simple and effective techniques achieves higher overall testing accuracy than state-of-the-art ballistic target HRRP recognition models, while significantly reducing model complexity. It also exhibits a higher recall rate for warhead recognition compared to other methods. Based on these compelling results, we believe this work is valuable in reducing workload and enhancing missile interception rates in missile defense systems.

Collaborative deep semi-supervised learning with knowledge distillation for surface defect classification

Defect inspection plays a vital role in ensuring high-quality production in industrial automation. While supervised approaches have been successful, they rely on costly labeled data. To address this limitation, semi-supervised methods have gained popularity, utilizing both labeled and unlabeled data for training. This research addresses the challenge of noisy semi-supervised training caused by incorrect pseudo-labels in Convolutional Neural Network based models. To enhance the accuracy and reliability of pseudo-label selection, a novel collaborative learning strategy with knowledge distillation for defect classification is proposed. The proposed approach involves training a set of teacher networks collaboratively, allowing them to collectively determine the pseudo-labels for each unlabeled image and improving the quality of pseudo-labeling. Subsequently, each teacher network is trained using these pseudo-labeled data. Finally, the acquired collaborative knowledge is transferred to a single student network, reducing model complexity, memory requirements, and enabling faster inference during deployment. The proposed approach demonstrates competitive performance on three publicly available defect classification datasets: NEU steel surfaces, SLS laser powder beds, and Surface Textures, achieving results comparable to the state-of-the-art. Notably, remarkable accuracy is achieved even with limited labeled data during training. For instance, on the SLS dataset, the proposed approach achieves 97% accuracy, which is comparable to the state-of-the-art’s 98% accuracy when using 100% of labeled data. Remarkably, the proposed approach accomplishes this level of accuracy using only 3% of the labeled training data, showcasing its efficiency and effectiveness in leveraging limited labeled data to achieve impressive results. Source code is available at https://github.com/.11Source code will be made available after the acceptance of this manuscript.

DelayNet: Enhancing Temporal Feature Extraction for Electronic Consumption Forecasting with Delayed Dilated Convolution

In the face of increasing irregular temperature patterns and climate shifts, the need for accurate power consumption prediction is becoming increasingly important to ensure a steady supply of electricity. Existing deep learning models have sought to improve prediction accuracy but commonly require greater computational demands. In this research, on the other hand, we introduce DelayNet, a lightweight deep learning model that maintains model efficiency while accommodating extended time sequences. Our DelayNet is designed based on the observation that electronic series data exhibit recurring irregular patterns over time. Furthermore, we present two substantial datasets of electricity consumption records from South Korean buildings spanning nearly two years. Empirical findings demonstrate the model’s performance, achieving 21.23%, 43.60%, 17.05% and 21.71% improvement compared to recurrent neural networks, gated-recurrent units, temporal convolutional neural networks and ARIMA models, as well as greatly reducing model complexity and computational requirements. These findings indicate the potential for micro-level power consumption planning, as lightweight models can be implemented on edge devices.