Compact Model Research Articles

Smart Distillation for Land Use Monitoring: A Lightweight Remote Sensing Segmentation Strategy

As the demand for land use monitoring continues to grow, high-precision remote sensing products have become increasingly important. Compared to traditional methods, deep learning networks demonstrate significant advantages in automatic feature extraction, handling complex scenes, and improving classification accuracy. However, as the complexity of these networks increases, so does the computational cost. To address this challenge, we propose an innovative knowledge distillation model, integrating two key modules—spatial-global attention feature distillation (SGAFD) and channel attention-based relational distillation (CARD). This model enables a lightweight “student” network to be guided by a large “teacher” network, enhancing classification performance while maintaining a compact model size. We validated our approach on the large-scale public remote sensing datasets GID15 and LoveDA, and the results show that these modules effectively improve classification performance, overcoming the limitations of lightweight models and advancing the practical applications of land use monitoring.

Machine Learning for Sparse Nonlinear Modeling and Control

Machine learning is rapidly advancing nearly every field of science and engineering, and control theory is no exception. In particular, it has shown incredible promise for handling several of the main challenges facing modern dynamics and control, including complexity, unmodeled dynamics, strong nonlinearity, and hidden variables. However, machine learning models are often expensive to train and deploy, fail to generalize beyond the training data, and suffer from a lack of explainability, interpretability, and guarantees, all of which limit their use in real-world and safety-critical control applications. Sparse nonlinear modeling and control techniques are a powerful class of machine learning that promote parsimony through sparse optimization, providing data-efficient models that are more interpretable and generalizable and have proven effective for control. In this review, we explore the use of sparse optimization in the context of machine learning to develop compact models and controllers that are easy to train, require significantly less data, and make low-latency predictions. In particular, we focus on applications in model predictive control and reinforcement learning, two of the foundational algorithms in control theory.

Efficient and Effective Role Player: A Compact Knowledge-grounded Persona-based Dialogue Model Enhanced by LLM Distillation

Efficient and Effective Role Player: A Compact Knowledge-grounded Persona-based Dialogue Model Enhanced by LLM Distillation

Hybrid quantum neural networks: harnessing dressed quantum circuits for enhanced tsunami prediction via earthquake data fusion

Tsunami is one of the deadliest natural disasters which can occur, leading to great loss of life and property. This study focuses on predicting tsunamis, using earthquake dataset from the year 1995 to 2023. The research introduces the Hybrid Quantum Neural Network (HQNN), an innovative model that combines Neural Network (NN) architecture with Parameterized Quantum Circuits (PmQC) to tackle complex machine learning (ML) problems where deep learning (DL) models struggle, aiming for higher accuracy in prediction while maintaining a compact model size. The hybrid model’s performance is compared with the classical model counterpart to investigate the quantum circuit’s effectivity as a layer in a DL model. The model has been implemented using 2-6 features through Principle Component Analysis (PCA) method. HQNN’s quantum circuit is a combination of Pennylane’s embedding (Angle Embedding (AE) and Instantaneous Quantum Polynomial (IQP) Embedding) and layer circuits (Basic Entangler Layers (BEL), Random Layers (RL), and Strongly Entangling Layers (SEL)), along with the classical layers. Results show that the proposed model achieved high performance, with a maximum accuracy up to 96.03% using 4 features with the combination of AE and SEL, superior to the DL model. Future research could explore the scalability and diverse applications of HQNN, as well as its potential to address practical ML challenges.

Ship target detection method based on improved YOLOv8 for SAR images

ABSTRACT Due to the complex background and less effective information in low-resolution and noisy images, the detection of small ships in SAR images suffers from low detection rate and high false alarm rate. In order to solve the above problems, this paper proposes a method to detect the small ship targets in SAR images based on improved YOLOv8. Firstly, the deformable convolutional networks are added to the leading network to improve feature extraction. Then, the efficient multi-scale attention module is fused with the backbone network to enhance the detection effect of small targets. The weighted intersection over union loss function is used to optimize the regression process of the prediction frame and the detection frame to enhance the localization capacity of small targets. Finally, there is an addition of a specialized small target detection layer, and a reconstruction of the feature extraction and fusion network to enhance the detection performance of small ship targets in SAR images. To verify the effectiveness and robustness of the method, we conduct experiments on SSDD and HRSID. The proposed method achieves high detection accuracy while also offering a more compact model size and less computation time compared to other prevalent methods.

Big data-driven global modeling of cohesive soil compaction across conceptual and arbitrary energies through machine learning

Big data-driven global modeling of cohesive soil compaction across conceptual and arbitrary energies through machine learning

A data-based compact pipe model for district-heating-and-cooling networks with variable flow conditions

A data-based compact pipe model for district-heating-and-cooling networks with variable flow conditions

Compact MILP Models for Double Row Layout Problem with Pairwise Clearance

Compact MILP Models for Double Row Layout Problem with Pairwise Clearance

A high performance heterogeneous hardware architecture for brain computer interface.

Brain-computer interface (BCI) has been widely used in human-computer interaction. The introduction of artificial intelligence has further improved the performance of BCI system. In recent years, the development of BCI has gradually shifted from personal computers to embedded devices, which boasts lower power consumption and smaller size, but at the cost of limited device resources and computing speed, thus can hardly improve the support of complex algorithms. This paper proposes a heterogeneous BCI architecture based on ARM + FPGA, enabling real-time processing of electroencephalogram (EEG) signals. Adopting data quantization, layer fusion and data augmentation to optimize the compact neural network model EEGNet, and design dedicated hardware engines to accelerate the network. Experimental results show that the system achieves 93.3% classification accuracy for steady-state visual evoked potential signals, with a time delay of 0.2ms per trail, and a power consumption of approximately (1.91W). That is 31.5 times faster acceleration is realized at the cost of only 0.7% lower accuracy compared with the conventional processor. The results show that the BCI architecture proposed in this study has strong practicability and high research significance.

Inspect quantitative signals in placental histopathology: Computer-assisted multiple functional tissues identification through multi-model fusion and distillation framework.

Pathological analysis of placenta is currently a valuable tool for gaining insights into pregnancy outcomes. In placental histopathology, multiple functional tissues can be inspected as potential signals reflecting the transfer functionality between fetal and maternal circulations. However, the identification of multiple functional tissues is challenging due to (1) severe heterogeneity in texture, size and shape, (2) distribution across different scales and (3) the need for comprehensive assessment at the whole slide image (WSI) level. To solve aforementioned problems, we establish a brand new dataset and propose a computer-aided segmentation framework through multi-model fusion and distillation to identify multiple functional tissues in placental histopathologic images, including villi, capillaries, fibrin deposits and trophoblast aggregations. Specifically, we propose a two-stage Multi-model Fusion and Distillation (MMFD) framework. Considering the multi-scale distribution and heterogeneity of multiple functional tissues, we enhance the visual representation in the first stage by fusing feature from multiple models to boost the effectiveness of the network. However, the multi-model fusion stage contributes to extra parameters and a significant computational burden, which is impractical for recognizing gigapixels of WSIs within clinical practice. In the second stage, we propose straightforward plug-in feature distillation method that transfers knowledge from the large fused model to a compact student model. In self-collected placental dataset, our proposed MMFD framework demonstrates an improvement of 4.3% in mean Intersection over Union (mIoU) while achieving an approximate 50% increase in inference speed and utilizing only 10% of parameters and computational resources, compared to the parameter-efficient fine-tuned Segment Anything Model (SAM) baseline. Visualization of segmentation results across entire WSIs on unseen cases demonstrates the generalizability of our proposed MMFD framework. Besides, experimental results on a public dataset further prove the effectiveness of MMFD framework on other tasks. Our work can present a fundamental method to expedite quantitative analysis of placental histopathology.

Compact MILP models for double row layout problem with pairwise clearance

Compact MILP models for double row layout problem with pairwise clearance

Impact Ionization in LDMOS Transistors: Improved Compact Model and Asymmetry under Forward and Reverse Modes of Operation

Impact Ionization in LDMOS Transistors: Improved Compact Model and Asymmetry under Forward and Reverse Modes of Operation

A compact model for the home healthcare routing and scheduling problem

A compact model for the home healthcare routing and scheduling problem

Identifying and understanding the nonlinear behavior of memristive devices

AbstractNonlinearity is a crucial characteristic for implementing hardware security primitives or neuromorphic computing systems. The main feature of all memristive devices is this nonlinear behavior observed in their current-voltage characteristics. To comprehend the nonlinear behavior, we have to understand the coexistence of resistive, capacitive, and inertia (virtual inductive) effects in these devices. These effects originate from corresponding physical and chemical processes in memristive devices. A physics-inspired compact model is employed to model and simulate interface-type RRAMs such as Au/BiFeO$$_{3}$$/Pt/Ti, Au/Nb$$_{\textrm{x}}$$O$$_{\textrm{y}}$$/Al$$_{2}$$O$$_{3}$$/Nb, while accounting for the modeling of capacitive and inertia effects. The simulated current-voltage characteristics align well with experimental data and accurately capture the non-zero crossing hysteresis generated by capacitive and inductive effects. This study examines the response of two devices to increasing frequencies, revealing a shift in their nonlinear behavior characterized by a reduced hysteresis range Fourier series analysis utilizing a sinusoidal input voltage of varying amplitudes and frequencies indicates harmonics or frequency components that considerably influence the functioning of RRAMs. Moreover, we propose and demonstrate the use of the frequency spectra as one of the fingerprints for memristive devices.

Improving Generalization of Genetic Programming for High-Dimensional Symbolic Regression with Shapley Value Based Feature Selection

AbstractSymbolic Regression (SR) on high-dimensional datasets often encounters significant challenges, resulting in models with poor generalization capabilities. While feature selection has the potential to enhance the generalization and learning performance in general, its application in Genetic Programming (GP) for high-dimensional SR remains a complex problem. Originating from game theory, the Shapley value is applied to additive feature attribution approaches where it distributes the difference between a model output and a baseline average across input variables. By providing an accurate assessment of each feature importance, the Shapley value offers a robust approach to select features. In this paper, we propose a novel feature selection method leveraging the Shapley value to identify and select important features in GP for high-dimensional SR. Through a series of experiments conducted on ten high-dimensional regression datasets, the results indicate that our algorithm surpasses standard GP and other GP-based feature selection methods in terms of learning and generalization performance on most datasets. Further analysis reveals that our algorithm generates more compact models, focusing on the inclusion of important features.

Compact anisotropic stellar model in Tolman–Kuchowicz spacetime in extended theory of gravity

Compact anisotropic stellar model in Tolman–Kuchowicz spacetime in extended theory of gravity

Cryo-SIMPLY: A Reliable STT-MRAM-Based Smart Material Implication Architecture for In-Memory Computing.

This paper presents Cryo-SIMPLY, a reliable smart material implication (SIMPLY) operating at cryogenic conditions (77 K). The assessment considers SIMPLY schemes based on spin-transfer torque magnetic random access memory (STT-MRAM) technology with single-barrier magnetic tunnel junction (SMTJ) and double-barrier magnetic tunnel junction (DMTJ). Our study relies on a temperature-aware macrospin-based Verilog-A compact model for MTJ devices and a 65 nm commercial process design kit (PDK) calibrated down to 77 K under silicon measurements. The DMTJ-based SIMPLY demonstrates a significant improvement in read margin at 77 K, overcoming the conventional SIMPLY scheme at room temperature (300 K) by approximately 2.3 X. When implementing logic operations with the SIMPLY scheme operating at 77 K, the DMTJ-based scheme assures energy savings of about 69%, as compared to its SMTJ-based counterpart operating at 77 K. Overall, our results prove that the SIMPLY scheme at cryogenic conditions is a promising solution for reliable and energy-efficient logic-in-memory (LIM) architectures.

Dark energy stars with cosmological constant as a black-hole mimickers

In this study, we investigated an interior solution of a static spherically symmetric dark energy star model incorporating anisotropic fluid and a state parameter of the type cosmological constant, achieved through parametrization of the Finch and Skea metric function. The motivation for this study stems from the lack of investigation of dark energy star models in relation to BHs in the previous studies, and from recent proposals suggesting that dark energy might originate inside BHs. This opens up the opportunity to explore the dark energy star model and establish its relationship with BHs. The structural profiles of the models, including mass function, energy density, compactness, surface redshift, and local acceleration due to gravity, were investigated. Analysis involving the matching of the exterior Schwarzschild vacuum solution to the interior spacetime at a junction interface was also explored. The results demonstrate that this solution is free of singularities, possessing outward gravitational repulsion with an infinite property near the surface boundary. Notably, the model exhibited an infinite redshift surface, and a compactness of one half, thus evading the Buchdahl limit. It fulfills the energy conditions except for the strong energy conditions and remains in a state of static equilibrium, upheld by both hydro-static and anisotropic forces. Numerical values of physical properties for various types of astrophysical BH candidates have been determined. Overall, the obtained model is physically unique and represents the most compact and extreme model of dark energy stars. Some features of this model resemble those of BHs, rendering it indistinguishable from BHs.

Towards Real-Time Detection of Wakes for Various Sea States with Lightweight Deep Learning Model in Synthetic Aperture Radar Images

Synthetic aperture radar (SAR) is an essential tool for monitoring and managing maritime traffic and ensuring safety. It is particularly valuable because it can provide surveillance in all weather conditions. Ship wake detection has attracted considerable attention in offshore management as it has potential for widespread use in ship positioning and motion parameter inversion, surpassing conventional ship detection methods. Traditional wake detection methods depend on linear feature extraction through image transformation processing techniques, which are often ineffective and time-consuming when applied to large-scale SAR data. Conversely, deep learning (DL) algorithms have been infrequently utilized in wake detection and encounter significant challenges due to the complex ocean background and the effect of the sea state. In this study, we propose a lightweight rotating target detection network designed for detecting ship wakes under various sea states. For this purpose, we initially analyzed the features of wake samples across various frequency domains. In the framework, a YOLO structure-based deep learning is implemented to achieve wake detection. Our network design enhances the YOLOv8’s structure by incorporating advanced techniques such as deep separation convolution and combined frequency domain–spatial feature extraction modules. These modules are used to replace the usual convolutional layer. Furthermore, it integrates an attention technique to extract diverse features. By conducting experiments on the OpenSARWake dataset, our network exhibited outstanding performance, achieving a wake detection accuracy of 66.3% while maintaining a compact model size of 51.5 MB and time of 14 ms. This model size is notably less than the existing techniques employed for rotating target detection and wake detection. Additionally, the algorithm exhibits excellent generalization ability across different sea states, addressing to a certain extent the challenge of wake detection being easily influenced by varying sea states.

Hybrid Edge–Cloud Models for Bearing Failure Detection in a Fleet of Machines

Real-time condition monitoring of machinery is increasingly being adopted to minimize costs and enhance operational efficiency. By leveraging large-scale data acquisition and intelligent algorithms, failures can be detected and predicted, thereby reducing machine downtime. In this paper, we present a novel hybrid edge–cloud system for detecting rotational bearing failures using accelerometer data. We evaluate both supervised and unsupervised neural network approaches, highlighting their respective strengths and limitations. Supervised models demonstrate high accuracy but require labeled datasets representative of the failures of interesting data that are challenging to acquire due to the rarity of anomalies. Conversely, unsupervised models rely on data from normal operational conditions, which is more readily available. However, these models classify all deviations from normalcy as anomalies, including those unrelated to failure, leading to costly false positives. To address these challenges, we propose a distributed system that integrates supervised and unsupervised learning. A compact unsupervised model is deployed on edge devices near the machines to compress sensor data, which are then transmitted to a centralized cloud-based system. Over time, these data are automatically labeled and used to train a supervised model, improving the accuracy of failure predictions. Our approach enables efficient, scalable failure detection across a fleet of machines while balancing the trade-offs between supervised and unsupervised learning.