Increasing Model Complexity Research Articles

A brain tumor diagnosis approach based on deep separable convolutional networks and attention mechanisms

Brain tumors rank among the most lethal types of tumors. Magnetic Resonance Imaging (MRI) technology can clearly display the position, size, and borders of the tumors. Hence, MRI is frequently used in clinical diagnostics to detect brain tumors. Deep learning and related methods have been widely used in computer vision research recently for diagnosis and classification of MRI images of brain tumors. One trend is to achieve better performance by increasing model complexity. However, at the same time, the trainable parameters of the model increases accordingly. Too many parameters will lead to more difficult model training and optimization, and are prone to overfitting. To address this problem, this study constructs a model that incorporates a depthwise separable convolution technique and an attention mechanism to balance model performance and complexity. The model achieves 97.41% accuracy in the brain tumor classification task, which exceeds the 96.72% accuracy of the pre-trained model MobileNetV2, and shows good image classification ability. Future work could test the model's classification results on noisy images and investigate how to optimize the model's generalization ability.

A brain tumor diagnosis approach based on deep separable convolutional networks and attention mechanisms

Brain tumors rank among the most lethal types of tumors. Magnetic Resonance Imaging (MRI) technology can clearly display the position, size, and borders of the tumors. Hence, MRI is frequently used in clinical diagnostics to detect brain tumors. Deep learning and related methods have been widely used in computer vision research recently for diagnosis and classification of MRI images of brain tumors. One trend is to achieve better performance by increasing model complexity. However, at the same time, the trainable parameters of the model increases accordingly. Too many parameters will lead to more difficult model training and optimization, and are prone to overfitting. To address this problem, this study constructs a model that incorporates a depthwise separable convolution technique and an attention mechanism to balance model performance and complexity. The model achieves 97.41% accuracy in the brain tumor classification task, which exceeds the 96.72% accuracy of the pre-trained model MobileNetV2, and shows good image classification ability. Future work could test the model's classification results on noisy images and investigate how to optimize the model's generalization ability.

Computational comparison study of virtual compression and shear test for estimation of apparent elastic moduli under various boundary conditions.

Virtual compression tests based on finite element analysis are representative noninvasive methods to evaluate bone strength. However, owing to the characteristic porous structure of bones, the material obtained from micro-computed tomography images in the finite-element model is not uniformly distributed. These characteristics cause differences in the apparent elastic moduli depending on the boundary conditions and affect the accuracy of bone-strength evaluation. Therefore, this study aimed to evaluate and compare the apparent elastic moduli under various, virtual-compression and shear-test boundary conditions. Four, nonuniform models were constructed with increasing model complexity. For representative boundary conditions, two, different, testing directions, and constrained surfaces were applied. As a result, the apparent elastic moduli of the nonuniform model varied up to 55.2% based on where the constrained surface was located in the single-end-cemented condition. Additionally, when connectivity in the test direction was lost, the accuracy of the apparent elastic moduli was low. A graphical comparison showed that the equivalent-stress distribution was more advantageous for analyzing load transferability and physical behavior than the strain-energy distribution. These results clearly show that the prediction accuracy of the apparent elastic moduli can be guaranteed if the boundary condition on the constraint and loading surfaces of the nonuniform model are applied symmetrically and the connectivity of the elements in the testing direction is well maintained. This study will aid in precision improvement of bone-strength-indicator determination for osteoporosis prevention.

OPT-CO: Optimizing pre-trained transformer models for efficient COVID-19 classification with stochastic configuration networks

Building upon pre-trained ViT models, many advanced methods have achieved significant success in COVID-19 classification. Many scholars pursue better performance by increasing model complexity and parameters. While these methods can enhance performance, they also require extensive computational resources and extended training times. Additionally, the persistent challenge of overfitting, due to limited COVID-19 dataset sizes, remains a hurdle. To address these challenges, we proposed a novel method to optimize pre-trained transformer models for efficient COVID-19 classification with stochastic configuration networks (SCNs), referred to as OPT-CO. We proposed two optimization methods: sequential optimization (SeOp) and parallel optimization (PaOp), by incorporating optimizers in a sequential and parallel manner, respectively. Our method can enhance model performance without necessitating a significant parameter expansion. Additionally, we introduced OPT-CO-SCN to avoid overfitting problems through the adoption of random projection for head augmentation. The experiments were carried out to evaluate the performance of our proposed model based on two publicly available datasets. Based on the evaluation results, our method achieved superior, performance surpassing other state-of-the-art methods.

Residential building and sub-building level flood damage analysis using simple and complex models

AbstractFlood damage assessment is critical for optimal risk management investments. Damage models evaluate physical damage or monetary loss from direct building exposure to flood hazard processes. Traditional models represent a simple relationship whereby physical damage increases with water depth. More complex models offer an improved understanding of vulnerability, analysing interactions between multiple hazard and exposure variables that drive damage. Our study investigates whether increasing model complexity and explanatory damage variables improves prediction precision and reliability at residential building and sub-building (component) levels. We evaluate simple and complex empirical univariable and multivariable models for flood damage prediction. The Random Forest algorithm learned on multiple hazard and exposure explanatory variables outperformed linear and non-linear univariable regression approaches. Random Forest model predictive precision was highest when learning was limited to water depth and several important explanatory damage variables (flow velocity, area and floor height). Component damage models demonstrated high predictive precision for internal finishes and services. Precision reduced for structure and external finishes as damage samples for model learning were limited. High performing but complex multivariable models require further spatio-temporal transfer investigation to determine opportunities for accurate and reliable object-specific flood damage prediction in data scarce locations.

Improved understanding of calibration efficiency, difficulty and parameter uniqueness of conceptual rainfall runoff models using fitness landscape metrics

The ease and efficiency with which conceptual rainfall runoff (CRR) models can be calibrated, as well as issues related to the uniqueness of their parameters, has received significant attention in literature. While several studies have tried to gain a better understanding of the underlying factors affecting these issues by examining the features of model error surfaces, this has generally been done in an ad-hoc fashion using lower-dimensional representations of higher-dimensional surfaces. In this paper, it is suggested that exploratory landscape analysis (ELA) metrics can be used to quantify key features of the error surfaces of CRR models, including their roughness and flatness, as well as their degree of optima dispersion throughout the surface. This enables key error surface features of CRR models to be compared in a consistent, efficient and easily communicable fashion for models with different combinations of attributes (e.g. model structure, catchment climate conditions, error metrics, and calibration data set lengths). Results from the application of ELA metrics to the error surfaces of 420 CRR models with different combinations of the above attributes indicate that increasing model complexity results in an increase in relative error surface roughness and relative optima dispersion and that, while increasing catchment wetness increases the relative roughness of error surfaces, it also decreases optima dispersion. This suggests that for the models considered in this study, optimisation efficiency is likely to decrease with increasing model complexity and catchment wetness, while optimisation difficulty is likely to increase and parameter uniqueness likely to decrease with model complexity and catchment dryness. While implications for choice of model complexity will need further work, this study highlights the potential value of the proposed approach to understanding the calibration efficiency, difficulty and parameter uniqueness of conceptual rainfall runoff models.

Radiological image analysis using effective channel extension and fusion network based on COVID CT images

Attention mechanisms demonstrate significant capabilities in improving the performance of COVID CT image classification networks. Existing strategies often employ global pooling to acquire feature vectors, overlooking local feature information. Complex attention modules are then applied to enhance network performance, inevitably leading to increased model complexity. To address these issues, this paper introduces an Effective Channel Expansion and Fusion (ECEF) module specifically designed to enhance the performance of medical COVID CT image classification. The module represents a simple yet effective convolutional neural network module. In the channel direction of the given feature map, the module performs segmentation, conducts global pooling operations separately on the segmented and overall feature maps to obtain corresponding attention vectors, and merges them to produce high-dimensional attention vectors. Channel attention feature vectors are derived along the two pooling operation directions and dynamically fused. The resulting attention vectors are multiplied by the input feature map for adaptive feature refinement. Through these operations, the ECEF module adeptly captures local features in COVID CT images, enhancing the classification performance of medical COVID CT images. As a lightweight and versatile module, ECEF seamlessly integrates into any CNN architecture with negligible overhead. It can undergo end-to-end training alongside basic CNNs. ResNet is selected as the backbone network for empirical studies conducted on COVID CT images, including the COVID-CT dataset. Experimental results indicate that, compared to current advanced channel attention mechanisms, the ECEF module exhibits superior performance in COVID CT image classification tasks. The successful application of the ECEF module establishes a foundation for its widespread use in areas such as COVID CT medical image segmentation and object detection.

Advancements in Image Enhancement and Attention based EfficientDet Optimization Classifier for Precise Osteosarcoma Lung Nodule Detection

Introduction: Osteosarcoma is a malignant bone tumor that frequently spreads to the lungs, hence therapy effectiveness depends on early identification. However, noise and subtle characteristics still pose a challenge for reliable Lung Nodules Detection (LND) in medical pictures. In earlier work, SSD-VGG16 was implemented to provide a bounding box with an accuracy score that represented a single osteosarcoma nodule. Increasing model complexity is sometimes necessary to achieve improved accuracy with current approaches, which might worsen their computing inefficiencies.Methods: For accurate osteosarcoma lung nodule identification, this study offers the hybrid Dynamic Virtual Bats Algorithm with Attention based Efficient Object identification (A- EfficientDet). In order to improve the quality and informativeness of clinical pictures, this study suggests including Chebyshev filtering into the pre-processing pipeline. It focuses on CT scans for the purpose of detecting lung nodules associated with osteosarcoma. Additionally, provide the optimized A-EfficientDet model, a hybrid EfficientDet model improved using the DVBA optimization technique for accurate lung nodule identification. Results: The effectiveness of the suggested strategy in attaining accurate osteosarcoma LND is demonstrated by the experimental findings. Chebyshev filtering is incorporated during the pre-processing step, which leads to more accurate detection findings by improving the signal-to-noise ratio (SNR) and lung nodule visibility. Conclusion: Additionally, the improved EfficientDet model demonstrates its suitability for clinical applications in early osteosarcoma detection and treatment monitoring by achieving (SOTA) State-Of-The-Art execution by the metrics of sensitivity, specificity, and F1 score.

Developing a climatological simplification of aerosols to enter the cloud microphysics of a global climate model

Abstract. Aerosol particles influence cloud formation and properties. Hence climate models that aim for a physical representation of the climate system include aerosol modules. In order to represent more and more processes and aerosol species, their representation has grown increasingly detailed. However, depending on one's modelling purpose, the increased model complexity may not be beneficial, for example because it hinders understanding of model behaviour. Hence we develop a simplification in the form of a climatology of aerosol concentrations. In one approach, the climatology prescribes properties important for cloud droplet and ice crystal formation, the gateways for aerosols to enter the model cloud microphysics scheme. Another approach prescribes aerosol mass and number concentrations in general. Both climatologies are derived from full ECHAM-HAM simulations and can serve to replace the HAM aerosol module and thus drastically simplify the aerosol treatment. The first simplification reduces computational model time by roughly 65 %. However, the naive mean climatological treatment needs improvement to give results that are satisfyingly close to the full model. We find that mean cloud condensation nuclei (CCN) concentrations yield an underestimation of cloud droplet number concentration (CDNC) in the Southern Ocean, which we can reduce by allowing only CCN at cloud base (which have experienced hygroscopic growth in these conditions) to enter the climatology. This highlights the value of the simplification approach in pointing to unexpected model behaviour and providing a new perspective for its study and model development.

High-Order Extended Kalman Filter for State Estimation of Nonlinear Systems

In general, the extended Kalman filter (EKF) has a wide range of applications, aiming to minimize symmetric loss function (mean square error) and improve the accuracy and efficiency of state estimation. As the nonlinear model complexity increases, rounding errors gradually amplify, leading to performance degradation. After multiple iterations, divergence may occur. The traditional extended Kalman filter cannot accurately estimate the nonlinear model, and these errors still have an impact on the accuracy. To improve the filtering performance of the extended Kalman filter (EKF), this paper proposes a new extended Kalman filter (REKF) method that utilizes the statistical properties of the rounding error to enhance the estimation accuracy. After establishing the state model and measurement model, the residual term is used to replace the higher-order term in the Taylor expansion, and the least squares method is applied to identify the residual term step by step. Then, the iterative process of updating the extended Kalman filter is carried out. Within the Kalman filter framework, a higher-order rounding error-based extended Kalman filter (REKF) is designed for the joint estimation of rounding error and random variables, and the solution method for the rounding error is considered for the multilevel approximation of the original function. Through numerical simulations on a general nonlinear model, the higher-order rounding error-based extended Kalman filter (REKF) achieves better estimation results than the extended Kalman filter (EKF) and improves the filtering accuracy by utilizing the higher-order rounding error information, which also proves the effectiveness of the proposed method.

CosUKG: A Representation Learning Framework for Uncertain Knowledge Graphs

Knowledge graphs have been extensively studied and applied, but most of these studies assume that the relationship facts in the knowledge graph are correct and deterministic. However, in the objective world, there inevitably exist uncertain relationship facts. The existing research lacks effective representation of such uncertain information. In this regard, we propose a novel representation learning framework called CosUKG, which is specifically designed for uncertain knowledge graphs. This framework models uncertain information by measuring the cosine similarity between transformed vectors and actual target vectors, effectively integrating uncertainty into the embedding process of the knowledge graph while preserving its structural information. Through multiple experiments on three public datasets, the superiority of the CosUKG framework in representing uncertain knowledge graphs is demonstrated. It achieves improved representation accuracy of uncertain information without increasing model complexity or weakening structural information.

Assessing the disassembly performance of washing machines through the design for circular disassembly methodology

AbstractTo enable the circular economy paradigm, it is important to design easy-to-disassemble products. A new method, known as Design for Circular Disassembly (DfCD), has been proposed to enhance product disassembly performances toward circularity. The method was tested on a small-sized product, showing promising results. However, its applicability to medium-sized products remains unclear. The goal of this article is to assess the effectiveness of DfCD on medium-sized products, particularly washing machines. Results showed DfCD can be extended to medium-sized products, increasing model complexity.

Benthos as a key driver of morphological change in coastal regions

Abstract. Benthos has long been recognized as an important factor influencing local sediment stability, deposition, and erosion rates. However, its role in long-term (annual to decadal scale) and large-scale coastal morphological change remains largely speculative. This study aims to derive a quantitative understanding of the importance of benthos in the morphological development of a tidal embayment (Jade Bay) as representative of tidal coastal regions. To achieve this, we first applied a machine-learning-aided species abundance model to derive a complete map of benthos (functional groups, abundance, and biomass) in the study area, based on abundance and biomass measurements. The derived data were used to parameterize the benthos effect on sediment stability, erosion rates and deposition rates, erosion and hydrodynamics in a 3-dimensional hydro-eco-morphodynamic model, which was then applied to Jade Bay to hindcast the morphological and sediment change for 2000–2009. Simulation results indicate significantly improved performance with the benthos effect included. Simulations including benthos show consistency with measurements regarding morphological and sediment changes, while abiotic drivers (tides, storm surges) alone result in a reversed pattern in terms of erosion and deposition contrary to measurement. Based on comparisons among scenarios with various combinations of abiotic and biotic factors, we further investigated the level of complexity of the hydro-eco-morphodynamic models that is needed to capture long-term and large-scale coastal morphological development. The accuracy in the parameterization data was crucial for increasing model complexity. When the parameterization uncertainties were high, the increased model complexity decreased the model performance.

Root and shoot phenology, architecture, and organ properties: an integrated trait network among 44 herbaceous wetland species.

Integrating traits across above- and belowground organs offers comprehensive insights into plant ecology, but their various functions also increase model complexity. This study aimed to illuminate the interspecific pattern of whole-plant trait correlations through a network lens, including a detailed analysis of the root system. Using a network algorithm that allows individual traits to belong to multiple modules, we characterize interrelations among 19 traits, spanning both shoot and root phenology, architecture, morphology, and tissue properties of 44 species, mostly herbaceous monocots from Northern Ontario wetlands, grown in a common garden. The resulting trait network shows three distinct yet partially overlapping modules. Two major trait modules indicate constraints of plant size and form, and resource economics, respectively. These modules highlight the interdependence between shoot size, root architecture and porosity, and a shoot-root coordination in phenology and dry-matter content. A third module depicts leaf biomechanical adaptations specific to wetland graminoids. All three modules overlap on shoot height, suggesting multifaceted constraints of plant stature. In the network, individual-level traits showed significantly higher centrality than tissue-level traits do, demonstrating a hierarchical trait integration. The presented whole-plant, integrated network suggests that trait covariation is essentially function-driven rather than organ-specific.

Toward Atmospheric Retrievals of Panchromatic Light Curves: ExPLOR-ing Generalized Inversion Techniques for Transiting Exoplanets with JWST and Ariel

Conventional atmospheric retrieval codes are designed to extract information, such as chemical abundances, thermal structures, and cloud properties, from fully “reduced” spectra obtained during transit or eclipse. Reduced spectra, however, are assembled by fitting a series of simplified light curves to time-series observations, wavelength by wavelength. Thus, spectra are postprocessed summary statistics of the original data, which by definition do not encode all the available information (i.e., astrophysical signal, model covariance, and instrumental noise). Here, we explore an alternative inversion strategy where the atmospheric retrieval is performed on the light curve directly, i.e., closer to the data. This method is implemented in EXoplanet Panchromatic Light curve Observation and Retrieval (ExPLOR), a novel atmospheric retrieval code inheriting from the TauREx project. By explicitly considering time in the model, ExPLOR naturally handles transits, eclipses, phase curves, and other complex geometries for transiting exoplanets. In this paper, we have validated this new technique by inverting simulated panchromatic light curves. The model was tested on realistic simulations of a WASP-43 b-like exoplanet as observed with the James Webb Space Telescope (JWST) and Ariel telescope. By comparing our panchromatic light-curve approach against conventional spectral retrievals on mock scenarios, we have identified key breaking points in information and noise propagation when employing past literature techniques. Throughout the paper, we discuss the importance of developing “closer-to-data” approaches such as the method presented in this work, and highlight the inevitable increase in model complexity and computing requirements associated with the recent JWST revolution.

An Efficient Image Compression Technique using Long Short-Term Memory Networks (LSTM)

The emergence of big data has imposed significant challenges on data storage and transmission. One pressing issue is leveraging deep learning techniques to achieve superior compression ratios and enhance image quality. Recurrent Neural Networks (RNNs) offer a promising avenue for controlling image bit rates iteratively, thereby enhancing compression performance. However, integrating Long Short-Term Memory (LSTM) into RNNs to address long-term dependencies increases model complexity. To expedite training and enhance image reconstruction quality, this study proposes several innovations.Initially, we enhance the activation function within LSTM to more effectively manage information retention and omission, thereby reducing parameter count and expediting training. Additionally, we introduce an image recovery block within the decoder to reconstruct high-resolution images. Finally, to expedite loss convergence, we replace L1 loss with SmoothL1 loss. Experimental outcomes demonstrate the efficacy of our approach, showcasing higher compression ratios

Token-Event-Role Structure-Based Multi-Channel Document-Level Event Extraction

Document-level event extraction is a long-standing challenging information retrieval problem involving a sequence of sub-tasks: entity extraction, event type judgment, and event type-specific multi-event extraction. However, addressing the problem as multiple learning tasks leads to increased model complexity. Also, existing methods insufficiently utilize the correlation of entities crossing different events, resulting in limited event extraction performance. This article introduces a novel framework for document-level event extraction, incorporating a new data structure called token-event-role and a multi-channel argument role prediction module. The proposed data structure enables our model to uncover the primary role of tokens in multiple events, facilitating a more comprehensive understanding of event relationships. By leveraging the multi-channel prediction module, we transform entity and multi-event extraction into a single task of predicting token–event pairs, thereby reducing the overall parameter size and enhancing model efficiency. The results demonstrate that our approach outperforms the state-of-the-art method by 9.5 percentage points in terms of the F 1 score, highlighting its superior performance in event extraction. Furthermore, an ablation study confirms the significant value of the proposed data structure in improving event extraction tasks, further validating its importance in enhancing the overall performance of the framework.

Bi-SGTAR: A simple yet efficient model for circRNA-disease association prediction based on known association pair only

Identifying circRNA (circular RNA) associated with diseases holds promise as diagnostic and prognostic biomarkers, offering potential avenues for novel therapeutics. Several computational methods have been designed to predict circRNA-disease associations. Unfortunately, current computational models face issues stemming from the integration of data from multiple sources, leading to blind spots in data combination and increased model complexity. Thus, this article introduces a novel method named Bi-SGTAR (Bi-view Sparse Gating and True Association Regression). Notably, Bi-SGTAR demonstrates comparable performance to existing multi-source information fusion methods while utilizing only known circRNA-disease association pairs. In contrast to previous methods, the model divides the adjacency matrix into two views and employs an encoder with sparse gating to assess the reliability of all associations. Additionally, a supervised reconstructor is employed to define the true association probability, quantifying the truthfulness of all associations. The Encoding-Reconstruction-Regression (ERR) framework adeptly merges both reliable and truthful associations from both views. The experimental results unequivocally show that Bi-SGTAR surpasses state-of-the-art models across seven circRNA-disease datasets, one lncRNA-disease dataset, and one microbe-drug dataset, with fewer data needed.

Power System Frequency Dynamics Modeling, State Estimation, and Control using Neural Ordinary Differential Equations (NODEs) and Soft Actor-Critic (SAC) Machine Learning Approaches

With the global energy transition of the electric power system, grid control, supervision, and protection is becoming more challenging. With the increasing integration of renewable energy sources (RES), the system dynamics are changing, causing traditional power system dynamic modeling with swing equation-based modeling approaches to fail. Additionally, the converter-dominated power grid is decreasing the system inertia, making the power system more fragile to the frequency swings. This paper first investigates and compares the application of a model-based Kalman filter state estimation approach with (i) a model-free machine learning approach --- neural ordinary differential equations (NODEs) --- and (ii) a data-driven system identification (SysId) approach to model and infer critical state values of the power system frequency dynamics. Then a model predictive control (MPC) framework is compared to a model-free Soft Actor-Critic (SAC) reinforcement learning (RL) control algorithm in providing efficient fast frequency response (FFR) to the power system frequency dynamics. The approaches are compared in terms of their performance goals as well as their per-timestep computational efficiency. The comparative study for state estimation shows that for the model-free requirement, both NODEs and SysId can provide accurate state estimates; however, with increasing model complexity, NODEs can be a better choice for model identification. Similarly, the results from the FFR comparative study show that the SAC RL-based FFR, once trained, outperforms MPC with better control signals and faster computation time, making the SAC RL-based FFR better option for providing FFR to the power system.

Anomaly detection in time-series data using evolutionary neural architecture search with non-differentiable functions

Deep neural networks have become the benchmark in diverse fields such as energy consumption forecasting, speech recognition, and anomaly detection, owing to their ability to efficiently process and analyse data. However, they face challenges in managing the complexity and variability in time series data, often leading to increased model complexity and prolonged search duration during parameter tuning. This paper proposes a novel anomaly detection approach through evolutionary neural architecture search (AD-ENAS), which is specifically designed for time series data. The proposed approach focuses on the search for the optimal and minimal neural network architecture. The AD-ENAS method consists of two main phases: architecture evolution and weight adjustment. The architecture evolution phase highlights the importance of neural network architecture by evaluating the fitness of each network agent using shared weight values. Subsequently, the convolutional matrix adaptation technique is used in the next phase for optimal weight adjustment of the neural network. The proposed AD-ENAS method operates without relying on differentiable functions, thus expanding the scope of neural network design beyond traditional backpropagation-based approaches. Various non-differentiable loss functions are explored to facilitate effective architecture search and weight adjustment. Comparative experiments are conducted with five baseline anomaly detection methods on three well-known datasets from reputable sources such as NASA SMAP, NASA MSL and Yahoo S5-A1. The results demonstrate that the AD-ENAS approach effectively evolves neural network architectures, outperforming baseline methods with F1 scores across the three datasets (MSL: 0.942, SMAP: 0.961, Yahoo S5-A1: 0.988) with non-differentiable loss functions, showcasing its efficacy in detecting anomalies in time series data.