Minimization Algorithm Research Articles

Regularizing Scale-Adaptive Central Moment Sharpness for Neural Networks.

In deep learning, finding flat minima of loss function is a hot research topic in improving generalization. The existing methods usually find flat minima by sharpness minimization algorithms. However, these methods suffer from insufficient flexibility for optimization and generalization due to their ignorance of loss value. This article theoretically and experimentally explores the sharpness minimization algorithms for neural networks. First, a novel scale-invariant sharpness which is called scale-adaptive central moment sharpness (SA-CMS) is proposed. This sharpness is not only scale-invariant but can characterize the nature of loss surface clearly. Based on the proposed sharpness, this article further derives a new regularization term by integrating the different orders of the sharpness. Particularly, a host of sharpness minimization functions such as local entropy can be covered by this regularization term. Then the central moment sharpness generating function is introduced as a new objective function. Moreover, theoretical analyses indicate that the new objective function has a smoother landscape and prefer converging to flat local minima. Furthermore, a computationally efficient two-stage algorithm is developed to minimize the objective function. Compared with other algorithms, the two-stage loss-sharpness minimization (TSLSM) algorithm offers a more flexible optimization target for different training stages. On a variety of learning tasks with both small and large batch sizes, this algorithm is more universal and effective, and meanwhile achieves or surpasses the generalization performance of the state-of-the-art sharpness minimization algorithms.

An efficient test suit reduction methodology for regression testing

<span lang="EN-US">This paper's goal is to provide a more effective algorithm for reducing the amount of test cases in a test suit during the regression testing phase. This algorithm divide the entire test suit into equivalence classes in first step and then apply boundary value coverage to select test case out of repeated test cases which has same importance in test suit. This algorithm is based on the concept that before selecting best test cases out of repeated test case in test suit to prepare reduced test suit we can divide all test cases in number of equivalence classes so number of test case under consideration reduced by great extend. This paper proposed a method of experimentation involving test cases from different software application areas; minimization algorithms and the maths and algorithms of minimization algorithms in details. Test case techniques are equivalence portioning and boundary value analysis. Along with this concept, I also discuss a case study to verify and check new algorithm for its efficiency, for that I apply my algorithm on one of the program or group of program. This complete proposed methodology shall be applied to different software applications belonging to soft computing, Engineering software, Financial Software, Cloud Applications, Business Applications, AI and Machine Learning, Data Analytics are being identified. This selection has been made keeping in mind the trends, industrial importance, economic values and research challenges. Minimization in test cases would lead to lesser testing effort and desirable test completion.</span>

Q-learning guided algorithms for bi-criteria minimization of total flow time and makespan in no-wait permutation flowshops

Combining Deep Reinforcement Learning and meta-heuristic techniques represents a new research direction for enhancing the search capabilities of meta-heuristic methods in the context of production scheduling. Q-learning is a prominent reinforcement learning in which its utilization aims to direct the selection of actions, thus preventing the necessity for a random exploration in the iterative process of the metaheuristics. In this study, we provide Q-learning guided algorithms for the Bi-Criteria No-Wait Flowshop Scheduling Problem (NWFSP). The problem is treated as a bi-criteria combinatorial optimization problem where total flow time and makespan are optimized simultaneously. Firstly, a deterministic mixed-integer linear programming (MILP) model is provided. Then, Q-learning guided algorithms are developed: Bi-Criteria Iterated Greedy Algorithm with Q-Learning (BC-IGQL). Bi-Criteria Block Insertion Heuristic Algorithm with Q-Learning (BC-BIHQL). Moreover, the performance of the proposed Q-learning guided algorithms is compared over a collection of Bi-Criteria Genetic Local Search Algorithms (BC-GLS), Bi-Criteria Iterated Greedy Algorithm (BC-IG), Bi-Criteria Iterated Greedy Algorithm with a Local Search (BC-IGALL) and Bi-Criteria Variable Block Insertion Heuristic Algorithm (BC-VBIH). The complete computational experiment, performed on 480 problem instances of Vallada et al. (2015), which is known as the VRF benchmark set, indicates that the BC-BIHQL and the BC-IGQL algorithms outperform the BC-GLS, BC-IG, BC-IGALL, and BC-VBIH algorithms in comparative performance metrics. More specifically, the proposed BC-BIHQL and BC-IGQL algorithms can yield more non-dominated bi-criteria solutions with the most substantial competitiveness than the remaining algorithms. At the same time, both are competitive with each other on the benchmark problems. Moreover, the BC-IGQL algorithm dominates almost 97% and 99% of the solutions reached by the BC-IG, BC-IGALL, and BC-VBIH algorithms in small and large datasets. Similarly, The BC-BIHQL algorithm dominates almost 98% and 99% of the solutions reached by the BC-IG, BC-IGALL, and BC-VBIH algorithms in small and large datasets, respectively. This means that, among all the features that have been compared, the Q-learning-guided algorithms demonstrate the highest level of competitiveness. The outcomes of this study encourage us to discover many more bi-criteria NWFSPs to reveal the trade-off between other conflicting objectives, such as makespan & the number of early jobs, to overcome various industries' problems.

3081: Algorithm for PTV minimisation in linear accelerator-based liver Stereotactic Ablative Radiotherapy

3081: Algorithm for PTV minimisation in linear accelerator-based liver Stereotactic Ablative Radiotherapy

Comparison of Crow Search and Practice Swarm Algorithm for Minimization of Losses in Unbalanced Radial Distribution System

Comparison of Crow Search and Practice Swarm Algorithm for Minimization of Losses in Unbalanced Radial Distribution System

Feature selection by Universum embedding

Feature selection in classification is an important task in machine learning. Inspired by the success of Universum support vector machine proposed by Weston et al. on improving the classification ability of classical support vector machine, this paper considers a special type of Universum and further lets it play its role in both useful feature identification and separating hyperplane construction, aiming to improve both the feature selection ability and classification performance of Universum support vector machine. By introducing this special Universum, a redundant feature can be identified by observing whether some Universum sample is useful. In fact, we prove that by observing the dual solution of the optimization problem, useful features can be selected from a set satisfying some properties. Due to the introduction of these extra Universum samples, it needs to cope with a large-scale optimization problem. To improve the training efficiency, we modify the sequential minimal optimization algorithm and further combine it with the coordinate descent technique to solve the proposed model. Experimental results on artificial datasets, benchmark datasets, and text classification datasets demonstrate that the proposed method improves the classification performance of support vector machine and Universum support vector machine, and also has good feature selection ability.

Decentralized stochastic sharpness-aware minimization algorithm

In recent years, distributed stochastic algorithms have become increasingly useful in the field of machine learning. However, similar to traditional stochastic algorithms, they face a challenge where achieving high fitness on the training set does not necessarily result in good performance on the test set. To address this issue, we propose to use of a distributed network topology to improve the generalization ability of the algorithms. We specifically focus on the Sharpness-Aware Minimization (SAM) algorithm, which relies on perturbation weights to find the maximum point with better generalization ability. In this paper, we present the decentralized stochastic sharpness-aware minimization (D-SSAM) algorithm, which incorporates the distributed network topology. We also provide sublinear convergence results for non-convex targets, which is comparable to consequence of Decentralized Stochastic Gradient Descent (DSGD). Finally, we empirically demonstrate the effectiveness of these results in deep networks and discuss their relationship to the generalization behavior of SAM.

Boosting sharpness-aware training with dynamic neighborhood

Learning algorithms motivated by minimizing the sharpness of loss surface is a hot research topic in improving generalization. The existing methods usually solve a constrained min–max problem to minimize sharpness and find flat minima. However, most constraints (i.e., the neighborhood of the sharpness) are inappropriate, leading to sub-optimal results. This paper theoretically explores the optimal neighborhood from the view of Probably Approximately Correct-Bayesian (PAC-Bayesian) framework. A closed form of the optimal neighborhood is provided. This neighborhood is determined by the Hessian matrix and the scales of parameters. Then a generalization bound is derived that serves as a guiding principle in the design of the sharpness minimization algorithm. The Dynamic neighborhood-based Sharpness-Aware Minimization algorithm is proposed, which can adaptively adjust the neighborhood during the training process to gain better performance. Also, the algorithm is proved can convergent at the rate O(logT/T). Experimental results demonstrate that the proposed algorithm outperforms the other methods (e.g., accuracy ＋2.86% over baseline on CIFAR-100 for VGG-16).

Image restoration for ring-array photoacoustic tomography system based on blind spatially rotational deconvolution

Ring-array photoacoustic tomography (PAT) system has been widely used in noninvasive biomedical imaging. However, the reconstructed image usually suffers from spatially rotational blur and streak artifacts due to the non-ideal imaging conditions. To improve the reconstructed image towards higher quality, we propose a concept of spatially rotational convolution to formulate the image blur process, then we build a regularized restoration problem model accordingly and design an alternating minimization algorithm which is called blind spatially rotational deconvolution to achieve the restored image. Besides, we also present an image preprocessing method based on the proposed algorithm to remove the streak artifacts. We take experiments on phantoms and in vivo biological tissues for evaluation, the results show that our approach can significantly enhance the resolution of the image obtained from ring-array PAT system and remove the streak artifacts effectively.

Smooth hard shrinkage operator for tensor completion based on self-adaptive transforms

The tensor completion(TC) problem has attracted wide attention due to its significant practical significance and broad application background. Various approaches have been used to solve this problem, including approximating the rank function through its convex envelope, the tensor nuclear norm. However, due to the gap between the rank function and its convex envelope, this approach often fails to achieve satisfactory recovery. Recent developments have highlighted that non-explicit non-convex functions, especially the one induced by the Smooth Hard(SH) shrinkage operator, can approximate the l0 norm more effectively compared to other methods, and this function facilitates the reconstruction of noisy MRI data with minimal sampling rate requirements. In our paper, we implement the SH shrinkage operator within the TC framework, achieving accurate tensor data recovery at lower sampling rates. Moreover, we adopt a method that adaptively generates transformation matrices based on specific features of the data, which, compared to traditional TC models, can better exploit the inherent low-rank properties of tensor data. Additionally, our approach integrates a reweighting strategy to efficiently merge tensors recovered from various dimensions. Finally, a feasible minimization algorithm is proposed, which combines the non-explicit non-convex regularizer and self-adaptive transforms. It is proven that the constructed sequence of the proposed algorithm is a Cauchy sequence and can converge to a stable point. Extensive experimental results demonstrate that our proposed approach can achieve accurate tensor recovery at lower sampling rates compared to other methods.

GlmmPen: High Dimensional Penalized Generalized Linear Mixed Models.

Generalized linear mixed models (GLMMs) are widely used in research for their ability to model correlated outcomes with non-Gaussian conditional distributions. The proper selection of fixed and random effects is a critical part of the modeling process, where model misspecification may lead to significant bias. However, the joint selection of fixed and random effects has historically been limited to lower dimensional GLMMs, largely due to the use of criterion-based model selection strategies. Here we present the R package glmmPen, one of the first to select fixed and random effects in higher dimension using a penalized GLMM modeling framework. Model parameters are estimated using a Monte Carlo expectation conditional minimization (MCECM) algorithm, which leverages Stan and RcppArmadillo for increased computational efficiency. Our package supports the Binomial, Gaussian, and Poisson families and multiple penalty functions. In this manuscript we discuss the modeling procedure, estimation scheme, and software implementation through application to a pancreatic cancer subtyping study. Simulation results show our method has good performance in selecting both the fixed and random effects in high dimensional GLMMs.

Effect of a nutraceutical combination on sleep quality among people with impaired sleep: a randomised, placebo-controlled trial

In this randomised, placebo-controlled trial, adults with impaired sleep (Pittsburgh Sleep Quality Index ≥ 5) were randomly assigned using a minimization algorithm to receive a formulation containing l-theanine plus lemon balm, valerian, and saffron extracts, or placebo, during 6 weeks. Objective sleep quality parameters were measured using an actigraphy device. We enrolled and randomised 64 individuals, 31 from the active group and 27 from the placebo group completed the 6 week follow-up. Mean sleep efficiency remained unmodified in the active group, and increased by 3% in the placebo group, the between-group difference in the change was not statistically significant (p = 0.49). Total sleep time also improved more with placebo (13.0 vs. 1.33 min, p = 0.66). Time wake after sleep onset (WASO) decreased more in the active group (4.6% vs. 2.4%), but the difference was not significant (p = 0.33). Mean PSQI decreased by 3.11 points (32.3%) in the active group, and by 3.86 points (39.5%) in the placebo group (p = 0.41). SF-36 increased more with placebo (+ 18.3 in active, + 32.1 in placebo, p = 0.68). Salivary cortisol remained unchanged in both groups. No serious adverse events were reported. Among adults with impaired sleep, a nutraceutical combination did not improve objective or subjective sleep parameters more than a placebo infusion.

An Integrated All‐Optical Ising Machine with Unlimited Spin Array Size and Coupling

Artificial spin systems are used to solve combinatorial optimization problems by mapping them to the ground state search of the Ising model. Ising machines of various designs, in fields as diverse as photonics and electronics, have been proposed and demonstrated in recent years. One important mathematical operation required in photonic Ising machines for nearly all Hamiltonian minimization algorithms is repeated matrix–vector multiplication with the vector size limited by the physical size of the optical system. Herein, an integrated photonic Ising machine based on matrix partitioning and phase encoding, which effectively allows the use of physically small optical circuits to handle arbitrarily large spin vectors, is proposed. All the complicated calculations are carried out optically, rather than electronically, in this approach. How the required surface area of a hypothetical chip‐based photonic Ising machine can be resized according to convenience, with the trade‐off being the solution time, and that this trade‐off can be done in a way that does not impact the algorithm's efficacy, is shown. Through simulation, the system is predicted to have a high success rate, high noise tolerance, and high error tolerance.

CIDR: A Cooperative Integrated Dynamic Refining Method for Minimal Feature Removal Problem

The minimal feature removal problem in the post-hoc explanation area aims to identify the minimal feature set (MFS). Prior studies using the greedy algorithm to calculate the minimal feature set lack the exploration of feature interactions under a monotonic assumption which cannot be satisfied in general scenarios. In order to address the above limitations, we propose a Cooperative Integrated Dynamic Refining method (CIDR) to efficiently discover minimal feature sets. Specifically, we design Cooperative Integrated Gradients (CIG) to detect interactions between features. By incorporating CIG and characteristics of the minimal feature set, we transform the minimal feature removal problem into a knapsack problem. Additionally, we devise an auxiliary Minimal Feature Refinement algorithm to determine the minimal feature set from numerous candidate sets. To the best of our knowledge, our work is the first to address the minimal feature removal problem in the field of natural language processing. Extensive experiments demonstrate that CIDR is capable of tracing representative minimal feature sets with improved interpretability across various models and datasets.

Hierarchical and Incremental Structural Entropy Minimization for Unsupervised Social Event Detection

As a trending approach for social event detection, graph neural network (GNN)-based methods enable a fusion of natural language semantics and the complex social network structural information, thus showing SOTA performance. However, GNN-based methods can miss useful message correlations. Moreover, they require manual labeling for training and predetermining the number of events for prediction. In this work, we address social event detection via graph structural entropy (SE) minimization. While keeping the merits of the GNN-based methods, the proposed framework, HISEvent, constructs more informative message graphs, is unsupervised, and does not require the number of events given a priori. Specifically, we incrementally explore the graph neighborhoods using 1-dimensional (1D) SE minimization to supplement the existing message graph with edges between semantically related messages. We then detect events from the message graph by hierarchically minimizing 2-dimensional (2D) SE. Our proposed 1D and 2D SE minimization algorithms are customized for social event detection and effectively tackle the efficiency problem of the existing SE minimization algorithms. Extensive experiments show that HISEvent consistently outperforms GNN-based methods and achieves the new SOTA for social event detection under both closed- and open-set settings while being efficient and robust.

A Best-Path Approach to the Design of a Hybrid Space–Ground Quantum Network with Dynamic Constraints

The design and operation of quantum networks are both decisive in the current push towards a global quantum internet. Although space-enabled quantum connectivity has already been identified as a beneficial candidate for long-range quantum channels for over two decades, the architecture of a hybrid space–ground network is still a work in progress. Here, we propose an analysis of such a network based on a best-path approach, where either fiber- or satellite-based elementary links can be concatenated to form a repeater chain. The network consisting of quantum information processing nodes, equipped with both ground and space connections, is mapped into a graph structure, where edge weights represent the achievable secret key rates, chosen as the figure of merit for the network analysis. A weight minimization algorithm allows for identifying the best path dynamically, i.e., as the weather conditions, stray light radiance, and satellite orbital position change. From the results, we conclude that satellite links will play a significant role in the future large-scale quantum internet, in particular when node distances exceed 500 km, and both a constellation of satellites—spanning 20 or more satellites—and significant advances in filtering technology are required to achieve continuous coverage.

Multicomponent Intervention for Overactive Bladder in Women

Although the cognitive components of behavioral therapy for overactive bladder (OAB) are widely recognized, there is a lack of studies evaluating the effectiveness of multicomponent interventions that include cognitive components as a treatment for OAB. To examine the efficacy of a multicomponent intervention in improving health-related quality of life (HRQOL) for women with moderate to severe OAB. This multicenter, open-label, randomized clinical trial was conducted in Japan among women aged 20 to 80 years who had moderate to severe OAB. Participants were recruited from 4 institutions between January 16, 2020, and December 31, 2022, through self-referral via advertisement or referral from the participating institutions. Participants were randomized 1:1 by minimization algorithm using an internet-based central cloud system to four 30-minute weekly sessions of a multicomponent intervention or waiting list. Both groups continued to receive baseline treatment throughout the study period. The primary outcome was the least-squares mean changes from baseline through week 13 in HRQOL total scores of the OAB questionnaire between 2 groups. Secondary outcomes included OAB symptom score and frequency volume chart. A total of 79 women were randomized to either the intervention group (39 participants; mean [SD] age, 63.5 [14.6] years) or the waiting list control group (40 participants; mean [SD] age, 63.5 [12.9] years). One participant from each group dropped out from the allocated intervention, while 5 participants in the intervention group and 2 in the control group dropped out from the primary outcome assessment at week 13. Thirty-six participants (92.3%) in the intervention group and 35 (87.5%) in the control group had moderate OAB. The change in HRQOL total score from baseline to week 13 was 23.9 points (95% CI, 18.4-29.5 points) in the intervention group and 11.3 points (95% CI, 6.2-16.4 points) in the waiting list group, a significant difference of 12.6 points (95% CI, 6.6-18.6 points; P < .001). Similar superiority of the intervention was confirmed for frequency of micturition and urgency but not for OAB symptom score. These findings demonstrate that a multicomponent intervention improves HRQOL for women with moderate to severe OAB and suggest that the cognitive component may be an effective treatment option for women with OAB. UMIN Clinical Trials Registry Identifier: UMIN000038513.

Usage of cause‐effect graphs for reliability analysis of an embedded IoT alarm software system

AbstractSoftware is a part of most complex systems used for different purposes in today's world. The reliability of these systems must be verified before putting them to use, because their malfunction may lead to catastrophic failure, the loss of human lives, or huge costs. Software reliability can be verified by performing black‐box testing, without knowledge of the complex internal structure of these systems. In this work, an embedded IoT alarm system for detection of intruders and potential robbery is proposed. It is composed of three subsystems: embedded IoT, web‐server, and mobile phone subsystems. Cause‐effect graphing technique was applied for generating three types of black‐box test suites, by using the forward‐propagation and two different minimization algorithms. These test suites were used for black‐box testing the system, which was performed after injecting a maximum of 12 faults into the system in 100 iterations. The resulting software quality and reliability metrics showed that the forward‐propagation algorithm always achieves maximum fault detection rate (FDR), and this algorithm should be used when the highest demand for reliability must be met. The optimized minimization should only be used in cases where testing is very expensive, because it did not find faults in all testing iterations. The basic optimization retained high FDR, while reducing the impact of the minimized test suite and achieved a significant increase of test effect coverage values.

Imageless robotic total knee arthroplasty determines similar coronal plane alignment of the knee (CPAK) parameters to long leg radiographs

BackgroundThe coronal plane alignment of the knee (CPAK) classification was first developed using long leg radiographs (LLR) and has since been reported using image-based and imageless robotic total knee arthroplasty (TKA) systems. However, the correspondence between imageless robotics and LLR-derived CPAK parameters has yet to be investigated. This study therefore examined the differences in CPAK parameters determined with LLR and imageless robotic navigation using either generic or optimized cartilage wear assumptions.MethodsMedial proximal tibial angle (MPTA) and lateral distal femoral angle (LDFA) were determined from the intraoperative registration data of 61 imageless robotic TKAs using either a generic 2 mm literature-based wear assumption (Navlit) or an optimized wear assumption (Navopt) found using an error minimization algorithm. MPTA and LDFA were also measured from preoperative LLR by two observers and intraclass correlation coefficients (ICCs) were calculated. MPTA, LDFA, joint line obliquity (JLO), and arithmetic hip-knee-ankle angle (aHKA) were compared between the robotic and the average LLR measurements over the two observers.ResultsICCs between observers for LLR were over 0.95 for MPTA, LDFA, JLO, and aHKA, indicating excellent agreement. Mean CPAK differences were not significant between LLR and Navlit (all differences within 0.6°, P > 0.1) or Navopt (all within 0.1°, P > 0.83). Mean absolute errors (MAE) between LLR and Navlit were: LDFA = 1.4°, MPTA = 2.0°, JLO = 2.1°, and aHKA = 2.7°. Compared to LLR, the generic wear classified 88% and the optimized wear classified 94% of knees within one CPAK group. Bland–Altman comparisons reported good agreement for LLR vs. Navlit and Navopt, with > 95% and > 91.8% of measurements within the limits of agreement across all CPAK parameters, respectively.ConclusionsImageless robotic navigation data can be used to calculate CPAK parameters for arthritic knees undergoing TKA with good agreement to LLR. Generic wear assumptions determined MPTA and LDFA with MAE within 2° and optimizing wear assumptions showed negligible improvement.

Structural knowledge-driven meta-learning for task offloading in vehicular networks with integrated communications, sensing and computing

Task offloading is a potential solution to satisfy the strict requirements of computation-intensive and latency-sensitive vehicular applications due to the limited onboard computing resources. However, the overwhelming upload traffic may lead to unacceptable uploading time. To tackle this issue, for tasks taking environmental data as input, the data perceived by roadside units (RSU) equipped with several sensors can be directly exploited for computation, resulting in a novel task offloading paradigm with integrated communications, sensing and computing (I-CSC). With this paradigm, vehicles can select to upload their sensed data to RSUs or transmit computing instructions to RSUs during the offloading. By optimizing the computation mode and network resources, in this paper, we investigate an I-CSC-based task offloading problem to reduce the cost caused by resource consumption while guaranteeing the latency of each task. Although this non-convex problem can be handled by the alternating minimization (AM) algorithm that alternatively minimizes the divided four sub-problems, it leads to high computational complexity and local optimal solution. To tackle this challenge, we propose a creative structural knowledge-driven meta-learning (SKDML) method, involving both the model-based AM algorithm and neural networks. Specifically, borrowing the iterative structure of the AM algorithm, also referred to as structural knowledge, the proposed SKDML adopts long short-term memory (LSTM) network-based meta-learning to learn an adaptive optimizer for updating variables in each sub-problem, instead of the handcrafted counterpart in the AM algorithm. Furthermore, to pull out the solution from the local optimum, our proposed SKDML updates parameters in LSTM with the global loss function. Simulation results demonstrate that our method outperforms both the AM algorithm and the meta-learning without structural knowledge in terms of both the online processing time and the network performance.