Algorithm Configuration Research Articles

Enhancing unsupervised anomaly-based cyberattacks detection in smart homes through hyperparameter optimization

As smart homes become increasingly interconnected, it is crucial to ensure their security against cyber threats. This research focuses on improving anomaly detection within smart home environments through the application of unsupervised learning methods. Unlike traditional methods that require prior knowledge of specific attack types, unsupervised methods learn from normal operational data, identifying anomalies as deviations from this norm. This study investigates the impact of hyperparameter tuning on the effectiveness of these methods compared to their default settings. This research aims to identify optimal strategies for the configuration of unsupervised learning algorithms in smart home environments. We applied four models (Elliptic Envelope, Isolation Forest, Local Outlier Factor, and One-class SVM) in four datasets (Bot-IoT, IoTID20, N-baiot, and Ton-IoT). Our findings indicate a significant improvement in performance with hyperparameter optimization.

Finding Near-Optimal Weight Independent Sets at Scale

Computing maximum weight independent sets in graphs is an important NP-hard optimization problem. The problem is particularly difficult to solve in large graphs for which data reduction techniques do not work well. To be more precise, state-of-the-art branch-and-reduce algorithms can solve many large-scale graphs if reductions are applicable. Otherwise, their performance quickly degrades due to branching requiring exponential time. In this paper, we develop an advanced memetic algorithm to tackle the problem, which incorporates recent data reduction techniques to compute near-optimal weight independent sets in huge sparse networks. More precisely, we use a memetic approach to recursively choose vertices that are likely to be in a large-weight independent set. We include these vertices into the solution, and further reduce the graph. We show that identifying and removing vertices likely to be in large-weight independent sets opens up the reduction space and speeds up the computation of large-weight independent sets remarkably. Our experimental evaluation indicates that we are able to outperform state-of-the-art algorithms. For example, our two algorithm configurations compute the best results among all competing algorithms for all instances tested. Thus, it can be seen as a useful tool when large-weight independent sets need to be computed in practice.

Automated search of an optimal configuration of FETI-based algorithms with the swarm and evolutionary algorithms

Finite Element Tearing and Interconnecting (FETI) methods are used in the engineering community to solve extremely large engineering simulations on clusters and supercomputers with thousands of computational nodes. This paper focuses on minimizing the execution time of these methods by searching for an optimal configuration with swarm and evolutionary algorithms (SEAs). It incorporates optimization of an expensive cost function (taking tens to hundreds of seconds) of discrete search space with up to hundreds of thousands of combinations constrained by its boundaries and incompatible individuals (invalid combination of FETI solver parameters). In addition, the optimization occurs in real time, i.e., during a simulation. Hence, the number of objective function evaluations must remain low (tens to lower hundreds). The paper compares the performance of 3 basic SEAs with a reduced population size (Differential Evolution, Particle Swarm Optimization, and Self-Organizing Migrating Algorithm), 4 micro SEAs (Micro Differential Evolution Ray, Improved Micro-Particle Swarm Optimization, Micro-Particle Swarm Optimization, and Micro-Genetic Algorithm), and a random search on 4 distinct time-dependent simulations of a heat transfer. The experiments show that basic SEAs with a small population (about 5 individuals) and a penalty system as protection against incompatible configurations represent the most effective solution. One can use them to find the optimal configuration of FETI-based methods in approximately 130 evaluations. It can improve the utilization of expensive hardware resources of modern computational clusters.

Experimental study of a distributed active noise control system with multi-device nodes based on augmented diffusion strategy.

Recently, distributed active noise control (DANC) algorithms have been explored as a way to reduce computational complexity while ensuring system stability, thereby outperforming conventional centralized and decentralized algorithms. Most existing DANC algorithms assume that each node has only one pair of loudspeaker and microphone, limiting their flexibility in practical applications. In contrast, this paper proposes a DANC algorithm with general multi-device nodes based on the recently developed augmented diffusion strategy, allowing flexible and scalable ANC applications. A real-time distributed ANC system based on a multi-core digital signal processor platform is developed in order to compare the control performance of the proposed extended augmented diffusion algorithm with that of existing centralized, decentralized and augmented diffusion algorithms. Real-time experiments demonstrate that the proposed algorithm exhibits noise reduction performance consistent with that of the centralized algorithm while achieving lower global computational complexity and avoiding the system instability risk of the decentralized algorithm. Further, the new algorithm improves convergence speed and reduces the global communication cost compared to the previous augmented diffusion algorithm. Experimental results indicate the application potential of the proposed DANC algorithm for a generalized system configuration.

Machine learning and deep learning models for the diagnosis of apical periodontitis: a scoping review.

To assess the existing literature on the use of machine learning (ML) and deep learning (DL) models for diagnosing apical periodontitis (AP) in humans. A scoping review was conducted following the Arksey and O'Malley framework. The PubMed, SCOPUS, and Web of Science databases were searched, focusing on articles using ML/DL approaches for AP diagnosis. No restrictions were applied. Two independent reviewers screened publications and charted data in predefined Excel tables for analysis. Nineteen publications focused on diagnosing AP by identifying periapical radiolucent lesions (PRLs) in dental radiographs were included. The average sensitivity and specificity for reviewed models were 83% and 90%, respectively. Only three studies explored the direct impact of artificial intelligence (AI) assistance on clinicians' diagnostic performance. Both consistently showed improved sensitivity without compromising specificity. Significant variability in dataset sizes, labeling techniques, and algorithm configurations was noticed. Findings affirm AI models' effectiveness and transformative potential in diagnosing AP by improving the accurate detection of periapical radiolucencies using dental radiographs. However, the lack of standardized reporting on crucial aspects of methodology and performance metrics prevents establishing a definitive diagnostic approach using AI. Further studies are needed to expand AI applications in AP diagnosis beyond radiographic analysis. AI can potentially improve diagnostic accuracy in AP diagnosis by enhancing the sensitivity of PRL detection in dental radiographs without compromising specificity.

Structured pruning of neural networks for constraints learning

In recent years, the integration of Machine Learning (ML) models with Operation Research (OR) tools has gained popularity in applications such as cancer treatment, algorithmic configuration, and chemical process optimization. This integration often uses Mixed Integer Programming (MIP) formulations to represent the chosen ML model, that is often an Artificial Neural Networks (ANNs) due to their widespread use. However, ANNs frequently contain a large number of parameters, resulting in MIP formulations impractical to solve. In this paper we showcase the effectiveness of a ANN pruning, when applied to models prior to their integration into MIPs. We discuss why pruning is more suitable in this context than other ML compression techniques, and we highlight the potential of appropriate pruning strategies via experiments on MIPs used to construct adversarial examples to ANNs. Our results demonstrate that pruning offers remarkable reductions in solution times without hindering the quality of the final decision, enabling the resolution of previously unsolvable instances.

Control parameter optimisation using the evidence framework for the ant colony optimisation algorithm

The ant colony optimization (ACO) algorithm is a metaheuristic initially designed to solve the travelling salesman problem (TSP). The design of experiments, finding the suitable ACO algorithm configuration, and calibrating the adaptive control parameters are exhaustive and time-consuming exercises, especially for TSPs where the number of cities can exceed 1000. This paper presents an evidence framework driven control parameter optimisation (EFCPO) algorithm for an ACO algorithm solving TSPs. EFCPO performs auto-tuning of the adaptive control parameters and makes recommendations about the ACO algorithms that are best suited for the TSPs in question using the log evidence. In addition, with this ability, the algorithm can take a solution provided by an ACO algorithm and improve the results. The EFCPO accomplishes this over a number of cycles through auto-tuning of the control parameters and re-running the ACO until the process is completed. The capabilities of EFCPO are compared to another configuration tool, irace, using benchmark ACO algorithms to test the efficiency of the framework. The benchmark algorithms make use of a local search strategy to solve TSPs. The results show that ACO algorithms are able to find new and improved solution tours within reasonable times. The improvements are also significant. In addition, ACO algorithms that are best suited for the TSP in question are preferred, making the EFCPO an effective tool for real-time configuration of ACO algorithms for solving TSPs.

Research on Digital Twin Method for Spaceborne Along-Track Interferometric Synthetic Aperture Radar Velocity Inversion of Ocean Surface Currents

In this paper, an end-to-end system framework is proposed for the Digital Twin study of spaceborne ATI-SAR ocean current velocity inversion. Within this framework, a fitting inversion approach is proposed to enhance the conventional spaceborne ATI-SAR ocean current velocity inversion algorithm. Consequently, the issue of possible local inversion errors stemming from the mismatch between the traditional spaceborne ATI-SAR inversion algorithm and various dual-antenna configurations is resolved to a certain extent. A simulated spaceborne ATI-SAR system, featuring a dual-antenna configuration comprising a baseline direction perpendicular to the track and a squint angle, is presented to validate the efficacy of the Digital Twin methodology. Under the specified simulation parameters, the average inversion error for the final ocean current velocity is recorded at 0.0084 m/s, showcasing a reduction of 0.0401 m/s compared with the average inversion error prior to optimization.

Modular stochastic configuration network with attention mechanism for soft measurement of water quality parameters in wastewater treatment processes

Due to the complex and multi-distributed nature of wastewater treatment process data, stochastic configuration networks (SCNs) face difficulties such as large network structure and complex construction process when dealing with data modeling tasks. To address these challenges, this paper proposes a modular stochastic configuration network with attention mechanism, which integrates the advantages of modular partitioning, attention mechanism, and SCN. The model employs a fuzzy mean clustering strategy to simplify and decompose the complex multi-case and multi-distribution nonlinear modeling task, thereby reducing the modeling complexity of a single model. Furthermore, the model utilizes a stochastic configuration algorithm to learn the subtasks and construct corresponding sub-networks. The output weights are then assigned to each sub-network based on the attention mechanism to obtain the model’s final output. The validity of the proposed method is validated through two benchmark experiments. Then the model is applied to measure effluent ammonia nitrogen. The results indicate that compared with the classical randomized and modularized network models, the proposed model exhibits certain advantages in terms of learning accuracy and generalization performance.

Pflacco: Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems in Python.

The herein proposed Python package pflacco provides a set of numerical features to characterize single-objective continuous and constrained optimization problems. Thereby, pflacco addresses two major challenges in the area of optimization. Firstly, it provides the means to develop an understanding of a given problem instance, which is crucial for designing, selecting, or configuring optimization algorithms in general. Secondly, these numerical features can be utilized in the research streams of automated algorithm selection and configuration. While the majority of these landscape features are already available in the R package flacco, our Python implementation offers these tools to an even wider audience and thereby promotes research interests and novel avenues in the area of optimization.

Development of a robotic-assisted handling and manipulation system for the high-scale bioproduction of 3D-bioprinted organ-on-a-chip devices

Organs-on-a-chip (OoCs) have proven to mimic the basic physiological behavior of organs and the influence of therapeutics on them in greater detail than conventional models, resulting in enormous projected market growth rates. However, the breakthrough to profitable commercialization of that technology has not yet been achieved, partly because the production process chain is characterized by a high proportion of manual laboratory work. The present work addresses this point. Utilizing affordable components, a demonstrator was developed that can be integrated into an existing 3D-bioprinting system and enables the automated production of perfusion-ready OoC devices starting from pre-fabricated injection-molded microfluidic chips. To this end, a corresponding process chain was first defined, and an expandable, configurable algorithm was developed and validated in the form of a finite state machine (FSM). This algorithm controls a modified 4-axis robot arm that covers the steps upstream and downstream of the printing process in the manufacturing process and achieves success rates of up to 100 %. A virtual interface between the robot and printer enables mutual communication and full integration of the algorithm into the process chain. Steps that pose a challenge for the automation of the process chain and appropriate countermeasures and optimizations were identified. This lays the foundation for scaling and standardizing the automated production of OoCs.

Benchmarking for platform-aggregated manufacturing service collaboration: Methodology and implementation

In light of the global economic downturn and the intricate division of labor in manufacturing, the imperative for advanced manufacturing services and Manufacturing Service Collaboration (MSC) has escalated significantly. As manufacturing services gravitate towards aggregation on manufacturing service platforms, platform-aggregated MSC has emerged as an inevitable and compelling trend, capturing the attention of researchers worldwide. However, despite the existence of numerous frameworks, models, operational mechanisms, and algorithms proposed for the platform-aggregated MSC, drawing comparisons between these studies remains a complex endeavor. To address this predicament, this article proposes and explores a novel benchmarking methodology for platform-aggregated MSC. By employing complex network theory, a comprehensive model of platform-aggregated MSC is constructed and supplemented with corresponding methodologies for data generation and the configuration of optimization algorithms. Moreover, pertinent performance evaluation metrics are scrutinized to assess their applicability in the context of platform-aggregated MSC. The article culminates with the execution of a series of platform operation experiments designed to test the effectiveness and practicality of the proposed benchmarking system, thereby contributing to the ongoing evolution of the MSC domain.

A fast algorithm for the two-dimensional Helmholtz transmission problem with large multiple scattering configurationsa).

We develop an efficient three-stage algorithm for simulating multiple acoustic scattering by two-dimensional configurations comprising large numbers of penetrable scatterers. Our approach is based on a boundary integral equation reformulation of the Helmholtz transmission partial differential equation, and a reduction of the boundary integral system for computationally efficient evaluation of wave interactions between scatterers. A key ingredient of our algorithm is to represent the interactions between scatterers using expansions of cylindrical wavefunctions. For large numbers of scatterers, this approach facilitates the application of the fast multipole method, leading to linear complexity of the algorithm with respect to the number of scatterers. Numerical results demonstrate the efficiency of our algorithm for configurations containing a few hundred to hundreds of thousands of individual scatterers.

Selecting fast algorithms for the capacitated vehicle routing problem with machine learning techniques

AbstractWe present machine learning (ML) methods for automatically selecting a “best” performing fast algorithm for the capacitated vehicle routing problem (CVRP) with unit demands. Algorithm selection is to automatically choose among a portfolio of algorithms the one that is predicted to work best for a given problem instance, and algorithm configuration is to automatically select algorithm's parameters that are predicted to work best for a given problem instance. We present a framework incorporating both algorithm selection and configuration for a portfolio that includes the automatically configured “Sweep Algorithm,” the first generated feasible solution of the hybrid genetic search algorithm, and the Clarke and Wright algorithm. The automatically selected algorithm is shown here to deliver high‐quality feasible solutions within very small running times making it highly suitable for real‐time applications and for generating initial feasible solutions for global optimization methods for CVRP. These results bode well to the effectiveness of utilizing ML for improving combinatorial optimization methods.

A hierarchical reinforcement learning-aware hyper-heuristic algorithm with fitness landscape analysis

The automation of meta-heuristic algorithm configuration holds the utmost significance in evolutionary computation. A hierarchical reinforcement learning-aware hyper-heuristic algorithm with fitness landscape analysis (HRLHH) is proposed to flexibly configure the suitable algorithms under various optimization scenarios. Two kinds of fitness landscape analysis techniques improved based on specific problem characteristics construct the state spaces for hierarchical reinforcement learning. Among them, an adaptive classification based on dynamic ruggedness of information entropy is designed to discern the complexity of problems, which serves as the basis for decision-making actions in upper-layer space. Additionally, an online dispersion metric based on knowledge is further presented to distinguish the precise landscape features in lower-layer space. In light of the characteristics of the state spaces, the hierarchical action spaces composed of meta-heuristics with disparate exploration and exploitation are designed, and various action selection strategies are introduced. Taking into account the real-time environment and algorithm evolution behavior, dynamic reward mechanisms based on evolutionary success rate and population convergence rate are utilized to enhance search efficiency. The experimental results on the IEEE Congress on Evolutionary Computation (CEC) 2017, CEC 2014, and large-scale CEC 2013 test suites demonstrate that the proposed HRLHH exhibits superiority in terms of accuracy, stability, and convergence speed, and possesses strong generalization.

Per-Instance Algorithm Configuration in Homogeneous Instance Spaces: A Use Case in Reconfigurable Assembly Systems

The physical capabilities of a reconfigurable assembly system (RAS) increase the agility and responsiveness of the system in highly volatile market conditions. However, achieving optimal RAS utilization entails solving complex optimization problems effectively and efficiently. These optimizations often define homogenous sets of problem instances. While algorithm configuration in such homogeneous contexts traditionally adopts a “one-size-fits-all” approach, recent studies have shown the potential of per-instance algorithm configuration (PIAC) methods in these settings. In this work, we evaluate and compare the performance of different PIAC methods in this context, namely Hydra—a state-of-the-art PIAC method—and a simpler case-based reasoning (CBR) approach. We evaluate the impact of the tuning time budget and/or the number of unique problem instances used for training on each of the method’s performance and robustness. Our experiments show that whilst Hydra fails to improve upon the default algorithm configuration, the CBR method can lead to 16% performance increase using as few as 100 training instances. Following these findings, we evaluate Hydra’s methodology when applied to homogenous instance spaces. This analysis shows the limitations of Hydra’s inference mechanisms in these settings and showcases the advantages of distance-based approaches used in CBR.

Predicting the Influence of Ammonium Toxicity Levels in Water Using Fuzzy Logic and ANN Models

The study aimed to address the complex and critical issue of surface water quality monitoring by proposing a diversified approach that incorporates a range of chemical indicators. (1) Background: the purpose of the study was to address the problem of surface water quality monitoring in relation to the toxic effects of ammonium on aquatic ecosystems by developing predictive models using fuzzy logic and artificial neural networks. (2) Water samples from the Styr River, influenced by the Rivne Nuclear Power Plant, were analyzed using certified standard methods and measured parameters, while fuzzy logic and artificial neural network models, including Mamdani’s algorithm and various configurations of activation functions and optimization algorithms, were employed to assess water quality and predict ammonium toxicity. (3) A fuzzy logic system was developed to classify water quality based on ammonia content and other parameters, and six Artificial Neural Network (ANN) models were tested, with the ANN#2 model (using ReLU activation and ADAM optimizer) showing the best performance. (4) This study emphasizes the critical need for precise monitoring and modeling of total ammonium in surface water, considering its variable toxicity and interactions with environmental factors, to effectively protect aquatic ecosystems, namely ichthyofauna.

Opt2Vec - a continuous optimization problem representation based on the algorithm's behavior: A case study on problem classification

Characterization of the optimization problem is a crucial task in many recent optimization research topics (e.g., explainable algorithm performance assessment, and automated algorithm selection and configuration). The state-of-the-art approaches use exploratory landscape analysis to represent the optimization problem, where for each one, a set of features is extracted using a set of candidate solutions sampled by a sampling strategy over the whole decision space. This paper proposes a novel representation of continuous optimization problems by encoding the information found in the interaction between an algorithm and an optimization problem. The new problem representation is learned using the information from the states/positions in the optimization run trajectory (i.e., the candidate solutions visited by the algorithm). With the novel representation, the problem can be characterized dynamically during the optimization run, instead of using a set of candidate solutions from the whole decision space that have never been observed by the algorithm. The novel optimization problem representation is called Opt2Vec and uses an autoencoder type of neural network to encode the information found in the interaction between an optimization algorithm and optimization problem into an embedded subspace. The Opt2Vec representation efficiency is shown by enabling different optimization problems to be successfully identified using only the information obtained from the optimization run trajectory.

A data-driven LSTMSCBLS model for soft sensor of industrial process

Abstract In the chemical industry, data-driven soft sensor modeling plays a crucial role in efficiently monitoring product quality and status. Industrial data in these applications typically exhibit significant temporal characteristics, meaning that current information is influenced by data from previous periods. Effectively extracting and utilizing these temporal features is essential for achieving accurate soft sensor modeling in complex chemical scenarios. To address this challenge, this study proposes a data-driven Broad Learning System (BLS) model, which combines Long Short-Term Memory (LSTM) networks with an adaptive algorithm known as the Stochastic Configuration Algorithm (SC), referred to as LSTMSCBLS. The model operates in two stages: temporal feature extraction and final prediction. In the temporal feature extraction stage, the integration of the LSTM network with a feature attention mechanism allows for efficient extraction of temporal features from high-dimensional time-series data. In the final prediction stage, the SC is integrated into the BLS, effectively mitigating issues related to node space redundancy and the determination of the number of nodes. The effectiveness and superiority of the proposed model are demonstrated through two industrial case studies involving a debutanizer column and a sulfur recovery unit.

Configuring Mixed-Integer Programming Solvers for Large-Scale Instances

Algorithm configuration techniques automatically search for parameters of solvers and algorithms that provide minimal runtime or maximal solution quality on specified instance sets. Mixed-integer programming (MIP) solvers pose a particular challenge for algorithm configurators due to the difficulty of finding optimal, or even feasible, solutions on the large-scale problems commonly found in practice. We introduce the OPTANO Algorithm Tuner (OAT) to find configurations for MIP solvers and other optimization algorithms. We present and evaluate several critical components of OAT for solving MIPs in particular and show that OAT can find configurations that significantly improve the solution time of MIPs on two different datasets.