Optimization-based Techniques Research Articles

A survey of deep meta-learning

Deep neural networks can achieve great successes when presented with large data sets and sufficient computational resources. However, their ability to learn new concepts quickly is limited. Meta-learning is one approach to address this issue, by enabling the network to learn how to learn. The field of Deep Meta-Learning advances at great speed, but lacks a unified, in-depth overview of current techniques. With this work, we aim to bridge this gap. After providing the reader with a theoretical foundation, we investigate and summarize key methods, which are categorized into (i) metric-, (ii) model-, and (iii) optimization-based techniques. In addition, we identify the main open challenges, such as performance evaluations on heterogeneous benchmarks, and reduction of the computational costs of meta-learning.

Genetic interactions effects for cancer disease identification using computational models: a review.

Genome-wide association studies (GWAS) provide clear insight into understanding genetic variations and environmental influences responsible for various human diseases. Cancer identification through genetic interactions (epistasis) is one of the significant ongoing researches in GWAS. The growth of the cancer cell emerges from multi-locus as well as complex genetic interaction. It is impractical for the physician to detect cancer via manual examination of SNPs interaction. Due to its importance, several computational approaches have been modeled to infer epistasis effects. This article includes a comprehensive and multifaceted review of all relevant genetic studies published between 2001 and 2020. In this contemporary review, various computational methods are as follows: multifactor dimensionality reduction-based approaches, statistical strategies, machine learning, and optimization-based techniques are carefully reviewed and presented with their evaluation results. Moreover, these computational approaches' strengths and limitations are described. The issues behind the computational methods for identifying the cancer disease through genetic interactions and the various evaluation parameters used by researchers have been analyzed. This review is highly beneficial for researchers and medical professionals to learn techniques adapted to discover the epistasis and aids to design novel automatic epistasis detection systems with strong robustness and maximum efficiency to address the different research problems in finding practical solutions effectively.

Autonomous 3-D UAV Localization Using Cellular Networks: Deep Supervised Learning Versus Reinforcement Learning Approaches

Unmanned aerial vehicles (UAVs) are becoming an integral part of numerous commercial and military applications. In many of these applications, the UAV is required to self-navigate in highly dynamic urban environments. This means that the UAV must have the ability to determine its location in an autonomous and real time manner. Existing localization techniques rely mainly on the Global Positioning System (GPS) and do not provide a reliable real time localization solution, particularly in dense urban environments. Our objective is to propose an effective alternative solution to enable the UAV to autonomously determine its location independent of the GPS and without message exchanges. We therefore propose utilizing the existing 5G cellular infrastructure to enable the UAV to determine its 3-D location without the need to interact with the cellular network. We formulate the UAV localization problem to minimize the error of the RSSI measurements from the surrounding cellular base stations. While exact optimization techniques can be applied to accurately solve such a problem, they cannot provide the real time calculation that is needed in such dynamic applications. Machine learning based techniques are strong candidates to provide an attractive alternative to provide a near-optimal localization solution with the needed practical real-time calculation. Accordingly, we propose two machine learning-based approaches, namely, deep neural network and reinforcement learning based approaches, to solve the formulated UAV localization problem in real time. We then provide a detailed comparative analysis for each of the proposed localization techniques along with a comparison with the optimization-based techniques as well as other techniques from the literature.

Six-DOF Spacecraft Optimal Trajectory Planning and Real-Time Attitude Control: A Deep Neural Network-Based Approach.

This brief presents an integrated trajectory planning and attitude control framework for six-degree-of-freedom (6-DOF) hypersonic vehicle (HV) reentry flight. The proposed framework utilizes a bilevel structure incorporating desensitized trajectory optimization and deep neural network (DNN)-based control. In the upper level, a trajectory data set containing optimal system control and state trajectories is generated, while in the lower level control system, DNNs are constructed and trained using the pregenerated trajectory ensemble in order to represent the functional relationship between the optimized system states and controls. These well-trained networks are then used to produce optimal feedback actions online. A detailed simulation analysis was performed to validate the real-time applicability and the optimality of the designed bilevel framework. Moreover, a comparative analysis was also carried out between the proposed DNN-driven controller and other optimization-based techniques existing in related works. Our results verify the reliability of using the proposed bilevel design for the control of HV reentry flight in real time.

Novel Double Compensation for Impedance–Frequency Characteristics of Rotary Ultrasonic Machining via Multiobjective Genetic Algorithm

Rotary ultrasonic machining (RUM) is a superior technology to machine hard and brittle materials. Traditionally, the compensation optimization for the RUM system is limited to a single resonant frequency. This article presents a novel double compensation approach for the impedance and frequency regulations of RUM via multiobjective genetic algorithm (MOGA) aiming to achieve the system resonance and monitor the machining process in real time. For this, we first establish the impedance model of the rotary ultrasonic holder (RUH) by adopting the T-type circuit that includes the comprehensive electromagnetic parameters. The obtained impedance model reveals that both frequency mismatch and impedance mismatch exist in the RUM system, causing the low voltage gain and low vibration transmission. To obtain the optimal compensation to match both the frequency and impedance, an optimization model-based MOGA is developed to intelligently search the accurate capacitance values, where the Pareto frontier is employed to visualize the capacitance solution distribution. Moreover, the response feature of the RUH system is attained by using the state-space equation. Detailed comparisons of four compensation topologies show that the series–series (SS) topology offers the optimal performance, which can match both of the frequency and the impedance well, improving the output active power 7.634 times compared to the conventional one. Finally, the validity of the proposed optimization method is confirmed by using both simulations and experiments. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article was motivated by the transmission efficiency and the stability of ultrasonic vibration in the rotary ultrasonic machining (RUM). The matching issue is critical in RUM because the whole system is very complicated with multiple components such as electrical devices, piezoelectric rings, and mechanical horn. In the conventional RUM, the system matching is limited to a single resonant frequency between the piezoelectric transducer and the electrical driver. However, the impedance matching is also important to influence the ultrasonic vibration. The problem of the ultrasonic matching stems from empirical experiences of the engineers’ or simple heuristic rules instead of advanced approaches. On the other hand, optimization-based techniques are viewed as a great success in industrial and engineering applications. With the assistance of our proposed numerical model and the corresponding simulation, we found that the multiobjective genetic algorithm (MOGA) can intelligently search the optimal compensated parameters to achieve the double matching for both the resonant frequency and the impedance in the RUM system. Finally, our experiments prove that the MOGA optimization can attain optimal transmission efficiency for the electrical voltage and current compared with conventional approaches. This significantly benefits the application of the RUM technology in industry, especially for the precision machining of the hard and brittle materials.

Runtime Task Scheduling Using Imitation Learning for Heterogeneous Many-Core Systems

Domain-specific systems-on-chip, a class of heterogeneous many-core systems, are recognized as a key approach to narrow down the performance and energy-efficiency gap between custom hardware accelerators and programmable processors. Reaching the full potential of these architectures depends critically on optimally scheduling the applications to available resources at runtime. Existing optimization-based techniques cannot achieve this objective at runtime due to the combinatorial nature of the task scheduling problem. As the main theoretical contribution, this paper poses scheduling as a classification problem and proposes a hierarchical imitation learning (IL)-based scheduler that learns from an Oracle to maximize the performance of multiple domain-specific applications. Extensive evaluations with six streaming applications from wireless communications and radar domains show that the proposed IL-based scheduler approximates an offline Oracle policy with more than 99% accuracy for performance- and energy-based optimization objectives. Furthermore, it achieves almost identical performance to the Oracle with a low runtime overhead and successfully adapts to new applications, many-core system configurations, and runtime variations in application characteristics.

Optimal Corrective Dispatch of Uncertain Virtual Energy Storage Systems

High penetrations of intermittent renewable energy resources in the power system require large balancing reserves for reliable operations. Aggregated and coordinated behind-the-meter loads can provide these fast reserves, but represent energy-constrained and uncertain reserves (in their energy state and capacity). To optimally dispatch uncertain, energy-constrained reserves, optimization-based techniques allow one to develop an appropriate trade-off between closed-loop performance and robustness of the dispatch. Therefore, this paper investigates the uncertainty associated with energy-constrained aggregations of flexible, behind-the-meter distributed energy resources (DERs). The uncertainty studied herein is associated with estimating the state of charge and the capacity of an aggregation of DERs (i.e., a virtual energy storage system or VESS). To that effect, a risk-based chance constrained control strategy is developed that optimizes the operational risk of unexpectedly saturating the VESS against deviating generators from their scheduled set-points. The controller coordinates energy-constrained VESSs to minimize unscheduled participation of and overcome ramp-rate limited generators for balancing variability from renewable generation, while taking into account grid conditions. To illustrate the effectiveness of the proposed method, simulation-based analysis is carried out on an augmented IEEE RTS-96 network with uncertain energy resources and temperature-based dynamic line ratings.

Crisp and interval inventory models for ameliorating item with Weibull distributed amelioration and deterioration via different variants of quantum behaved particle swarm optimization-based techniques

ABSTRACTThis paper presents two inventory models for ameliorating items under crisp and interval environments. In these models, three-parameter Weibull distribution is considered to represent both the amelioration and deterioration rates. In crisp, an inventory model is formulated for ameliorating item with fixed values of different inventory parameters. Due to uncertainty, these parameters may not be fixed. In this context, another inventory model with interval valued parameters is developed. Also, demand is dependent on the selling price and advertisement frequency of the product. The corresponding profit maximization problem has been developed. For solving the problem, different variants of quantum behaved particle swarm optimization technique (QPSO) are applied. To validate the proposed models, two numerical examples are considered and solved. The results are compared for different variants of QPSO techniques. Finally, graphical sensitivity analyses are presented to study the impact of several system parameters on cycle length, initial stock level along with average profit for both the models.

Real-Time Reentry Trajectory Planning of Hypersonic Vehicles: A Two-Step Strategy Incorporating Fuzzy Multiobjective Transcription and Deep Neural Network

A two-step strategy is developed for real-time trajectory planning of a hypersonic vehicle (HV) in the reentry phase. The first step generates the optimal trajectory for the HV using a recently proposed fuzzy multiobjective transcription method. In the second step, the optimally generated trajectories are utilized to train a deep neural network (DNN), which is then acted as the optimal command generator in real time. A detailed simulation study is carried out to verify the effectiveness and real-time applicability of the proposed integrated design. The DNN-driven controller is further compared against other optimization-based techniques existing in relative works. Moreover, extension works on the real-time trajectory planning of a six-degree-of-freedom HV model are performed. The results confirm the feasibility and reliability of applying the proposed method for the planning of the HV entry flight path in real time.

Issue: Selected paper of OIPE 2016 on modeling and optimal design of electromagnetic and electronic devices

Issue: Selected paper of OIPE 2016 on modeling and optimal design of electromagnetic and electronic devices

An Overview on Denial-of-Service Attacks in Control Systems: Attack Models and Security Analyses.

In this paper, we provide an overview of recent research efforts on networked control systems under denial-of-service attacks. Our goal is to discuss the utility of different attack modeling and analysis techniques proposed in the literature for addressing feedback control, state estimation, and multi-agent consensus problems in the face of jamming attacks in wireless channels and malicious packet drops in multi-hop networks. We discuss several modeling approaches that are employed for capturing the uncertainty in denial-of-service attack strategies. We give an outlook on deterministic constraint-based modeling ideas, game-theoretic and optimization-based techniques and probabilistic modeling approaches. A special emphasis is placed on tail-probability based failure models, which have been recently used for describing jamming attacks that affect signal to interference-plus-noise ratios of wireless channels as well as transmission failures on multi-hop networks due to packet-dropping attacks and non-malicious issues. We explain the use of attack models in the security analysis of networked systems. In addition to the modeling and analysis problems, a discussion is provided also on the recent developments concerning the design of attack-resilient control and communication protocols.

Efficiency ranking using dominance network and multiobjective optimization indexes

This paper presents a new approach for ranking organizational units within a benchmarking context. Instead of the conventional optimization-based techniques, the proposed approach uses Social network analysis and Multiobjective optimization concepts to extract second-order features from the input-output data, integrating the resulting multidimensional information using TOPSIS. It shows how several expert and intelligent systems techniques can be harmoniously integrated and applied to performance assessment. The proposed approach has been used for ranking the performance of 27 major US airlines, comparing the results with some existing Data Envelopment Analysis methods. It is shown that the use of a richer information set instead of the raw input-output data leads to an innovative and more effective way of discriminating between efficient units.

Three-operator splitting scheme with the reference image regularization for electrical capacitance tomography

The image reconstruction is an important step in the electrical capacitance tomography (ECT) technology, and its performance directly impacts the reconstruction precision (RP). Beyond existing optimization-based imaging techniques, in this study the data-dependent reference image abstracted by the regularized random vector functional link network (RVFLN) and the domain expertise about imaging targets (ITs) are simultaneously encapsulated as regularizers to form a more effective imaging model. The three-operator splitting (TOS) technique is developed to solve the proposed imaging model more effectively, which extends the flexibility of the TOS method with the improvement in the utilization of image priors. Numerical validation results indicate that the proposed imaging technique achieves better reconstructions as compared with the state-of-the-art methods.

Analysis of Competitor Intelligence in the Era of Big Data: An Integrated System Using Text Summarization Based on Global Optimization

Automatic text summarization can be applied to extract summaries from competitor intelligence (CI) corpora that organizations create by gathering textual data from the Internet. Such a representation of CI text is easier for managers to interpret and use for making decisions. This research investigates design of an integrated system for CI analysis which comprises clustering and automatic text summarization and evaluates quality of extractive summaries generated automatically by various text-summarization techniques based on global optimization. This research is conducted using experimentation and empirical analysis of results. A survey of practicing managers is also carried out to understand the effectiveness of automatically generated summaries from CI perspective. Firstly, it shows that global optimization-based techniques generate good quality extractive summaries for CI analysis from topical clusters created by the clustering step of the integrated system. Secondly, it shows the usefulness of the generated summaries by having them evaluated by practicing managers from CI perspective. Finally, the implication of this research from the point of view of theory and practice is discussed.

Particle swarm optimization for 3D object tracking in RGB-D images

3D object tracking allows augmented reality applications to add virtual content to the real world in a coherent way. With the popularization of RGB-D sensors, new 6 degree-of-freedom tracking techniques that use features extracted from 3D point clouds have been developed, providing more accurate results. This work proposes the use of particle swarm optimization to handle multiple pose hypotheses during top-down tracking from RGB-D images. A fitness function based on 3D point coordinates, color, and normal information was designed, which is able to handle partial object occlusions by applying a threshold to the Euclidean distance between 3D points. The best particle found for a given frame is kept to compose the set of particles of the next swarm and its components are used as an initial pose to define the boundaries of the search space for tracking in the next frame. In addition, we have taken advantage of GPU processing to reduce the running time. Experiments with a publicly available dataset showed that the use of GPU allowed fast object tracking and that the proposed method presents more accurate tracking results in comparison to state-of-the-art optimization-based techniques, especially in situations where objects are partially occluded.

Viability and Feasibility of Constrained Kinematic Control of Manipulators

Recent advances in planning and control of robot manipulators make an increasing use of optimization-based techniques, such as model predictive control. In this framework, ensuring the feasibility of the online optimal control problem is a key issue. In the case of manipulators with bounded joint positions, velocities, and accelerations, feasibility can be guaranteed by limiting the set of admissible velocities and positions to a viable set. However, this results in the imposition of nonlinear optimization constraints. In this paper, we analyze the feasibility of the optimal control problem and we propose a method to construct a viable convex polyhedral that ensures feasibility of the optimal control problem by means of a given number of linear constraints. Experimental and numerical results on an industrial manipulator show the validity of the proposed approach.

Review and Analysis of Evolutionary Optimization-Based Techniques for FIR Filter Design

Recent times have witnessed a wide application of evolutionary optimization by researchers, in design of digital FIR filters, based on frequency domain specifications. A significant growth has been reported in the field of evolutionary optimization-based FIR filter design. Optimization-based techniques are used to solve the filter design problem by framing the design task as an error function which is further solved to determine the filter coefficients that satisfies the desired specifications. However, the nonlinear, non-differentiable, non-convex, multimodal nature of the associated optimization problem makes the design task quite challenging. In this regard, a number of evolutionary optimization- based techniques have been applied for FIR filter design. This paper provides a comprehensive review of the various evolutionary optimization-based techniques for FIR filter design. In addition to the review, the reported techniques have been analyzed by implementing them on a common platform and comparing them in terms of their effectiveness in meeting the desired specifications.

Complexity reduction of Model Predictive Control for a de-manufacturing plant

The use of advanced optimization-based techniques will be a fundamental step towards performance enhancement of flexible manufacturing plants. However, the mixed integer nature of the resulting optimization problems and the associated computational issues can represent a bottleneck and a severe limitation to their diffusion. This paper describes the design and efficient implementation of a Model Predictive Control (MPC) algorithm for a de-manufacturing plant. To reduce the computational effort due to the on-line optimization, the main idea is to resort to the so-called control horizon, widely used in classical MPC applications. A heuristic control strategy, requiring a lower computational effort, is first designed. Then, the MPC algorithm is implemented by considering a prediction horizon larger than the control horizon, and assuming that the heuristic rules are used from the end of the control horizon onwards. This significantly reduces the number of optimization variables to be computed through the solution of a Mixed Integer Linear Programming (MILP) problem. Notably, by adopting a rolling horizon strategy, the heuristic rules are never applied in practice. Simulation results are reported to compare the performances of the algorithm here developed, in terms of computational time and plant throughput, to those of a standard MPC problem and of the heuristic rules. These results witness the ability to the developed method to reduce the on-line optimization time without a significant performance reduction with respect to the standard MPC.

Investigation of optimization-based reconstruction with an image-total-variation constraint in PET

Interest remains in reconstruction-algorithm research and development for possible improvement of image quality in current PET imaging and for enabling innovative PET systems to enhance existing, and facilitate new, preclinical and clinical applications. Optimization-based image reconstruction has been demonstrated in recent years of potential utility for CT imaging applications. In this work, we investigate tailoring the optimization-based techniques to image reconstruction for PET systems with standard and non-standard scan configurations. Specifically, given an image-total-variation (TV) constraint, we investigated how the selection of different data divergences and associated parameters impacts the optimization-based reconstruction of PET images. The reconstruction robustness was explored also with respect to different data conditions and activity up-takes of practical relevance. A study was conducted particularly for image reconstruction from data collected by use of a PET configuration with sparsely populated detectors. Overall, the study demonstrates the robustness of the TV-constrained, optimization-based reconstruction for considerably different data conditions in PET imaging, as well as its potential to enable PET configurations with reduced numbers of detectors. Insights gained in the study may be exploited for developing algorithms for PET-image reconstruction and for enabling PET-configuration design of practical usefulness in preclinical and clinical applications.

Mixed-Integer and Constraint Programming Techniques for Mobile Robot Task Planning

We investigate the use of optimization-based techniques to model and solve two real-world single robot task planning problems. In the first problem, a robot must plan a set of tasks, each with different temporal constraints. In the second problem, a socially interacting robot must plan a set of tasks while considering the schedules of multiple human users, as well as physical constraints including battery level. We apply existing mixed-integer programming and constraint programming techniques, yielding two exact methods for each problem. Numerical experiments show that both approaches produce better solutions in less time compared to techniques previously proposed for these problems. In order to confirm their physical utility, we implement the plans for the second problem in both simulated and real environments on Tangy, a socially interacting robot. We conclude that both mixed-integer programming and constraint programming are promising general approaches to robot task planning that should be considered when solving these problems.