Simple Heuristic Algorithm Research Articles

Experimental analysis of algorithms for the independent quadratic assignment problem

The independent quadratic assignment problem is a variation of the well-known Koopmans–Beckman quadratic assignment problem. The model has a wide variety of applications but it is strongly NP-hard and is also hard to approximate. In this paper we present innovative and simple heuristic algorithms to optimize the problem based on the paradigms of local search, very large-scale neighborhood (VLSN) search, and local search with path-relinking. The performance of these algorithms are analyzed using results generated by extensive computational experiments. Our experimental study highlights that the strategies of path-relinking and VLSN search when worked in unison produced superior outcomes, outperforming algorithms that employ these strategies individually. This underscores the power of hybrid algorithmic approaches in solving complex optimization problems, in particular, the independent quadratic assignment problem. This is the first experimental study available on the independent quadratic assignment problem. We also have constructed (and made available) test instances with different characteristics which can be used as benchmarks in future experimental studies on the problem.

Optimisation of pipes with constant diameter using the heuristic optimality criterion

Background: Minimising internal pressure drop in pipes is crucial for energy efficiency of fluid flow applications. Numerous computational optimisation tools that are capable of modifying flow geometries to reduce the pressure drop have been developed. Among these is a comparably simple heuristic optimisation algorithm which mimics erosion and sedimentation processes based on the shear stress in the vicinity of the domain boundaries. Although this method succeeds in modifying flow geometries for reduced pressure drop, it allows the fluid domain to widen during the reshaping process. Therefore, a reported reduction of pressure drop is not only caused by an improvement of the flow path, but also by an increase in the domain width. However, pipes with a constant circular diameter are favoured in many applications because they can be easily manufactured. Methods: Here we combine the heuristic optimisation approach with a new geometrical constraint that kept the average diameter constant. This way, a reduction of pressure drop caused solely by the modification of the flow path can be assessed. We determined the applicability of the modified algorithm for 2D channel and 3D pipe geometries, conducting numerical simulations using the Lattice Boltzmann method with Reynolds numbers ranging from 40 to 500. Results: For all investigated cases, potentially optimal shapes could be derived at all tested Reynolds numbers. However, the shape originally derived at a simulated flow with Re = 40 yielded a smaller pressure drop even for flows at Re ≥ 100 than shapes derived specifically at these higher Reynolds numbers. Conclusions: This observed trend should be kept in mind when employing this approach as a simple way to improve channel and pipe layouts.

FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placement

With the increasing data volume, there is a trend of using large-scale pre-trained models to store the knowledge into an enormous number of model parameters. The training of these models is composed of lots of dense algebras, requiring a huge amount of hardware resources. Recently, sparsely-gated Mixture-of-Experts (MoEs) are becoming more popular and have demonstrated impressive pretraining scalability in various downstream tasks. However, such a sparse conditional computation may not be effective as expected in practical systems due to the routing imbalance and fluctuation problems. Generally, MoEs are becoming a new data analytics paradigm in the data life cycle and suffering from unique challenges at scales, complexities, and granularities never before possible. In this paper, we propose a novel DNN training framework, FlexMoE, which systematically and transparently address the inefficiency caused by dynamic dataflow. We first present an empirical analysis on the problems and opportunities of training MoE models, which motivates us to overcome the routing imbalance and fluctuation problems by a dynamic expert management and device placement mechanism. Then we introduce a novel scheduling module over the existing DNN runtime to monitor the data flow, make the scheduling plans, and dynamically adjust the model-to-hardware mapping guided by the real-time data traffic. A simple but efficient heuristic algorithm is exploited to dynamically optimize the device placement during training. We have conducted experiments on both NLP models (e.g., BERT and GPT) and vision models (e.g., Swin). And results show FlexMoE can achieve superior performance compared with existing systems on real-world workloads --- FlexMoE outperforms DeepSpeed by 1.70x on average and up to 2.10x, and outperforms FasterMoE by 1.30x on average and up to 1.45x.

Spatio-temporal chaos and clustering induced by nonlocal information and vaccine hesitancy in the SIR epidemic model

Human behavior, and in particular vaccine hesitancy, is a critical factor for the control of childhood infectious disease. Here we propose a spatio-temporal behavioral epidemiology model where the vaccine propensity depends on information that is non-local in space and in time. The properties of the proposed model are analyzed under different hypotheses on the spatio-temporal kernels tuning the vaccination response of individuals. As a main result, we could numerically show that vaccine hesitancy induces the onset of many dynamic patterns of relevance for epidemiology. In particular we observed: behavior-modulated patterns and spatio-temporal chaos. This is the first known example of human behavior-induced spatio-temporal chaos in statistical physics of vaccination. Patterns and spatio-temporal chaos are difficult to deal with, from the Public Health viewpoint, hence showing that vaccine hesitancy can cause them could be of interest. Additionally, we propose a new simple heuristic algorithm to estimate the Maximum Lyapunov Exponent.

Minimizing the Average Packet Access Time of the Application Layer for Buffered Instantly Decodable Network Coding

In B-IDNC (buffered instantly decodable network coding), each receiver can cache the non-instantly decodable network coded packets (NIDPs) which contain two wanted packets of the network layer for subsequent network decoding, so that more packets can be recovered at the network layer than the traditional instantly decodable network coding (IDNC). By employing B-IDNC, we consider a radio access network wherein a base station (BS) is required to broadcast a block of packets to a set of receivers. After completing network decoding at the network layer, each receiver can deliver its recovered packets from the network layer to the application layer in order. We consider minimizing the average packet access time of the application layer for B-IDNC. For the optimization problem is intractable, we approximate it to reduce the sum minimum access delay of the application layer across all receivers. The approximate problem is shown to be equivalent to a maximum weight encoding clique problem over the B-IDNC graph. We propose a simple heuristic algorithm based on greedy maximum weight vertex search to solve the approximate problem. Simulation results verify the effectiveness of our proposed algorithm as compared with the existing techniques.

Modeling the competition between multiple Automated Mobility on-Demand operators: An agent-based approach

Automated Mobility-on-Demand (AMoD) systems, in which fleets of automated vehicles provide on-demand services, are expected to transform urban mobility systems. Motivated by the rapid development of AMoD services delivered by self-driving car companies, an agent-based model (ABM) has been developed to study the coexistence phenomena of multiple AMoD operators competing for customers. The ABM is used to investigate how changes in pricing strategies, assignment methods, and fleet sizes affect travelers’ choice of different AMoD services and the operating performance of competing operators in the case-study city of The Hague, in the Netherlands. Findings suggest that an optimal assignment algorithm can reduce the average waiting time by up to 24% compared to a simple heuristic algorithm. We also find that a larger fleet could increase demand but lead to higher waiting times for its users and higher travel times for competing operators’ users due to the added congestion. Notably, pricing strategies can significantly affect travelers’ choice of AMoD services, but the effect depends strongly on the time of the day. Low-priced AMoD services can provide high service levels and effectively attract more demand, with up to 64.7% of customers choosing the very early morning service [5:30 AM,7:20 AM]. In the subsequent morning hours, high-priced AMoD services are more competitive in attracting customers as more idle vehicles are available. Based on the quantitative analysis, policies are recommended for the government and service operators.

Multi-strategy Jaya algorithm for industrial optimization tasks

Jaya, a simple heuristic algorithm, has shown attractive features, especially parameter-free. However, the simple structure of Jaya algorithm may result in poor performances, to boost the performance, a multi-strategy Jaya (MJaya) algorithm based on multi-population has been proposed in this paper. Three strategies correspond to three groups of solutions. The first strategy based on the first population is to introduce an adaptive weight parameter to the position-updating equation to improve the local search. The second strategy is based on rank-based mutation to enhance the global search. The third strategy is to exploit around the best solution to reinforce the local search. Three strategies cooperate well during the evolution process. The experimental results based on CEC 2014 have proven that the proposed MJaya is superior compared with Jaya and its latest variants. Then, the proposed MJaya algorithm is used to solve three industrial problems and the results have shown that the proposed MJaya algorithm can also solve complex industrial applications effectively.

Efficient and Simple Heuristic Algorithm for Portfolio Optimization

Markowitz model considers what is termed as standard portfolio optimization. The portfolio optimization problem is a problem which based on asset allocation and diversification for maximum return with minimum risk. Thus, the standard portfolio optimization problem happens when the constraints considered are budget and no-short selling. In reality however, portfolio optimization has realistic constraints to be incorporated such as holding sizes, cardinality and transaction cost. When realistic constraints are added into portfolio optimization problem, it becomes too complex to be solved by standard optimization methods which in this case turns to be an extended portfolio optimization problem. Markowitz solution and the standard methods like quadratic programming become inapplicable. With such limitation, heuristic methods are usually used to deal with this extended portfolio optimization problem. Therefore, this paper proposes a heuristic algorithm for the extended portfolio optimization problem. It is a hill climbing algorithm named Hill Climbing Simple (HC-S) which is then validated by solving the standard Markowitz model. In fact, the proposed algorithm is benchmarked with the quadratic programming (QP), which is a standard method. By benchmarking HC-S with QP, it showed that HC-S can attains similar accurate solutions. Also, HC-S demonstrated to be more effective and efficient than threshold accepting (TA), an established algorithm for portfolio optimization since HC-S find solutions with significant higher objective value and require less computing time as compared to standard methods.

Modeling and solving the waste valorization production and distribution scheduling problem

Bio-based waste valorization is one of the current trends in municipal waste management. It decreases the amount of waste to be disposed of, reduces the sourcing of limited chemical compounds used in fertilizer production, and promotes a circular economy perspective vital in big cities. However, modeling and optimizing a biorefinery plant’s operations is challenging and requires innovative approaches and solutions. In this paper, we model and solve the integrated production and distribution scheduling problem faced by an industrial partner. We propose three models for the waste valorization production and distribution scheduling problem: a time-discretized integer linear program, and two mixed-integer linear program with continuous timing variables. Moreover, several powerful and problem-specific valid inequalities and variable reduction procedures are proposed. We study some variants of the problem and propose a simple heuristic algorithm that mimics the logic of a decision maker. Through a series of computational experiments, we determine how critical operational parameters affect the performance of the system and demonstrate how significant improvements can be achieved in our industrial partner’s biorefinery plant.

Parameter estimation of soil water retention curve with Rao-1 algorithm

The soil water retention curve (SWRC) has a significant role in determining the unsaturated properties of soil. A stochastic optimization algorithm named Rao-1 algorithm is employed to estimate the parameters of the SWRC model in this paper. The Rao-1 algorithm is a simple heuristic search algorithm containing only addition and multiplication operations. This paper introduces the method and its application in the determination of soil water retention model parameters in detail. In this study, van Genuchten model is used to depict the SWRC for its good fitting capacity, and parameters of the van Genuchten model are determined using Rao-1 algorithm. The feasibility and efficiency of the method are validated via the experimental results of 24 soil samples of 12 soil textural classes. Besides, the performance of Rao-1 algorithm is compared with that of Particle Swarm Optimization algorithm, Differential Evolution algorithm, and RETC program. Through comparative analysis, Rao-1 algorithm outperforms other methods in determining SWRC parameters.

Heuristic algorithms in evolutionary computation and modular organization of biological macromolecules: Applications to in vitro evolution.

Evolutionary computing (EC) is an area of computer sciences and applied mathematics covering heuristic optimization algorithms inspired by evolution in Nature. EC extensively study all the variety of methods which were originally based on the principles of selectionism. As a result, many new algorithms and approaches, significantly more efficient than classical selectionist schemes, were found. This is especially true for some families of special problems. There are strong arguments to believe that EC approaches are quite suitable for modeling and numerical analysis of those methods of synthetic biology and biotechnology that are known as in vitro evolution. Therefore, it is natural to expect that the new algorithms and approaches developed in EC can be effectively applied in experiments on the directed evolution of biological macromolecules. According to the John Holland's Schema theorem, the effective evolutionary search in genetic algorithms (GA) is provided by identifying short schemata of high fitness which in the further search recombine into the larger building blocks (BBs) with higher and higher fitness. The multimodularity of functional biological macromolecules and the preservation of already found modules in the evolutionary search have a clear analogy with the BBs in EC. It seems reasonable to try to transfer and introduce the methods of EC, preserving BBs and essentially accelerating the search, into experiments on in vitro evolution. We extend the key instrument of the Holland's theory, the Royal Roads fitness function, to problems of the in vitro evolution (Biological Royal Staircase, BioRS, functions). The specific version of BioRS developed in this publication arises from the realities of experimental evolutionary search for (DNA-) RNA-devices (aptazymes). Our numerical tests showed that for problems with the BioRS functions, simple heuristic algorithms, which turned out to be very effective for preserving BBs in GA, can be very effective in in vitro evolution approaches. We are convinced that such algorithms can be implemented in modern methods of in vitro evolution to achieve significant savings in time and resources and a significant increase in the efficiency of evolutionary search.

Scheduling with lot streaming in a two-machine re-entrant flow shop

Lot streaming is splitting a job-lot of identical items into several sublots (portions of a lot) that can be moved to the next machines upon completion so that operations on successive machines can be overlapped; hence, the overall performance of a multi-stage manufacturing environment can be improved. In this study, we consider a scheduling problem with lot streaming in a two-machine re-entrant flow shop in which each job-lot is processed first on Machine 1, then goes to Machine 2 for its second operation before it returns to the primary machine (either Machine 1 or Machine 2) for the third operation. For the two cases of the primary machine, both single-job and multi-job cases are studied independently. Optimal and near-optimal solution procedures are developed. Our objective is to minimize the makespan, which is the maximum completion time of the sublots and job lots in the single-job and multi-job cases, respectively. We prove that the single-job problem is optimally solved in polynomial-time regardless of whether the third operation is performed on Machine 1 or Machine 2. The multi-job problem is also optimally solvable in polynomial time when the third operation is performed on Machine 2. However, we prove that the multi-job problem is NP-hard when the third operation is performed on Machine 1. A global lower bound on the makespan and a simple heuristic algorithm are developed. Our computational experiment results reveal that our proposed heuristic algorithm provides optimal or near-optimal solutions in a very short time.

An adaptive shortest-solution guided decimation approach to sparse high-dimensional linear regression

High-dimensional linear regression model is the most popular statistical model for high-dimensional data, but it is quite a challenging task to achieve a sparse set of regression coefficients. In this paper, we propose a simple heuristic algorithm to construct sparse high-dimensional linear regression models, which is adapted from the shortest-solution guided decimation algorithm and is referred to as ASSD. This algorithm constructs the support of regression coefficients under the guidance of the shortest least-squares solution of the recursively decimated linear models, and it applies an early-stopping criterion and a second-stage thresholding procedure to refine this support. Our extensive numerical results demonstrate that ASSD outperforms LASSO, adaptive LASSO, vector approximate message passing, and two other representative greedy algorithms in solution accuracy and robustness. ASSD is especially suitable for linear regression problems with highly correlated measurement matrices encountered in real-world applications.

Evaluation of field visit planning heuristics during rapid needs assessment in an uncertain post-disaster environment

A Rapid Needs Assessment process is carried out immediately after the onset of a disaster to investigate the disaster’s impact on affected communities, usually through field visits. Reviewing practical humanitarian guidelines reveals that there is a great need for decision support for field visit planning in order to utilize resources more efficiently at the time of great need. Furthermore, in practice, there is a tendency to use simple methods, rather than advanced solution methodologies and software; this is due to the lack of available computational tools and resources on the ground, lack of experienced technical staff, and also the chaotic nature of the post-disaster environment. We present simple heuristic algorithms inspired by the general procedure explained in practical humanitarian guidelines for site selection and routing decisions of the assessment teams while planning and executing the field visits. By simple, we mean methods that can be implemented by practitioners in the field using primary resources such as a paper map of the area and accessible software (e.g., Microsoft Excel). We test the performance of proposed heuristic algorithms, within a simulation environment , which enables us to incorporate various uncertain aspects of the post-disaster environment in the field, ranging from travel time and community assessment time to accessibility of sites and availability of community groups. We assess the performance of proposed heuristics based on real-world data from the 2011 Van earthquake in Turkey. Our results show that selecting sites based on an approximate knowledge of community groups’ existence leads to significantly better results than selecting sites randomly. In addition, updating initial routes while receiving more information also positively affects the performance of the field visit plan and leads to higher coverage of community groups than an alternative strategy where inaccessible sites and unavailable community groups are simply skipped and the initial plan is followed. Uncertainties in travel time and community assessment time adversely affect the community group coverage. In general, the performance of more sophisticated methods requiring more information deteriorates more than the performance of simple methods when the level of uncertainty increases.

Deep Q-Network and Traffic Prediction based Routing Optimization in Software Defined Networks

Software Defined Networking (SDN) is gaining momentum not only in research but also in IT industry representing the drivers of 5G networks, due to its capabilities of increasing the flexibility of a network and address a variety of network challenges, by logically centralizing the intelligence in software-based controllers. Thanks to Machine Learning (ML) techniques, the network performances and utilization can be optimized and enhanced. Neural Networks (NN) and Reinforcement Learning (RL), in particular, have demonstrated great success in cooperating with complex problems arising in network operation and management. To this end, we exploit in this paper, an SDN-based rules placement approach that aims to dynamically predict the traffic congestion by using mainly NN and learn optimal paths and reroute traffic to improve network utilization by deploying a Deep Q-Network (DQN) agent. To this end, we first formulate the Quality-of-Service (QoS)-aware routing problem as a Linear Program (LP), whose objective is to minimize the end-to-end (E2E) delay and link utilization. Then, we propose a simple yet efficient heuristic algorithm to solve it. Numerical results through emulation using ONOS controller and Mininet demonstrate that the proposed approach can significantly improve network performances in terms of decreasing the link utilization, the packet loss and the E2E delay.

Iterated Greedy Algorithms for Flow-Shop Scheduling Problems: A Tutorial

An iterated greedy algorithm (IGA) is a simple and powerful heuristic algorithm. It is widely used to solve flow-shop scheduling problems (FSPs), an important branch of production scheduling problems. IGA was first developed to solve an FSP in 2007. Since then, various FSPs have been tackled by using IGA-based methods, including basic IGA, its variants, and hybrid algorithms with IGA integrated. Up until now, over 100 articles related to this field have been published. However, to the best of our knowledge, there is no existing tutorial or review paper of IGA. Thus, we focus on FSPs and provide a tutorial and comprehensive literature review of IGA-based methods. First, we introduce a framework of basic IGA and give an example to clearly show its procedure. To help researchers and engineers learn and apply IGA to their FSPs, we provide an open platform to collect and share related materials. Then, we make classifications of the solved FSPs according to their scheduling scenarios, objective functions, and constraints. Next, we classify and introduce the specific methods and strategies used in each phase of IGA for FSPs. Besides, we summarize IGA variants and hybrid algorithms with IGA integrated, respectively. Finally, we discuss the current IGA-based methods and already-solved FSP instances, as well as some important future research directions according to their deficiency and open issues. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Many practical scheduling problems can be transformed into flow-shop scheduling problems (FSPs), most of which are NP-hard. In order to solve them in an industrial system setting, designing effective heuristics is important and practically useful and has, thus, attracted much attention from both researchers and engineers. As an easy and high-performance heuristic, an iterated greedy algorithm (IGA) is widely used and adapted to solve numerous FSPs. Its simple framework makes it easy to be implemented by practitioners, and its high performance implies its great potential to solve industrial scheduling problems. In this work, we aim to give practitioners a comprehensive overview of IGA and help them apply IGA to solve their particular industrial scheduling problems. We review the papers that solve FSPs with IGA-based methods, including basic IGA, its variants, and hybrid algorithms with IGA integrated. First, we provide practitioners with a tutorial on IGA, where an example for solving an FSP is introduced and an open platform is constructed. The platform collects and shares the related materials, e.g., open-source code, benchmarks, and website links of important papers. Then, we introduce various FSPs and specific designs of IGA-based methods. Finally, we discuss the current research and point out future research issues.

OSCAR: On Optimizing Resource Utilization in Live Video Streaming

Live video streaming traffic and related applications have experienced significant growth in recent years. However, this has been accompanied by some challenging issues, especially in terms of resource utilization. Although IP multicasting can be recognized as an efficient mechanism to cope with these challenges, it suffers from many problems. Applying software-defined networking (SDN) and network function virtualization (NFV) technologies enable researchers to cope with IP multicasting issues in novel ways. In this article, by leveraging the SDN concept, we introduce OSCAR (Optimizing reSourCe utilizAtion in live video stReaming) as a new cost-aware video streaming approach to provide advanced video coding (AVC)-based live streaming services in the network. In this article, we use two types of virtualized network functions (VNFs): virtual reverse proxy (VRP) and virtual transcoder function (VTF). At the edge of the network, VRPs are responsible for collecting clients’ requests and sending them to an SDN controller. Then, by executing a mixed-integer linear program (MILP), the SDN controller determines a group of optimal multicast trees for streaming the requested videos from an appropriate origin server to the VRPs. Moreover, to elevate the efficiency of resource allocation and meet the given end-to-end latency threshold, OSCAR delivers only the highest requested quality from the origin server to an optimal group of VTFs over a multicast tree. The selected VTFs then transcode the received video segments and transmit them to the requesting VRPs in a multicast fashion. To mitigate the time complexity of the proposed MILP model, we present a simple and efficient heuristic algorithm that determines a near-optimal solution in polynomial time. Using the MiniNet emulator, we evaluate the performance of OSCAR in various scenarios. The results show that OSCAR surpasses other SVC- and AVC-based multicast and unicast approaches in terms of cost and resource utilization.

Generalized swarm intelligence algorithms with domain-specific heuristics

&lt;span lang="EN-US"&gt;In recent years, hybrid approaches on population-based algorithms are more often applied in industrial settings. In this paper, we present the approach of a combination of universal, problem-free Swarm Intelligence (SI) algorithms with simple deterministic domain-specific heuristic algorithms. The approach focuses on improving efficiency by sharing the advantages of domain-specific heuristic and swarm algorithms. A heuristic algorithm helps take into account the specifics of the problem and effectively translate the positions of agents (particle, ant, bee) into the problem's solution. And a Swarm algorithm provides an increase in the adaptability and efficiency of the approach due to stochastic and self-organized properties. We demonstrate this approach on two non-trivial optimization tasks: scheduling problem and finding the minimum distance between 3D isomers.&lt;/span&gt;

On a New Constraint Reduction Heuristic Using Improved Bisection Method for Mixed Integer Linear Programming

In this study, we develop a surrogate relaxation-based procedure to reduce mixed-integer linear programming (MILP) problem sizes. This technique starts with one surrogate constraint which is a nonnegative linear combination of multiple constraints of the problem. At this initial step, we calculate optimal Lagrangian multipliers from LP relaxation of the problem and use them as initial surrogate multipliers. We incorporate the improved bisection method (IBM) (B. Gavish, F. Glover, and H. Pirkul, Surrogate Constraints in Integer Programming, J. Inform. Optim. Sci. 12(2) (1991), 219-228.) into our algorithm. This simple heuristic algorithm is designed to iteratively generate a new surrogate cut that is to guarantee to satisfy the most violated two constraints of the corresponding iteration. The performance of the heuristic is tested using both some problems from the OR libraries and randomly generated ones.

Location of depots and allocation of buses to depots in urban road transport organisations: a mathematical model and greedy heuristic algorithm

Optimising the cost of operations is one of the major issues in any urban road transport organisations (URTOs). In this study, a decision problem on location of depots (adding new locations and removing existing ones) and allocation of buses to depots is considered, as observed in one of the major URTOs in India. The main focus of this research is to provide analytic methods to minimise the cost of operations comprising: 1) dead-kilometre cost; 2) fixed cost associated with introducing new depots; 3) salvage value due to closing the depots. To do so, a (0-1) mixed integer linear programming (MILP) model is proposed and its workability is demonstrated. In addition, a simple greedy heuristic algorithm is also proposed. A computational experiment is developed to understand the performance efficiency of the proposed greedy heuristic algorithm in comparison with the optimal solution. From the average and worst case analyses of the performance evaluation, it is observed that the proposed greedy heuristic algorithm provides near-optimal solution. The (0-1) MILP model and the efficient greedy heuristic algorithm proposed in this study can be used to help make better decisions on location of depots and allocation of buses to depots of URTOs in general.