Local Search Component Research Articles

GSK-LocS: Towards a more effective generalisation in population-based neural network training

Despite the effectiveness of deep neural networks, feed-forward neural networks (FFNNs) continue to play a crucial role in many applications, especially when dealing with limited data availability. The primary challenge in FFNNs is determining the optimal weights during the training process, aiming to minimise the disparity between actual and predicted outputs. Although gradient-based techniques like backpropagation (BP) have traditionally been popular for FFNN training, they come with inherent limitations, such as sensitivity to initial weights and susceptibility to getting trapped in local optima. To overcome these challenges, we introduce a novel approach based on the Gaining-Sharing Knowledge-based(GSK) algorithm. To the best of our knowledge, this paper represents the first exploration of GSK for neural network training. After obtaining the appropriate weights for the FFNN by the GSK, the weights and biases are utilised to initialise a Levenberg–Marquardt backpropagation (LMBP) algorithm, serving as a local search component. In other words, our proposed algorithm, GSK-LocS, leverages the global search capabilities of the GSK algorithm and combines them with the local search capabilities of LMBP for neural network training. This integration mitigates sensitivity to initial values and reduces the risk of being trapped in local optima. Experimental results conducted on classification and approximation problems provide compelling evidence that our proposed algorithm is highly competitive compared to other existing methods.

A parallel branch-and-cut and an adaptive metaheuristic to solve the Family Traveling Salesman Problem

This paper addresses the Family Traveling Salesman Problem (FTSP), a variant of the Traveling Salesman Problem that group nodes into families. The goal is to select the best route by visiting only a subset of nodes from each family. We developed two methods to solve the FTSP: (i) a parallel branch-and-cut algorithm with an efficient local search procedure (P-B&C) to obtain an optimal solution, and (ii) an adaptive metaheuristic that combines the Biased Random-key Genetic Algorithm (BRKGA) with a reinforcement learning algorithm. In this case, the Q-Learning algorithm controls the parameters of the BRKGA during the evolutionary process. We perform computational experiments on a well-known benchmark dataset with 185 instances. Our P-B&C proves the optimal value for 179 instances, improving the best upper bounds in 19 open instances. The new local search component of the P-B&C finds the best upper bounds for 50% of instances. The BRKGA-QL finds the optimal solution in 131 instances, improving the best upper bounds in 21 open instances. Finally, we compare our results with the best results in the literature, and both methods show robustness and efficiency in solving the FTSP.

Multi-league sports scheduling with different leagues sizes

This paper introduces a general multi-league sports scheduling problem where timetables for multiple leagues must be determined simultaneously, a practical and challenging problem in amateur and youth sports. We consider round robin leagues with different numbers of teams, and hence also requiring different numbers of rounds. As the number of simultaneous home games that clubs can organize for their teams is limited by the capacity of their venue, the objective is to minimize capacity violations. Along with a mixed integer programming model which is formulated to optimize the starting round of each league as well as to settle when teams have their home games, we develop various methods to construct an initial solution, and a heuristic with several local search and perturbation components to improve on this. Extensive computational experiments reveal that our heuristic can efficiently provide high-quality solutions for artificial and realistic instances. We also illustrate the impact of using different sets of home-away patterns on the total venue capacity violations. Results on a real-life application from the Belgian national football association indicate that schedules based on our heuristic allow teams to substantially decrease their venue capacity without causing meaningfully more violations.

A hybrid binary dwarf mongoose optimization algorithm with simulated annealing for feature selection on high dimensional multi-class datasets

The dwarf mongoose optimization (DMO) algorithm developed in 2022 was applied to solve continuous mechanical engineering design problems with a considerable balance of the exploration and exploitation phases as a metaheuristic approach. Still, the DMO is restricted in its exploitation phase, somewhat hindering the algorithm's optimal performance. In this paper, we proposed a new hybrid method called the BDMSAO, which combines the binary variants of the DMO (or BDMO) and simulated annealing (SA) algorithm. In the modelling and implementation of the hybrid BDMSAO algorithm, the BDMO is employed and used as the global search method and the simulated annealing (SA) as the local search component to enhance the limited exploitative mechanism of the BDMO. The new hybrid algorithm was evaluated using eighteen (18) UCI machine learning datasets of low and medium dimensions. The BDMSAO was also tested using three high-dimensional medical datasets to assess its robustness. The results showed the efficacy of the BDMSAO in solving challenging feature selection problems on varying datasets dimensions and its outperformance over ten other methods in the study. Specifically, the BDMSAO achieved an overall result of 61.11% in producing the highest classification accuracy possible and getting 100% accuracy on 9 of 18 datasets. It also yielded the maximum accuracy obtainable on the three high-dimensional datasets utilized while achieving competitive performance regarding the number of features selected.

An enhanced GRASP approach for the index tracking problem

AbstractIndex tracking models build portfolios of limited size that replicate the performance of a market index. As the size of the index grows, it becomes impractical to find an optimal solution. As far as we know, this work proposes the first greedy randomized adaptive search procedure (GRASP) approach for index tracking. GRASP has proven to be efficient in combinatorial optimization problems and offers a solution construction procedure different from the standard index tracking optimization approaches, bringing a new perspective to the field. Results showed that the proposed GRASP approach found solutions with almost the same quality as those found by CPLEX solver in a smaller time, and the proposed local search component was competitive depending on the problem parametrization. The results found have practical implications concerning the achievement of good solutions by GRASP approaches in a smaller time when compared with hybrid genetic algorithms, and new perspectives for building GRASP heuristics for portfolio optimization problems.

The consistent production routing problem

AbstractThis article introduces the consistent production routing problem in a setting with multiple plants and products. The problem consists in finding minimum‐cost production‐routing plans that also meet specific consistency requirements. In our context, consistency is defined as the degree to which some specified features of the solution remain invariant over time. We consider four forms of consistency, namely: driver, source, product, and plant consistency. For each of these consistency requirements, there is a target maximum value defining the decision‐maker's tolerance to deviations from a perfectly consistent solution. These targets are enforced as soft constraints whose violations need to be minimized when optimizing the integrated production and routing plan. We present a mathematical formulation for the problem and an exact branch‐and‐cut algorithm, enhanced with valid inequalities and specific branching priorities. We also propose a heuristic solution method based on iterated local search and several mathematical programming components. Experiments on a large benchmark set of newly introduced instances show that the enhancements substantially improve the performance of the exact algorithm and that the heuristic method performs robustly for production routing problems with different consistency requirements as well as for standard versions of the problem. We also analyze the cost‐consistency trade‐off of the solutions, confirming that it is possible to impose consistency without excessively increasing the cost. The results also reveal the impact of the first time period when optimizing and measuring the consistency features we study.

Many-Objective Container Stowage Optimization Based on Improved NSGA-III

The container ship stowage planning problem (CSPP) is a very complex and challenging issue concerning the interests of shipping companies and ports. This article has developed a many-objective CSPP solution that optimizes ship stability and reduces the number of shifts over the whole route while at the same time considering realistic constraints such as the physical structure of the ship and the layout of the container yard. Use the initial metacentric height (GM) along with the ship’s heeling angle and trim to measure its stability. Meanwhile, use the total amount of relocation in the container terminal yard, the voluntary shift in the container ship’s bay, and the necessary shift of the future unloading port to measure the number of shifts on the whole route. This article proposes a variant of the nondominated sorting genetic algorithm III (NSGA-III) combined with local search components to solve this problem. The algorithm can produce a set of non-dominated solutions, then decision-makers can choose the best practical implementation based on their experience and preferences. After carrying out a large number of experiments on 48 examples, our calculation results show that the algorithm is effective compared with NSGA-II and random weighted genetic algorithms, especially when applied to solve many-objective CSPPs.

Enhanced shuffled frog leaping algorithm with improved local exploration and energy-biased load reduction phase for load balancing of gateways in WSNs

AbstractWireless sensor networks (WSNs) have grown widely due to their application in various domains, such as surveillance, healthcare, telecommunication, etc. In WSNs, there is a necessity to design energy-efficient algorithms for different purposes. Load balancing of gateways in cluster-based WSNs is necessary to maximize the lifetime of a network. Shuffled frog leaping algorithm (SFLA) is a popular heuristic algorithm that incorporates a deterministic approach. Performance of any heuristic algorithm depends on its exploration and exploitation capability. The main contribution of this article is an enhanced SFLA with improved local search capability. Three strategies are tested to enhance the local search capability of SFLA to improve the load balancing of gateways in WSNs. The first proposed approach is deterministic in which the participation of the global best solution in information exchange is increased. The next two variations reduces the deterministic approach in the local search component of SFLA by introducing probability-based selection of frogs for information exchange. All three strategies improved the success of local search. Second contribution of article is increased lifetime of gateways in WSNs with a novel energy-biased load reduction phase introduced after the information exchange step. The proposed algorithm is tested with 15 datasets of varying areas of deployment, number of sensors and number of gateways. Proposed ESFLA-RW variation shows significant improvement over other variations in terms of successful local explorations, best fitness values, average fitness values and convergence rate for all datasets. Obtained results of proposed ESFLA-RW are significantly better in terms of network energy consumption, load balancing, first gateway die and network life. The proposed variations are tested to check the effect of various algorithm-specific parameters namely frog population size, probability of information exchange and probability of energy-biased load reduction phase. Higher population size and probabilities give better solutions and convergence rate.

Adaptive large neighborhood search for vehicle routing problems with transshipment facilities arising in city logistics

In this paper, we investigate vehicle routing problems with third-party transshipment facilities that arise in the context of city logistics. Contrary to classical vehicle routing problems, where each customer request is delivered directly to its destination, the problems considered in this paper feature the alternative possibility of delivering customer requests to third-party transshipment facilities, such as urban consolidation centers, for a fee. We present an adaptive large neighborhood search with an embedded random variable neighborhood descent as a local search component and a set-partitioning problem for the recombination of routes to solve various versions of the problem. Thereby, we consider location-dependent time windows as well as heterogeneous fleets and propose several new procedures that consider transshipment facilities within the components of our adaptive large neighborhood search. The proposed method is tested on benchmark instances from the literature as well as newly created benchmark instances. It shows promising results, leading to multiple improvements over existing algorithms from the literature. Moreover, a real-world study is presented to gain managerial insights on the impact of transshipment fees, order size, and heterogeneous fleets on the transshipment decisions.

A location-allocation-improvement heuristic for districting with multiple-activity balancing constraints and p-median-based dispersion minimization

A districting problem consisting of the minimization of a dispersion function subject to multiple-activity balancing and contiguity constraints is addressed. This problem arises from a real-world application in a commercial firm responsible for distributing bottled beverage products. The objective is to find a partition of a set of city blocks into a given number of territories that minimizes a p-median problem-based dispersion function. Minimizing this measure allows for territory compactness. Districts must be connected and balanced with respect to both activities: number of customers and workload. A heuristic procedure based on a location-allocation scheme is proposed for this NP-hard problem. This technique has been successfully used for similar territory design problems with single-activity balancing constraints. The proposed method is an iterative algorithm that consists of two phases: first centroids are determined or fixed (location phase) and then units are assigned to centroids (allocation phase). The success of this technique relies on some theoretical properties that no longer hold for problems having multiple-activity balancing constraints. The novelty of our work is to design a framework for handling both connectivity and multiple-activity balancing constraints simultaneously. The solution framework is enhanced by a local search phase that attempts to improve the quality of solutions found in the allocation phase. The empirical work includes a thorough study of the different components of the algorithm, solution of very large scale instances, and comparison with existing work. Extensive empirical testing reveals the effectiveness of the proposed approach, including the impact of both the location-allocation component and its local search component. It is based on the intelligent exploitation of problem structure. The algorithm significantly outperforms one of the best-known heuristics for this problem in terms of feasibility success and solution quality. It obtains similar quality solutions than the ones obtained by the other known heuristic in less computational effort.

Iterated local search with tabu search for the weighted vertex coloring problem

This paper proposes an iterated local search (ILS) based heuristic for the weighted vertex coloring problem (WVCP). Given a graph G(V,E) with a weight w(v) associated with each vertex v∈V, the WVCP asks to find a coloring {V1,…,Vk} of G that minimizes ∑i=1kmaxv∈Viw(v). This problem has many theoretical and practical applications, such as batch scheduling, buffer minimization, and traffic assignment in telecommunication. Our ILS heuristic relies on two new neighborhood structures for the problem, and its local search component is hybridized with a tabu search strategy. We compare our approach with state-of-the-art heuristics and exact methods for the problem. Experimental results on well-known benchmark instances demonstrate that, first, our heuristic is better than the other heuristics in both solution quality and computational time, and, second, it is a good alternative for large instances that cannot be solved by exact methods.

Stochastic optimization with adaptive restart: a framework for integrated local and global learning

A common approach to global optimization is to combine local optimization methods with random restarts. Restarts have been used as a performance boosting approach. They can be a means to avoid “slow progress” by exploiting a potentially good solution, and restarts can enable the potential discovery of multiple local solutions, thus improving the overall quality of the returned solution. A multi-start method is a way to integrate local and global approaches; where the global search itself can be used to restart a local search. Bayesian optimization methods aim to find global optima of functions that can only be point-wise evaluated by means of a possibly expensive oracle. We propose the stochastic optimization with adaptive restart (SOAR) framework, that uses the predictive capability of Gaussian process models as a means to adaptively restart local search and intelligently select restart locations with current information. This approach attempts to balance exploitation with exploration of the solution space. We study the asymptotic convergence of SOAR to a global optimum, and empirically evaluate SOAR performance through a specific implementation that uses the Trust Region method as the local search component. Numerical experiments show that the proposed algorithm outperforms existing methodologies over a suite of test problems of varying problem dimension with a finite budget of function evaluations.

Cooperative route planning for the drone and truck in delivery services: A bi-objective optimisation approach

The deployment of drones to support the last-mile delivery has been initially attempted by several companies such as Amazon and Alibaba. The complementary capabilities of the drone and the truck pose an innovative delivery mode. The relevant optimisation problem associated with this new mode, known as the travelling salesman problem with drone (TSP-D), aims to find the coordinated routes of a drone and a truck to serve a list of customers. In practice, managers sometimes intend to attain a compromise between operational cost and completion time. Therefore, this article addresses a bi-objective TSP-D considering both objectives. An improved non-dominated sorting genetic algorithm (INSGA-II) is proposed to solve the problem. Specifically, the label algorithm-based decoding method, the fast non-dominated sorting approach, the crowding-distance computation procedure, and the local search component are devised to accommodate the features of the problem. Furthermore, the first Pareto front obtained by the INSGA-II is improved by a post-optimisation component. Computational results validate the competitive performance of the proposed algorithm. Meanwhile, the trade-off analysis demonstrates the relationship between operational cost and completion time and provides managerial insights for managers designing reasonable compromise routes.

Prediction of mechanical properties of micro-alloyed steels via neural networks learned by water wave optimization

Searching optimal parameters for neural networks can be formulated as a multi-modal optimization problem. This paper proposes a novel water wave optimization (WWO)-based memetic algorithm to identify the optimal weights for neural networks. In the proposed water wave optimization-based memetic algorithm (WWOMA), we employ WWO to perform global search by both individual improvement and population co-evolution and then employ several local search components to enhance its local refinement ability. Moreover, an effective Meta-Lamarckian learning strategy is utilized to choose a proper local search component to concentrate computational efforts on more promising solutions. We carry out simulation experiments on six well-known neural network designing benchmark problems, both the simulation results and statistical comparisons demonstrate the feasibility, effectiveness and efficiency of applying WWOMA to design neural networks. Furthermore, we apply WWOMA to design neural networks and use well-trained neural networks to predict tensile strength of micro-alloyed steels. Evaluation on a practical industrial case with 2489 sample data shows that, in comparison with other algorithms, WWOMA-based neural networks can obtain notable and robust prediction accuracy, which further demonstrates that WWOMA is a promising and efficient algorithm for designing neural networks. It is worth mentioning that, to the best of our knowledge, this is the first report about applying water wave optimization to train neural networks.

An iterated greedy algorithm with variable neighborhood descent for the planning of specialized diagnostic services in a segmented healthcare system

In this paper, a problem arising in the planning of specialized diagnostic services in a segmented public healthcare system is addressed. The problem consists of deciding which hospitals will provide the service and their capacity levels, the allocation of demand in each institution, and the reallocation of uncovered demand to other institutions or private providers, while minimizing the total equivalent annual cost of investment and operating cost required to satisfy all the demand. An associated mixed-integer linear programming model can be solved by conventional branch and bound for relatively small instances; however, for larger instances the problem becomes intractable. To effectively address larger instances, a hybrid metaheuristic framework combining iterated greedy (IGA) and variable neighborhood descent (VND) components for this problem is proposed. Two greedy construction heuristics are developed, one starting with an infeasible solution and iteratively adding capacity and the other starting with a feasible, but expensive, solution and iteratively decrease capacity. The iterated greedy algorithm includes destruction and reconstruction procedures. Four different neighborhood structures are designed and tested within a VND procedure. In addition, the computation of local search components benefit from an intelligent exploitation of problem structure since, when the first-level location variables (hospital location and capacity) are fixed, the remaining subproblem can be solved efficiently as it is very close to a transshipment problem. All components and different strategies were empirically assessed both individually and within the IGA-VND framework. The resulting metaheuristic is able to obtain near optimal solutions, within 3% of optimality, when tested over a data base of 60- to 300-hospital instances.

An adaptive memetic algorithm for feature selection using proximity graphs

AbstractWe propose a multivariate feature selection method that uses proximity graphs for assessing the quality of feature subsets. Initially, a complete graph is built, where nodes are the samples, and edge weights are calculated considering only the selected features. Next, a proximity graph is constructed on the basis of these weights and different fitness functions, calculated over the proximity graph, to evaluate the quality of the selected feature set. We propose an iterative methodology on the basis of a memetic algorithm for exploring the space of possible feature subsets aimed at maximizing a quality score. We designed multiple local search strategies, and we used an adaptive strategy for automatic balancing between the global and local search components of the memetic algorithm. The computational experiments were carried out using four well‐known data sets. We investigate the suitability of three different proximity graphs (minimum spanning tree, k‐nearest neighbors, and relative neighborhood graph) for the proposed approach. The selected features have been evaluated using a total of 49 classification methods from an open‐source data mining and machine learning package (WEKA). The computational results show that the proposed adaptive memetic algorithm can perform better than traditional genetic algorithms in finding more useful feature sets. Finally, we establish the competitiveness of our approach by comparing it with other well‐known feature selection methods.

A Cognitively Inspired Hybridization of Artificial Bee Colony and Dragonfly Algorithms for Training Multi-layer Perceptrons

The objective of this article is twofold. On the one hand, we introduce a cognitively inspired hybridization metaheuristic that combines the strengths of two existing metaheuristics: the artificial bee colony (ABC) algorithm and the dragonfly algorithm (DA). The aim of this hybridization is to reduce the problems of slow convergence and trapping into local optima, by striking a good balance between global and local search components of the constituent algorithms. On the other hand, we use the proposed metaheuristic to train a multi-layer perceptron (MLP) as an alternative to existing traditional- and metaheuristic-based learning algorithms; this is for the purpose of improving overall accuracy by optimizing the set of MLP weights and biases. The proposed hybrid ABC/DA (HAD) algorithm comprises three main components: the static and dynamic swarming behavior phase in DA and two global search phases in ABC. The first one performs global search (DA phase), the second one performs local search (onlooker phase), and the third component implements global search (modified scout bee phase). The resultant metaheuristic optimizer is employed to train an MLP to reach a set of weights and biases that can yield high performance compared to traditional learning algorithms or even other metaheuristic optimizers. The proposed algorithm was first evaluated using 33 benchmark functions to test its performance in numerical optimization problems. Later, using HAD for training MLPs was evaluated against six standard classification datasets. In both cases, the performance of HAD was compared with the performance of several new and old metaheuristic methods from swarm intelligence and evolutionary computing. Experimental results show that HAD algorithm is clearly superior to the standard ABC and DA algorithms, as well as to other well-known algorithms, in terms of achieving the best optimal value, convergence speed, avoiding local minima and accuracy of trained MLPs. The proposed algorithm is a promising metaheuristic technique for general numerical optimization and for training MLPs. Specific applications and use cases are yet to be explored fully but they are supported by the encouraging results in this study.

MPGA: an evolutionary state assignment for dynamic and leakage power reduction in FSM synthesis

As the development of deep-submicron and nano-technology, leakage power minimisation becomes as important as dynamic power reduction in IC design. In order to achieve low-power state assignment for finite-state machine (FSM) synthesis, a multi-population genetic algorithm (MPGA)-based state assignment method is proposed. MPGA consists of an outer-loop and a set of inner-GAs. In MPGA, inner-GA is a local search component for finding low-power state assignment. Selection, crossover and mutation are used to perform variations on individuals. Cost function is defined based on power dissipation formulation of complementary metal oxide semiconductor (CMOS) gate for dynamic power and leakage power estimation. The outer-loop is used to optimise the parameters of inner-genetic algorithm (GA) through population variation schema, intra-specific competition and newborn. Twenty-three FSMs that were commonly used as benchmarks are employed to test the effectiveness of MPGA and compare different state assignment methods. Experimental results show MPGA achieves a significant improvement over the previous publications both on dynamic power and leakage power reduction in most benchmarks.

Large neighbourhood search with adaptive guided ejection search for the pickup and delivery problem with time windows

An effective and fast hybrid metaheuristic is proposed for solving the pickup and delivery problem with time windows. The proposed approach combines local search, large neighbourhood search and guided ejection search in a novel way to exploit the benefits of each method. The local search component uses a novel neighbourhood operator. A streamlined implementation of large neighbourhood search is used to achieve an effective balance between intensification and diversification. The adaptive ejection chain component perturbs the solution and uses increased or decreased computation time according to the progress of the search. While the local search and large neighbourhood search focus on minimising travel distance, the adaptive ejection chain seeks to reduce the number of routes. The proposed algorithm design results in an effective and fast solution method that finds a large number of new best-known solutions on a well-known benchmark dataset. Experiments are also performed to analyse the benefits of the components and heuristics and their combined use to achieve a better understanding of how to better tackle the subject problem.

An Adaptive Large Neighborhood Search for the Location-routing Problem with Intra-route Facilities

Recent research on location-routing problems has been focusing on locating facilities as the starting and end point of routes. In this paper, we investigate a new type of location-routing problem. In the location-routing problem with intra-route facilities, the location of depots is known, whereas the location of facilities for intermediate stops has to be determined to keep vehicles operational. We present an adaptive large neighborhood search which is enhanced by local search and dynamic programming components, and derive new penalty functions for time-efficient neighborhood evaluation. We show that this algorithm is suitable for solving various problems with intra-route facilities by deriving new best known solutions for the recently published electric location-routing problem with time windows and partial recharging, as well as for the battery swap station electric vehicle location-routing problem. Additionally, we create new real-world benchmark instances and show results as well. Furthermore, we assess the competitiveness of our algorithm on the electric vehicle routing problem with time windows for full and partial recharging, and derive new best known solutions for both problem variants. The online appendix is available at https://doi.org/10.1287/trsc.2017.0746 .