Knapsack Solutions Research Articles

Provably Good Solutions to the Knapsack Problem via Neural Networks of Bounded Size

The development of a satisfying and rigorous mathematical understanding of the performance of neural networks is a major challenge in artificial intelligence. Against this background, we study the expressive power of neural networks through the example of the classical NP-hard knapsack problem. Our main contribution is a class of recurrent neural networks (RNNs) with rectified linear units that are iteratively applied to each item of a knapsack instance and thereby compute optimal or provably good solution values. We show that an RNN of depth four and width depending quadratically on the profit of an optimum knapsack solution is sufficient to find optimum knapsack solutions. We also prove the following tradeoff between the size of an RNN and the quality of the computed knapsack solution: for knapsack instances consisting of n items, an RNN of depth five and width w computes a solution of value at least [Formula: see text] times the optimum solution value. Our results build on a classical dynamic programming formulation of the knapsack problem and a careful rounding of profit values that are also at the core of the well-known fully polynomial-time approximation scheme for the knapsack problem. A carefully conducted computational study qualitatively supports our theoretical size bounds. Finally, we point out that our results can be generalized to many other combinatorial optimization problems that admit dynamic programming solution methods, such as various shortest path problems, the longest common subsequence problem, and the traveling salesperson problem. History: Andrea Lodi, Area Editor for Design & Analysis of Algorithms–Discrete. An extended abstract of this article, including Figures 1 – 7 , appeared in the Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 7685–7693 ( Hertrich and Skutella 2021 ); see https://ojs.aaai.org/index.php/AAAI/article/view/16939 ; copyright © 2021, Association for the Advancement of Artificial Intelligence. Funding: This work was supported by the Deutsche Forschungsgemeinschaft [Grants DFG-GRK 2434 and EXC-2046/1, Project 390685689] and the H2020 European Research Council [ScaleOpt-757481].

An overview of analysis methods and evaluation results for caching strategies

We survey analytical methods and evaluation results for the performance assessment of caching strategies. Knapsack solutions are derived, which provide static caching bounds for independent requests and general bounds for dynamic caching under arbitrary request pattern. We summarize Markov and time-to-live-based solutions, which assume specific stochastic processes for capturing web request streams and timing. We compare the performance of caching strategies with different knowledge about the properties of data objects regarding a broad set of caching demands. The efficiency of web caching must regard benefits for network wide traffic load, energy consumption and quality-of-service aspects in a tradeoff with costs for updating and storage overheads.

Reinforcement learning using fully connected, attention, and transformer models in knapsack problem solving

SummaryKnapsack is a combinatorial optimization problem that involves a variety of resource allocation challenges. It is defined as non‐deterministic polynomial time (NP) hard and has a wide range of applications. Knapsack problem (KP) has been studied in applied mathematics and computer science for decades. Many algorithms that can be classified as exact or approximate solutions have been proposed. Under the category of exact solutions, algorithms such as branch‐and‐bound and dynamic programming and the approaches obtained by combining these algorithms can be classified. Due to the fact that exact solutions require a long processing time, many approximate methods have been introduced for knapsack solution. In this research, deep Q‐learning using models containing fully connected layers, attention, and transformer as function estimators were used to provide the solution for KP. We observed that deep Q‐networks, which continued their training by observing the reward signals provided by the knapsack environment we developed, optimized the total reward gained over time. The results showed that our approaches give near‐optimum solutions and work about 40 times faster than an exact algorithm using dynamic programming.

Provably Good Solutions to the Knapsack Problem via Neural Networks of Bounded Size

The development of a satisfying and rigorous mathematical understanding of the performance of neural networks is a major challenge in artificial intelligence. Against this background, we study the expressive power of neural networks through the example of the classical NP-hard Knapsack Problem. Our main contribution is a class of recurrent neural networks (RNNs) with rectified linear units that are iteratively applied to each item of a Knapsack instance and thereby compute optimal or provably good solution values. We show that an RNN of depth four and width depending quadratically on the profit of an optimum Knapsack solution is sufficient to find optimum Knapsack solutions. We also prove the following tradeoff between the size of an RNN and the quality of the computed Knapsack solution: for Knapsack instances consisting of n items, an RNN of depth five and width w computes a solution of value at least 1 - O(n^2 sqrt(w)) times the optimum solution value. Our results build upon a classical dynamic programming formulation of the Knapsack Problem as well as a careful rounding of profit values that are also at the core of the well-known fully polynomial-time approximation scheme for the Knapsack Problem. Finally, we point out that our results can be generalized to many other combinatorial optimization problems that admit dynamic programming solution methods, such as various Shortest Path Problems, the Longest Common Subsequence Problem, and the Traveling Salesperson Problem.

Resource-Aware Collaborative Allocation for CPU-FPGA Cloud Environments

Cloud Warehouses have been exploiting CPU-FPGA environments to accelerate multi-tenant applications to achieve scalability and maximize resource utilization. In this scenario, kernels are sent to CPU and FPGA concurrently, considering available resources and workload characteristics, which are highly variant. Therefore, we propose a multi-objective optimization strategy to improve resource provisioning in CPU-FPGA environments. It is based on the Genetic Multidimensional Knapsack solution and can be tuned to minimize makespan or energy. Our strategy provides similar results as the optimal Exhaustive Search, but with feasible execution time, while presenting 77% energy savings with 39% lower makespan than the commonly-used First-Fit strategy.

On the Stackelberg knapsack game

In this work we consider a bilevel knapsack problem, in which one player, the follower, decides on the optimal utilization of a bounded resource. The second player, the leader, can offer incentives, or shared profits, so that the follower chooses options attractive also for the leader. Formally, each of the two players is associated with a subset of the knapsack items. The leader may offer profits for its items as incentives to the follower, before the follower selects a subset of all items in order to maximize its overall profit. The leader receives as pay-off only the profits from those of its items that are included by the follower in the overall knapsack solution. This pay-off is then reduced by the profits offered to the follower. The resulting setting is a Stackelberg strategic game. The leader has to resolve the trade-off between offering high profits as incentives to the follower and offering low profits to gain high pay-offs.We analyze the problem for the case in which the follower solves the resulting knapsack problem to optimality and obtain a number of strong complexity results. Then we invoke a common assumption of the literature, namely that the follower’s computing power is bounded. Under this condition, we study several natural Greedy-type heuristics applied by the follower. The solution structure and complexity of the resulting problems are investigated and solution strategies are derived, in particular an Integer Linear Programming (ILP) model, but also pseudopolynomial and polynomial algorithms, when possible.

Test case selection-prioritization approach based on memoization dynamic programming algorithm

In the software industry, selection and prioritization techniques become a necessity in the regression and validation testing phases because a lot of test cases are available for reuse, yet time and project specific constraints must be respected. In this paper we propose a dynamic programming approach in solving test case selection-prioritization problems. We focus on low memory consumption in pseudo-polynomial time complexity applicable in both selection and selection-prioritization problems over sets of test cases or test suites. In dynamic programming optimization solutions, huge amounts of memory are required and unfortunately the memory is limited. Therefore, lower memory consumption leads to a higher number of test cases to be involved in the selection process. Our approach is suited for medium to large projects where the required memory space is not higher than the order of tens of GBytes. We employed both objective methods as the dynamic programming algorithm and subjective and empiric human decision as defining the prioritization criteria. Furthermore, we propose a method of employing multiple project specific criteria in evaluating the importance of a test case in the project context. To evaluate the proposed solution relative to the classical dynamic programming knapsack solution, we developed a suite of comparative case studies based on 1000 generated scenarios as close as possible to real project scenarios. The results of the comparative study reported the proposed algorithm requires up to 400 times less memory in the best-case scenarios and about 40 times less memory in average. The solution delivers optimal results in pseudo-polynomial time complexity, is effective for amounts of test cases up to the order of millions and compared with the classical dynamic programming methods leads to higher number of test cases to be involved in the selection process due to reduced memory consumption.

Faster FPTASes for counting and random generation of Knapsack solutions

In the #P-complete problem of counting 0/1 Knapsack solutions, the input consists of a sequence of n nonnegative integer weights w1,…,wn and an integer C, and we have to find the number of subsequences (subsets of indices) with total weight at most C. We give faster and simpler fully polynomial-time approximation schemes (FPTASes) for this problem, and for its random generation counterpart. Our method is based on dynamic programming and discretization of large numbers through floating-point arithmetic. We improve both deterministic counting FPTASes from Gopalan et al. (2011) [9], Štefankovič et al. (2012) [6] and the randomized counting and random generation algorithms in Dyer (2003) [5].Our method is general, and it can be directly applied on top of combinatorial decompositions (such as dynamic programming solutions) of various problems. For example, we also improve the complexity of the problem of counting 0/1 Knapsack solutions in an arc-weighted DAG.

An FPTAS for the knapsack problem with parametric weights

In this paper, we investigate the parametric weight knapsack problem, in which the item weights are affine functions of the form wi(λ)=ai+λ⋅bi for i∈{1,…,n} depending on a real-valued parameter λ. The aim is to provide a solution for all values of the parameter. It is well-known that any exact algorithm for the problem may need to output an exponential number of knapsack solutions. We present the first fully polynomial-time approximation schemes (FPTASs) for the problem that, for any desired precision ε∈(0,1), compute (1−ε)-approximate solutions for all values of the parameter. Among others, we present a strongly polynomial FPTAS running in On5ε2⋅log3nεlogn time and a weakly polynomial FPTAS with a running time of On3ε2⋅log2nε⋅logP⋅logMlogn, where P is an upper bound on the optimal profit and M≔max{|W|,n⋅max{|ai|,|bi|:i∈{1,…,n}}} for a knapsack with capacity W.

Combining dynamic programming with filtering to solve a four-stage two-dimensional guillotine-cut bounded knapsack problem

The two-dimensional knapsack problem consists in packing a set of small rectangular items into a given large rectangle while maximizing the total reward associated with selected items. We restrict our attention to packings that emanate from a k-stage guillotine-cut process. We introduce a generic model where a knapsack solution is represented by a flow in a directed acyclic hypergraph. This hypergraph model derives from a forward labelling dynamic programming recursion that enumerates all non-dominated feasible cutting patterns. To reduce the hypergraph size, we make use of further dominance rules and a filtering procedure based on Lagrangian reduced costs fixing of hyperarcs. Our hypergraph model is (incrementally) extended to account for explicit bounds on the number of copies of each item. Our exact forward labelling algorithm is used to solve the max-cost flow model in the base hyper-graph with side constraints to model production bounds. Results of numerical comparison against existing approaches are reported on instances from the literature and on datasets derived from a real-world application.

An FPTAS for the parametric knapsack problem

In this paper, we investigate the parametric knapsack problem, in which the item profits are affine functions depending on a real-valued parameter. The aim is to provide a solution for all values of the parameter. It is well-known that any exact algorithm for the problem may need to output an exponential number of knapsack solutions.We present a fully polynomial-time approximation scheme (FPTAS) for the problem that, for any desired precision ε∈(0,1), computes (1−ε)-approximate solutions for all values of the parameter. This is the first FPTAS for the parametric knapsack problem that does not require the slopes and intercepts of the affine functions to be non-negative but works for arbitrary integral values. Our FPTAS outputs O(n2ε) knapsack solutions and runs in strongly polynomial-time O(n2ε⋅A(n,ε)), where A(n,ε) denotes the running time of any FPTAS for the traditional (non-parametric) knapsack problem. Even for the special case of positive input data, this is the first FPTAS with a strongly polynomial running time.

Randomized strategies for cardinality robustness in the knapsack problem

We consider the following zero-sum game related to the knapsack problem. Given an instance of the knapsack problem, Alice chooses a knapsack solution and Bob, knowing Alice's solution, chooses a cardinality k. Then, Alice obtains a payoff equal to the ratio of the profit of the best k items in her solution to that of the best solution of size at most k. For α>0, a knapsack solution is called α-robust if it guarantees payoff α. If Alice adopts a deterministic strategy, the objective of Alice is to find a max-robust knapsack solution. By applying the argument in Kakimura and Makino [6] for robustness in general independence systems, a (1/μ)-robust solution exists and is found in polynomial time, where μ is the exchangeability of the independence system.In the present paper, we address randomized strategies for this zero-sum game. Randomized strategies in robust independence systems are introduced by Matuschke, Skutella, and Soto [11] and they presented a randomized strategy with (1/ln⁡4)-robustness for a certain class of independence systems. The knapsack problem, however, does not belong to this class. We first establish the intractability of the knapsack problem by showing an instance such that the robustness of an arbitrary randomized strategy is both O((log⁡log⁡μ)/log⁡μ) and O((log⁡log⁡ρ)/log⁡ρ), where ρ:=(the size of a maximum feasible set)(the size of a minimum infeasible set)−1. We then exhibit the power of randomness by designing two randomized strategies with robustness Ω(1/log⁡μ) and Ω(1/log⁡ρ), respectively, which substantially improve upon that of known deterministic strategies and almost attain the above upper bounds. It is also noteworthy that our strategy applies to not only the knapsack problem but also independence systems for which an (approximately) optimal solution under a cardinality constraint is computable.

Fast Link Scheduling Policies for Persistent Data Ferrying

The persistent data-ferrying problem has a naturally cascaded structure between vehicle and communication dynamics, allowing the wireless link scheduling optimization to be encapsulated within a path-planning optimization problem. This work derives and analyzes policies to solve the inner link scheduling problem quickly. Given a path and properties about the communication environment, the link scheduling problem is formulated as a binary integer linear program with unique ferrying constraints. The performances of several heuristic link scheduling policies are analyzed in terms of computation effort and achievable throughput of the solution. The performance characteristics are then validated through simulation, showing that the greedy knapsack solution achieves within 99% of the optimal allocation in less than 2% of the computation time compared to traditional solvers. The quickness of this policy reduces the computational complexity needed to solve the full data-ferry problem.

A deterministic fully polynomial time approximation scheme for counting integer knapsack solutions made easy

Given n elements with nonnegative integer weights w=(w1,…,wn), an integer capacity C and positive integer ranges u=(u1,…,un), we consider the counting version of the classic integer knapsack problem: find the number of distinct multisets whose weights add up to at most C. We give a deterministic algorithm that estimates the number of solutions to within relative error ϵ in time polynomial in n, log⁡U and 1/ϵ, where U=maxi⁡ui. More precisely, our algorithm runs in O(n3log2⁡Uϵlog⁡nlog⁡Uϵ) time. This is an improvement of n2 and 1/ϵ (up to log terms) over the best known deterministic algorithm by Gopalan et al. (2011) [5]. Our algorithm is relatively simple, and its analysis is rather elementary. Our results are achieved by means of a careful formulation of the problem as a dynamic program, using the notion of binding constraints.

A fully polynomial-time approximation scheme for approximating a sum of random variables

Given n independent integer-valued random variables X1,X2,…,Xn and an integer C, we study the fundamental problem of computing the probability that the sum X=X1+X2+⋯+Xn is at most C. We assume that each random variable Xi is implicitly given by an oracle Oi, which given two input integers n1,n2 returns the probability of n1≤Xi≤n2. We give the first deterministic fully polynomial-time approximation scheme (FPTAS) to estimate the probability up to a relative error of 1±ϵ. Our algorithm is based on the technique for approximately counting knapsack solutions, developed in Gopalan et al. (2011).

An employee transporting problem

Abstract An employee transporting problem is described and a set partitioning model is developed. An investigation of the model leads to a knapsack problem as a surrogate problem. Finding a partition corresponding to the knapsack problem provides a solution to the problem. An exact algorithm is proposed to obtain a partition (subset-vehicle combination) corresponding to the knapsack solution. It requires testing and matching too many alternatives to obtain a partition. The sweep algorithm is implemented in obtaining a partition (subset-vehicle combination) in an efficient manner. Illustrations are provided to show how the algorithms obtain solutions.

Multicriteria 0-1 knapsack problems with k-min objectives

This paper deals with the multicriteria 0-1 knapsack problem (KP) with k-min objectives (MkMIN-KP) in which the first objective is of classical sum type and the remaining objectives are k-min objective functions. The k-min objectives are ordinal objectives, aiming at the maximization of the k th smallest objective coefficient in any feasible knapsack solution with at least k items in the knapsack. We develop efficient algorithms for the determination of the complete nondominated set of MkMIN-KP.The MkMIN-KP can be transformed into a series of single objective 0-1 multidimensional KPs (mKPs) with cardinality constraints based on the ε-constraint method. The resulting mKPs are NP-hard. Therefore, it is necessary to solve the mKP efficiently. In this paper, we first develop different variants of hybrid algorithms for the mKP considering the properties of the MkMIN-KP problem and suggest a hybrid two stage solution approach that is the synthesis of various solution techniques such as linear programming relaxation, dynamic programming (DP), bounding techniques, state dominance relations, core concept and greedy principle.Numerical experiments with different types of MkMIN-KP instances with two k-min objectives show that the hybrid algorithm can find the complete nondominated set significantly faster with much less memory requirements than a sequential DP algorithm. We also evaluate the effect of different combinations of k values of the k-min objectives on the cardinality of the nondominated set, solution time and memory requirements of the algorithm.

Computing knapsack solutions with cardinality robustness

In this paper, we study the robustness over the cardinality variation for the knapsack problem. For the knapsack problem and a positive number α ≤ 1, we say that a feasible solution is α-robust if, for any positive integer k, it includes an α-approximation of the maximum k-knapsack solution, where a k-knapsack solution is a feasible solution that consists of at most k items. In this paper, we show that, for any $${\varepsilon >0 }$$ , the problem of deciding whether the knapsack problem admits a $${(\nu+\varepsilon)}$$ -robust solution is weakly NP-hard, where ν denotes the rank quotient of the corresponding knapsack system. Since the knapsack problem always admits a ν-robust knapsack solution, this result provides a sharp border for the complexity of the robust knapsack problem. On the positive side, we show that a max-robust knapsack solution can be computed in pseudo-polynomial time, and present a fully polynomial-time approximation scheme (FPTAS) for computing a max-robust knapsack solution.

A Deterministic Polynomial-Time Approximation Scheme for Counting Knapsack Solutions

Given n elements with nonnegative integer weights $w_1, \ldots, w_n$ and an integer capacity C, we consider the counting version of the classic knapsack problem: find the number of distinct subsets whose weights add up to at most the given capacity. We give a deterministic algorithm that estimates the number of solutions to within relative error $1\pm\varepsilon$ in time polynomial in n and $1/\varepsilon$ (fully polynomial approximation scheme). More precisely, our algorithm takes time $O(n^3\varepsilon^{-1}\log(n/\varepsilon))$. Our algorithm is based on dynamic programming. Previously, randomized polynomial-time approximation schemes were known first by Morris and Sinclair via Markov chain Monte Carlo techniques and subsequently by Dyer via dynamic programming and rejection sampling.

Stochastic Resource Allocation for Energy-Constrained Systems

Battery-powered wireless systems running media applications have tight constraints on energy, CPU, and network capacity, and therefore require the careful allocation of these limited resources to maximize the system's performance while avoiding resource overruns. Usually, resource-allocation problems are solved using standard knapsack-solving techniques. However, when allocating conservable resources like energy (which unlike CPU and network remain available for later use if they are not used immediately) knapsack solutions suffer from excessive computational complexity, leading to the use of suboptimal heuristics. We show that use of Lagrangian optimization provides a fast, elegant, and, for convex problems, optimal solution to the allocation of energy across applications as they enter and leave the system, even if the exact sequence and timing of their entrances and exits is not known. This permits significant increases in achieved utility compared to heuristics in common use. As our framework requires only a stochastic description of future workloads, and not a full schedule, we also significantly expand the scope of systems that can be optimized.