Constrained Optimization for Image Reshaping With Soft Conditions

Abstract

Conventional image resizing problems demand hard conditions on size and aspect ratio, which must be met with no tolerance. In this paper, a generalized optimization framework is presented, which can handle soft conditions as well as the hard ones. The soft condition can be given by an allowable range of the image parameter, which is incorporated as an inequality condition in the constrained optimization framework. Given the soft constraints, the proposed framework seeks to find the set of image parameters that minimize the cost function. A constrained optimization via a linear programming framework is employed to manage a diverse combination of soft and hard conditions for the target image. The optimization is based on the image line, which optimally selects a set of image lines (columns and rows) to be deleted for size reduction in accordance with the cost function and the constraints. As a case study, the line-based optimal image resizing method based on the linear programming framework is applied for the pre-processing of VGG-19 convolutional neural network (CNN). Although the target input size is a hard condition of $224\times 224$ for the VGG-19 CNN, the proposed optimization framework with a soft condition on the image size firstly finds an optimal near-square image with a tradeoff against the saliency level of image features. Then, the optimal near-square image is linearly scaled to the final image size to meet the hard condition.