We propose a novel method of arbitrarily focused image generation using multiple differently focused images. First, we describe our previously proposed select and merge method for all focused image acquisition. We can get good results by using this method but it is not easy to extend this method for generating arbitrarily focused images. Then, based on the assumption that depth of a scene changes stepwise, we derive a formula for reconstruction between the desired arbitrarily focused image and multiple acquired images; we can reconstruct the arbitrarily focused image by iterative use of the formula. We also introduce coarse-to-fine estimation of point spread functions (PSFs) of the acquired images. We reconstruct arbitrarily focused images for a natural scene. In other words, we simulate virtual cameras and generate images focused on arbitrary depths. © 1998 SPIE and IS&T. [S1017-9909(98)02201-6]