Heterogeneous paper documents (such as newspaper, magazine) are very common in our daily life. They are usually scanned and stored as images. Reading such images on a mobile device is very awkward, as they can only be partially displayed to ensure readability. The user needs to frequently switch among different portions of the image to read clearly. It would be very helpful if the system can automatically determine an appropriate reading area around the user’s click position and retarget the area to the whole screen. In this paper, we propose a localized layout analysis method for retargeting of heterogeneous images. Once the user clicks on a fully displayed heterogeneous image, our method can automatically extract an appropriate rectangular region and scale the region to the whole screen for reading. The region is semantically meaningful, and the content is guaranteed to be clear enough when fully displayed on the screen. The experimental results show that our method can effectively avoid those tedious scale and translation operations when reading heterogeneous images, and thus improve the user’s experience greatly.