Image Composition Method Research Articles

High-resolution cropland mapping in China’s Huang-Huai-Hai Plain: The coupling of machine learning methods and prior information

Numerous map products containing cropland data are currently available. However, due to their low temporal and spatial resolutions and limited accuracy, their practical applications are limited. Thus, research on the precise mapping of croplands with high temporal and spatial resolutions is still urgently needed. The Huang-Huai-Hai Plain, a critical grain production area in China, was used as the research area to produce high-resolution annual cropland maps from 2018 to 2022. Sentinel-2 imagery, digital elevation model (DEM) data, and China Agricultural Ecological Zoning (CAEZ) data were collected. Based on these data, the study area was stratified into different zones, and then the distinct characteristics and optimal timing for differentiating cropland from other land cover types within each zone were analyzed and used as prior information for guiding time-series image compositing. Furthermore, the spectral information divergence (SID) index was used to modify the existing sample migration technique to transfer training samples across years. Via the Google Earth Engine (GEE) platform, the random forest (RF) classification method and positive and unlabeled learning (PUL) method were combined to construct cropland maps from composited images and terrain information from DEMs. Compared with the seasonal or spectral-temporal metric (STM) image composition methods, the image composition method based on prior information significantly improved the distinction between cropland and other land cover types. Moreover, the newly proposed sample migration strategy substantially increased the accuracy of training samples transferred across years. Compared with existing cropland maps from products such as the ESA, ESRI, GlobeLand30, GLAD, and CLCD, the cropland maps generated in this study demonstrated superior accuracy, with average overall accuracy (OA) values from 88.81% to 91.66% for different zones, thus outperforming other products with OA values from 77.53% to 88.90%. This study supports the development of related agricultural and ecological research and provides a valuable reference for cropland mapping.

A Multi-Shot Approach for Spatial Resolution Improvement of Multispectral Images from an MSFA Sensor.

Multispectral imaging technology has advanced significantly in recent years, allowing single-sensor cameras with multispectral filter arrays to be used in new scene acquisition applications. Our camera, developed as part of the European CAVIAR project, uses an eight-band MSFA to produce mosaic images that can be decomposed into eight sparse images. These sparse images contain only pixels with similar spectral properties and null pixels. A demosaicing process is then applied to obtain fully defined images. However, this process faces several challenges in rendering fine details, abrupt transitions, and textured regions due to the large number of null pixels in the sparse images. Therefore, we propose a sparse image composition method to overcome these challenges by reducing the number of null pixels in the sparse images. To achieve this, we increase the number of snapshots by simultaneously introducing a spatial displacement of the sensor by one to three pixels on the horizontal and/or vertical axes. The set of snapshots acquired provides a multitude of mosaics representing the same scene with a redistribution of pixels. The sparse images from the different mosaics are added together to get new composite sparse images in which the number of null pixels is reduced. A bilinear demosaicing approach is applied to the composite sparse images to obtain fully defined images. Experimental results on images projected onto the response of our MSFA filter show that our composition method significantly improves image spatial resolution and minimizes reconstruction errors while preserving spectral fidelity.

Multispectral Imaging Method in Laparoscopy

Introduction. At present, video data acquired in narrow spectral bands are widely used to improve the efficiency of diagnostics in various medical fields, laparoscopy in particular. Conventional laparoscopy uses images obtained in the white light. Images obtained in the visible range suitably depict the color and textural features of tissues. However, it is difficult for a physician to use visible images for distinguishing between lesion areas and normal tissues, largely due to their proximity in color and texture. The efficiency of lesion detection can be improved using fluorescence images, which clearly differentiate lesion areas from normal tissues. However, the use multispectral data implies the need to present the images obtained both in the white and fluorescence light to the physician. In this paper, we propose an image composition method based on visible and fluorescence images, which facilitates their analysis by physicians. A distinctive feature of the method is the use of CIEDE 2000 metric for image fusion, which takes the properties of human vision into account.Aim. Development of a method for multispectral data visualization, which provides a physician with an image that combines white light data and a color-highlighted area of lesions.Materials and methods. The proposed method consists of the following steps: preprocessing of images obtained in visible and fluorescence light; segmentation of the lesion area in the fluorescence images; generation of a pseudo-color image of the segmented lesion area; and fusion of the pseudo-color image with the image obtained in the white light.Results. The proposed method forms an image that includes an image of the operation area obtained in the white light and a separated lesion area based on fluorescence information in the near infrared range. The image composite takes the properties of human vision into account. An experimental study of the method was carried out on actual laparoscopic images, involving endoscopists who were experts in subjective evaluation of video images. The method of paired comparisons was used to evaluated the presented images. The majority of experts preferred the fused image formed by the proposed method to those visualized simultaneously in the white and fluorescence light.Conclusion. The developed method ensures generation of images with an increased diagnostic value.

A Realistic Hand Image Composition Method for Palmprint ROI Embedding Attack

Palmprint recognition (PPR) has recently garnered attention due to its robustness and accuracy. Many PPR methods rely on preprocessing the region of interest (ROI). However, the emergence of ROI attacks capable of generating synthetic ROI images poses a significant threat to PPR systems. Despite this, ROI attacks are less practical since PPR systems typically take hand images as input rather than just the ROI. Therefore, there is a pressing need for a method that specifically targets the system by composing hand images. The intuitive approach involves embedding an ROI into a hand image, a comparatively simpler process requiring less data than generating entirely synthetic images. However, embedding faces challenges, as the composited hand image must maintain a consistent color and texture. To overcome these challenges, we propose a training-free, end-to-end hand image composition method incorporating ROI harmonization and palm blending. The ROI harmonization process iteratively adjusts the ROI to seamlessly integrate with the hand using a modified style transfer method. Simultaneously, palm blending employs a pretrained inpainting model to composite a hand image with a continuous transition. Our results demonstrate that the proposed method achieves a high attack performance on the IITD and Tongji datasets, with the composited hand images exhibiting realistic visual quality.

MT-GAN: toward realistic image composition based on spatial features

The purpose of image composition is to combine the visual elements of different natural images to produce a natural image. The performance of most existing image composition methods drops significantly when they solve multiple issues, such as image harmonization, image blending, shadow generation, object placement, and spatial transformation. To address this problem, we propose a multitask GAN for image compositing based on spatial features, aiming to simultaneously address the geometric and appearance inconsistency. We use three related learning objective functions to train the network. Moreover, a new dataset including 7756 images with RoI region annotations is contributed to help evaluate the multitask image compositing results. Extensive experiments demonstrate that our proposed method is effective on our dataset.

SupCAM: Chromosome cluster types identification using supervised contrastive learning with category-variant augmentation and self-margin loss.

Chromosome segmentation is a crucial analyzing task in karyotyping, a technique used in experiments to discover chromosomal abnormalities. Chromosomes often touch and occlude with each other in images, forming various chromosome clusters. The majority of chromosome segmentation methods only work on a single type of chromosome cluster. Therefore, the pre-task of chromosome segmentation, the identification of chromosome cluster types, requires more focus. Unfortunately, the previous method used for this task is limited by the small-scale chromosome cluster dataset, ChrCluster, and needs the help of large-scale natural image datasets, such as ImageNet. We realized that semantic differences between chromosomes and natural objects should not be ignored, and thus developed a novel two-step method called SupCAM, which could avoid overfitting only using ChrCluster and achieve a better performance. In the first step, we pre-trained the backbone network on ChrCluster following the supervised contrastive learning framework. We introduced two improvements to the model. One is called the category-variant image composition method, which augments samples by synthesizing valid images and proper labels. The other introduces angular margin into large-scale instance contrastive loss, namely self-margin loss, to increase the intraclass consistency and decrease interclass similarity. In the second step, we fine-tuned the network and obtained the final classification model. We validated the effectiveness of modules through massive ablation studies. Finally, SupCAM achieved an accuracy of 94.99% with the ChrCluster dataset, which outperformed the method used previously for this task. In summary, SupCAM significantly supports the chromosome cluster type identification task to achieve better automatic chromosome segmentation.

Shadow Generation for Composite Image in Real-World Scenes

Image composition targets at inserting a foreground object into a background image. Most previous image composition methods focus on adjusting the foreground to make it compatible with background while ignoring the shadow effect of foreground on the background. In this work, we focus on generating plausible shadow for the foreground object in the composite image. First, we contribute a real-world shadow generation dataset DESOBA by generating synthetic composite images based on paired real images and deshadowed images. Then, we propose a novel shadow generation network SGRNet, which consists of a shadow mask prediction stage and a shadow filling stage. In the shadow mask prediction stage, foreground and background information are thoroughly interacted to generate foreground shadow mask. In the shadow filling stage, shadow parameters are predicted to fill the shadow area. Extensive experiments on our DESOBA dataset and real composite images demonstrate the effectiveness of our proposed method. Our dataset and code are available at https://github.com/bcmi/Object-Shadow-Generation- Dataset-DESOBA.

Handwritten Digits Image Generation with help of Generative Adversarial Network: Machine Learning Approach

In recent years, research into Generative Adversarial Nets (GANs) has increased dramatically. GAN was first proposed in 2014 and has since used in various real-time applications,Includes computer vision and natural language processing for approximately accurate results. Image composition is the most popular study of the many applications of GAN,Studies in this area have already shown the great future of using GAN for image composition.This article shows how to classify image composition methods, reviews different models of text-to-image composition and image-to-image conversionand provides some metrics and future research on image composition using GAN. I will explain the direction of in this paper. In current years, frameworks using Generative Adversarial Networks (GAN) have been very successful in many areas many areas, especially in image generation, asthey can create very realistic and crisp images and train on large datasets. However, successful GAN training can be very difficult if you need high resolution images. Text-to-image compositing, image-to-image conversion, face manipulation, 3D image compositing, and deep master printing are five interesting areas that can be applied to image compositing based on the state-of-the-art GAN technology described in this article. It presents a comprehensive analysis of current GAN-based imaging models, including their strengths and weaknesses. At the same time, recent rediscovery of deep learning and widespread interest in generation methods in the scientific community have made it possible to generate realistic images by learning the data distribution from noise. If the input data contains information about the visual content of the image, the quality of the generated image will improve.

A deep learning network based end-to-end image composition

Currently, high-quality image composition largely depends on multiple user interactions and complex manual operations. In particular, the process of composition object extraction and region determination has become a burden that cannot be underestimated, restricting wider applications. Aiming at this problem, we propose an end-to-end image composition method that combines powerful deep-learning-based application modules such as image retrieval and instance segmentation to realize efficient non-interactive image composition. Specifically, the retrieval module, which is based on the attention mechanism, can determine semantically similar material images. Moreover, the content of interest (COI) extraction and optimization procedure is able to select the most proper instance among the material images. Finally, we propose the double-sieving strategy, which locates the best composition position in the target image. Using these effective modules, we carried out niche targeting experiments using an image database with high plausibility. The realistic experimental results illustrate that our method can achieve effective and reasonable end-to-end image composition.

Image Composition of Partially Occluded Objects

AbstractImage composition extracts the content of interest (COI) from a source image and blends it into a target image to generate a new image. In the majority of existing works, the COI is manually extracted and then overlaid on top of the target image. However, in practice, it is often necessary to deal with situations in which the COI is partially occluded by the target image content. In this regard, both tasks of extracting the COI and cropping its occluded part require intensive user interactions, which are laborious and seriously reduce the composition efficiency. This paper addresses the aforementioned challenges by proposing an efficient image composition method. First, we extract the semantic contents of the images by using state‐of‐the‐art deep learning methods. Therefore, the COI can be selected with clicks only, which can greatly reduce the demanded user interactions. Second, according to the user's operations (such as translation or scale) on the COI, we can effectively infer the occlusion relationships between the COI and the contents of the target image. Thus, the COI can be adaptively embedded into the target image without concern about cropping its occluded part. Therefore, the procedures of content extraction and occlusion handling can be significantly simplified, and work efficiency is remarkably improved. Experimental results show that compared to existing works, our method can reduce the number of user interactions to approximately one‐tenth and increase the speed of image composition by more than ten times.

Assessing the Ability of Luojia 1-01 Imagery to Detect Feeble Nighttime Lights.

The Luojia1-01 (LJ1-01) satellite launched on 2 June 2018 provides a new option for nighttime light (NTL) application research. In this paper, four types of human settlements, such as cities, counties, towns and villages, are sampled to evaluate the potential of LJ1-01 to detect feeble NTL by comparing with the NTL images from the Defense Meteorological Satellite Program’s Operational Linescan System (DMSP/OLS) and the Visible Infrared Imaging Radiometer Suite (VIIRS) on the Suomi National Polar-Orbiting Partnership Satellite. First, the landscape indices and cutoff threshold method are applied to enhance signal-noise ratio (SNR). Then, the detection accuracy of samples is evaluated to determine the optimal cutoff threshold for each NTL data source. After that, the spatial correspondence of different NTL images and the area consistency between the samples and NTL footprints are compared. Finally, after the discussion of feeble NTL detection and the influence of clouds, moonlight and image composites, it can be concluded that LJ1-01 is more suitable for detection feeble NTL objects, while great importance should be attached to the measures to eliminate the noise in LJ1-01 image and make LJ1-01 more widely used: (1) In the study area, a suitable cutoff threshold of LJ1-01 image can be set to 0.1 nano-Wcm−2sr−1, which is lower than that of VIIRS image (0.3 nano-Wcm−2sr−1), and this enables LJ1-01 to reserve more information of NTL, especially the feeble NTL. Moreover, the minimum area that can be identified by NTL footprints from LJ1-01 is 0.02 km2, while that of VIIRS and DMSP are 0.3 km2 and 4.5 km2, respectively. (2) The cutoff threshold method can identify the range of NTL with more noise, but cannot eliminate the noise separately. The filtering method and the image composition method may play more important role in the applications of LJ1-01 data.

Efficient image-based analysis of fruit surfaces using CCD cameras and smartphones

Today’s smartphones make a broad variety of sensors (gyroscope, magnetometer, camera, accelerometer, GPS, etc.) readily available and easily accessible through different APIs, favouring the acquisition of data. An everyday usage is the measurement of physical parameters, like sound or acceleration. The advances in terms of level of integration and its application to embedded devices power consumption and wide adoption of system on chips and more recently multiprocessors system on chip mean that a new sort of applications can be addressed. Applications are backed with powerful computing devices depending on batteries. Using these resource-limited devices and their parallel power efficiently is a challenging task. To fully exploit the potential of these hardware devices, parallelism has to be carefully applied to the most resource demanding parts of the application. This paper proposes an efficient image composition method to analyze fruit surfaces using CCD cameras and smartphones. The image composition is done by capturing video from which redundant frames are disposed using a data-parallel local feature detector. Relevant frames are then stitched using direct methods. The proposal was tested in the case of calculating the damaged surface of cherries.

Creative and high-quality image composition based on a new criterion

Image compositing techniques are primarily utilized to achieve realistic composite results. Some existing image compositing methods, such as gradient domain and alpha matting, are widely used in the field of computer vision, and can typically achieve realistic results, especially for seamless boundaries. However, when the candidate composite images and the target images have obvious differences, such as color, texture and brightness, the composite results are unrealistic and inconsistent. At the same time, traditional compositing methods focus on basic feature matching, ignoring semantic rationality in composition processing. Quite a few compositing methods thus generate composite results without semantic rationality.In this paper, a new multi-scale image composition method has been presented. In the composition process, wavelet pyramid and basic feature handling were used to achieve multi-scale compositions. More importantly, a new criterion was established, based on the semantic rationality of images, which could ensure that the composite images are semantically valid. A large database was created to facilitate experimentation. The experiments showed that the methodology introduced in this paper produced superior results compared to traditional composition methods; the composite results were not only consistent and seamless, but were also semantically valid.

Efficient Poisson Image Editing

Image composition refers to the process of composing two or more images to create an acceptable output image. It is one of the important techniques of image processing. In this paper, two efficient methods for composing color images are proposed. In the proposed methods, the Poisson equation is solved using image pyramid, and divide-and-conquer methods. The proposed methods are more efficient than other existing image composition methods. They reduce the time taken in the composition process while achieving almost identical results using the previous image composition methods. In the proposed methods, the Poisson equation is solved after converting it to a linear system using different methods. The results show that the time for composing color images is decreased using the proposed methods.

포아송 객체 편집을 이용한 스테레오 영상 합성

본 논문에서는 포아송 영상 편집을 이용한 스테레오 3차원 영상 합성 방법을 제안한다. 3차원 영상을 만들 때 객체의 깊이 값을 고려하지 않고 색상 영상을 합성하면 매우 부자연스러운 결과를 얻는다. 본 논문에서는 그랩컷 방법을 사용하여 추출한 전경 객체를 새로운 배경 영상에 포함시켜 자연스러운 2차원 영상을 만든다. 제안된 방법에서는 기하학적인 위치뿐 아니라 색상 톤도 고려하여 자연스러운 영상을 합성한 후, 합성된 영상의 경계선 주변에 블러링을 수행하여 전경과 배경이 잘 어우러지게 만든다. 이와 더불어, 임의로 주어진 객체의 깊이 값을 이용하여 추출된 객체의 변위를 구하고, 오른쪽 색상 영상과 깊이 영상을 자동으로 합성한다. 실험을 통해, 제안하는 방법이 자연스러운 스테레오 3차원 영상을 생성할 수 있음을 확인했다. In this paper, we propose a stereo image composition method based on Poisson image editing. If we synthesize images without considering their depth values, it may lead to unwanted consequences. When we segment an image into its background and foreground regions using Grabcut, we take into account their geometric positions to mix color tones; thus, the image is composited more naturally. After synthesizing images, we apply a blurring operation around object boundaries; then, the foreground object and background are composited more seamlessly. In addition, we can adjust the distance of the object by setting arbitrary depth values and generating right color and depth images automatically. Experimental results show that the proposed stereo image composition method provides naturally synthesized stereo images. Improved portions were subjectively confirmed as well.

키넥트 깊이 정보와 DSLR을 이용한 스테레오스코픽 비디오 합성

크로마키 방식에 의한 영상 합성은 색상 정보에 의해 전경 후경을 분리하기 때문에 객체 색상의 제약과 특정 스크린이 위치해 있어야 하는 공간의 제약이 있다. 특히 스테레오스코픽 3D 디스플레이를 위한 영상 합성은 크로마키 방식과는 달리 3D 공간에서의 자연스러운 영상 합성이 요구된다. 본 논문에서는 고해상도의 깊이 정보를 이용하여 깊이 키잉(depth keying) 방식에 의한 3D 공간에서의 스테레오스코픽 영상 합성을 제안하였다. 이를 위해 DSLR과 마이크로소프트사 키넥트 센서간의 카메라 캘리브레이션을 통해 고화질의 깊이 정보 획득 후 RGB 정보와의 정합 과정을 통해 3차원 데이터를 획득하였다. 깊이 정보에 의해 배경과 분리 된 객체는 3차원 공간에서의 포인트 클라우드 형태로 표현되어 가상 배경과 합성하였다. 이후 가상의 스테레오 카메라에 의해 Full HD 스테레오스코픽 비디오 합성 영상 획득 결과를 보였다. Chroma key technique which composes images by separating an object from its background in specific color has restrictions on color and space. Especially, unlike general chroma key technique, image composition for stereo 3D display requires natural image composition method in 3D space. The thesis attempted to compose images in 3D space using depth keying method which uses high resolution depth information. High resolution depth map was obtained through camera calibration between the DSLR and Kinect sensor. 3D mesh model was created by the high resolution depth information and mapped with RGB color value. Object was converted into point cloud type in 3D space after separating it from its background according to depth information. The image in which 3D virtual background and object are composed obtained and played stereo 3D images using a virtual camera.

2A2-L02 近赤外可視光複合内視鏡開発のための射影変換を用いたリアルタイム映像合成に関する研究(手術支援ロボティクス・メカトロニクス(2))

This paper proposed a real time image composition method for a near-infrared (NIR) endoscopic using projective transformation matrix and distance compensation. NIR light is superior in living body permeability. We are developing novel NIR endoscopic and already tested by animal experiments using Rare-Earth-Doped-Ceramic-Nano-Phosphor (RED-CNP). The images captured with the NIR camera and visible light camera are displayed separately in our currently system. This system causes a problem of doctor's burden. The proposed method can compose a NIR light image and a visible light image in real time. The experimental results show the proposed method is useful for the NIR endoscopic system.

모바일 장치에서 입체 영상 생성을 위한 깊이 영상 기반 렌더링의 부가 정보 영상 구성 방법에 관한 연구

본 논문에서는 낮은 대역폭을 가진 모바일 DMB 환경에서 사용할 수 있도록 깊이 영상 기반 렌더링(DIBR) 기법을 이용한 3차원 스테레오 입체 방송의 부가정보 영상 구성 방법을 설명한다. DIBR에서는 좌측 시점의 영상과 깊이 정보를 전송하고 수신측에서 우측 시점을 복원하는데, 화질이 손상될 수 있다. 본 논문에서는 부가 정보를 구성해서 함께 전송하여 복원된 영상의 질을 높이고 전송하는 데이터의 양을 줄이는 방법을 제시한다. This paper presents the delta image composition methods using Depth-Image-Based Rendering (DIBR) for 3D stereoscopic broadcasting for the low bandwidth mobile DMB broadcasting system. With DIBR, a left and depth images are transmitted to a mobile device, which restores the right view, whose quality may be poor. This paper describes delta image composition methods for the restoration while minimizing the amount of the transmitted data.

시각효과 영상제작에서 카메라 추적과 영상합성 기술의 활용

영화 등에서 컴퓨터그래픽 기술은 전체 흐름의 중심을 차지하는 핵심적인 요소이며, 미국 등에서 개발된 첨단 영상 기술을 바탕으로 디지털콘텐츠 산업을 견인하면서 비약적인 성장을 이루고 있다. 그러나 국내의 경우, 영상 콘텐츠의 제작 과정에서 연구개발을 병행하기 어려운 열악한 상황이고 해외 상용 툴에 대한 의존도가 높고, 노동 집약적인 제작 파이프라인(pipeline)으로 구성되어 있다. 이러한 문제점을 극복하기 위하여 개발된 하드웨어 및 소프트웨어 연구 결과물을 기술하였으며, 개발된 시각효과 기술을 바탕으로 시범 콘텐츠("The Sixty-miles-an-hour man")의 제작 과정을 정리하였다. 특히 정확한 카메라 모션추적 및 자연스러운 영상합성이 이루어지도록 실제의 동적 환경에서의 다양한 촬영 및 기술적 요소를 고려하였다. 또한 시나리오를 작성하는 과정에서 기술 개발자와 콘텐츠 제작자간의 상호이해와 협력이 중요하며, 이를 통해 제작현장에서 핵심 기술 요소의 도출 등의 중요성을 확인하였다. 그리고 제작효율성을 높이기 위해 필요한 요소기술을 도출해 제작환경의 구조개선 및 확장 가능성 등에 대한 고려가 요구된다. Visual effect production by computer graphics techniques has become more important in these days. However, there are three problems in the domestic contents production environments. First of all, most Korean film companies have insufficient generally potential to maintain and support their R&D (Research and Development) teams. Secondly, they are much dependent on the abroad commercial software tools. Finally, many people have to participate in the image production pipeline, called the labor-intensive pipeline. In producing a demonstration work, "The Sixty -miles-an-hour man", we have evaluated the usefulness of the developed camera tracking and image composition methods and then examined various production consideration elements. In addition, in order to develop a productive technical element and write a competitive film script, mutual understanding between the developers and the production users should be achieved. Also, this paper describes a role of the technical supervisor to direct the production environment in detail.

NPU-based image composition and display in parallel rendering system

In real-time rendering of massive data sets,it is a common solution to use PC clusters for the real-time parallel rendering.The image composition of all the results in parallel rendered by the cluster computers is a notorious bottleneck in sort-last clustered rendering systems.This paper presented a network processing unit(NPU)-based image composition method and a sort-last distributed rendering system,called NPUPR.The experimental results show that the NPU-based scheme can get three times faster frame rate than the 'direct send scheme' in the case of four rendering nodes.This paper also presented a scheme to extend four rendering nodes to more rendering nodes.Analysis shows that,the system is fully scalable with negligible penalty in frame rate.