Composite Image Research Articles

DATFuse: Infrared and Visible Image Fusion via Dual Attention Transformer

The fusion of infrared and visible images aims to generate a composite image that can simultaneously contain the thermal radiation information of an infrared image and the plentiful texture details of a visible image to detect targets under various weather conditions with a high spatial resolution of scenes. Previous deep fusion models were generally based on convolutional operations, resulting in a limited ability to represent long-range context information. In this paper, we propose a novel end-to-end model for infrared and visible image fusion via a dual attention Transformer termed DATFuse. To accurately examine the significant areas of the source images, a dual attention residual module (DARM) is designed for important feature extraction. To further model long-range dependencies, a Transformer module (TRM) is devised for global complementary information preservation. Moreover, a loss function that consists of three terms, namely, pixel loss, gradient loss, and structural loss, is designed to train the proposed model in an unsupervised manner. This can avoid manually designing complicated activity-level measurement and fusion strategies in traditional image fusion methods. Extensive experiments on public datasets reveal that our DATFuse outperforms other representative state-of-the-art approaches in both qualitative and quantitative assessments. The proposed model is also extended to address other infrared and visible image fusion tasks without fine-tuning, and the promising results demonstrate that it has good generalization ability. The source code is available at https://github.com/tthinking/DATFuse.

Seabed Classification From Multispectral Multibeam Data

Given the recent increase in the availability of multispectral multibeam echosounder data, this work aims to identify suitable processing and classification methodologies for seabed classification based on such data. We propose a complete processing and classification pipeline and investigate the adequacy of state-of-the-art classification algorithms to perform seabed classification based on multispectral backscatter data alone, and when additional data sources are considered. Starting from raw acquisition data, we generate region-wide multispectral backscatter composite images through noise removal, inpainting/gap-filling and mosaicking. Ground truth data from in situ seabed samples are used. We have tried different classification methods, including random forests, support vector machines, and multilayer perceptrons, with the latter providing the best results. Quantitative and qualitative evaluation on five surveys indicate high classification performance based only on multispectral backscatter data, while additional features, like bathymetry, bathymetric positional index (BPI), or positional encoding, offer limited gains. We offer a web service for seabed classification from multispectral multibeam echosounder data to further support and increase interest in the topic.

Erkek Vücut Geliştirme Sporcularında Vücut Kompozisyonu ile Benlik Saygısı ve Beden İmgesi Arasındaki İlişki

Being in optimal body composition is an important factor in athletes due to factors such as health, performance and success of athletes. In general, lean body mass is considered an indicator of muscle fitness and is a performance-related factor. Bodybuilding is a branch of sports that aims to increase the strength and volume of the muscles and is mostly done by men. After a while, the training goals can move towards body images that cannot be achieved under normal conditions. In this review, it is aimed to examine the relationship between body composition, self-esteem and body image on male bodybuilders. Recent studies have shown an increased interest in the relationship between body dissatisfaction, self-esteem and body image. It is seen that there is a negative relationship between self-esteem and body image in individuals with body dissatisfaction. Approaches that change one's self-esteem and body image can lead to positive results in a treatment that will provide an approach to body dissatisfaction. More work is needed on this subject.

Using the band ratio approach, FULL-PIXEL (SAM and SFF) and SUB-PIXEL (MF and SID) methodologies to discover areas with alteration using the Aster sensor: a case study in north westen Iran - Sivardaghi Area

ASTER sensor data is among the most potent satellite data accessible for doing geological investigations, with images for the whole earth's surface. In order to test the capability of this sensor to detect places with geochemical alterations, photographs of Mount Seiver Daghi in the western Iranian province of Samal were utilized in this study. This region, which comprises of magmatic and volcanic terrain, is part of the Arsbaran territory and is covered by intrusive masses with alluvial and sedimentary deposits. To conduct this study, an ASTER measuring frame was utilized, which, after performing atmospheric corrections using the internal average relative reflectance (IARR) method of false color composite images and principal component analysis (PCA), was able to differentiate between different lithological units using the Band assignment method, full-pixel methods of spectral angle mapper (SAM) and base spectrum algorithm of spectral feature fitting (SFF) as well as sub-pixel methods of matched filtering. The study demonstrates that the approach of principal component analysis and false color composition is efficient for distinguishing sedimentary rock units from igneous rock units, and its application is suggested for the designated rock units. Due to the lack of spectral characteristics of feldspars and quartz in the short infrared wavelength range, the basic spectrum methods utilized in this work are incapable of identifying such minerals. It is not advised to use these algorithms to distinguish between various magmatic units.

Seeing Cleft Lip from a New Angle: Crowdsourcing to Determine whether Scar Severity or Lip Angle Matters More to the General Public.

Modern cleft lip surgery aims to restore symmetry and create a level, normal-appearing Cupid's bow. However, families' concerns often center on the degree of scarring. The authors hypothesized that the lip angle would be less influential than scar severity in layperson ratings. The authors received institutional review board approval for modifying patients' postoperative photographs to create systematic variations displaying different levels of scarring and lip angle. Each child's resulting composite images were presented in pairs to internet raters using Amazon Mechanical Turk. Users selected the simulated postoperative result they felt to be most normal. The Bradley-Terry model was used to determine raters' preferences between different levels of scarring and lip angle. Four children with primary unilateral cleft lip repair had their postoperative photographs modified (mean age, 1.4 years; mean follow-up, 1.0 years). Twelve-hundred crowdsourced pairwise ratings were collected for each patient (4800 combined ratings). For all four children, raters preferred images with more severe scarring than those with a greater lip angle, suggesting uneven lip angle has a more negative effect on perceived appearance. Online crowdsourcing postoperative lip angle had a significantly greater influence on ratings of normal appearance than does the severity of scarring. Although patients may ask about scars more often, clinically, this study suggests perceptions of a cleft lip repair result may be more likely influenced by the angle of the cleft lip repair.

Painterly Image Harmonization in Dual Domains

Image harmonization aims to produce visually harmonious composite images by adjusting the foreground appearance to be compatible with the background. When the composite image has photographic foreground and painterly background, the task is called painterly image harmonization. There are only few works on this task, which are either time-consuming or weak in generating well-harmonized results. In this work, we propose a novel painterly harmonization network consisting of a dual-domain generator and a dual-domain discriminator, which harmonizes the composite image in both spatial domain and frequency domain. The dual-domain generator performs harmonization by using AdaIN modules in the spatial domain and our proposed ResFFT modules in the frequency domain. The dual-domain discriminator attempts to distinguish the inharmonious patches based on the spatial feature and frequency feature of each patch, which can enhance the ability of generator in an adversarial manner. Extensive experiments on the benchmark dataset show the effectiveness of our method. Our code and model are available at https://github.com/bcmi/PHDNet-Painterly-Image-Harmonization.

Image Processing

Image processing entails altering an image's composition to enhance its graphical content for human interpretation and autonomous machine perception. Digital image processing is a subset of the electronic domain in which an image is transformed into a collection of tin y numbers, or pixels, that reflect a physical property, like s cene radiance, are stored in a digital memory, and are then processed by a computer or other digital hardware. Two key application areas have sparked interest in digital i mage processing techniques: improving pictorial information for human interpretation and processing image data for storage, transmission, and representation for autonomous machine perception. Boundaries are described by edges, and as edge detection i s one of the most challenging image processing tasks, it is an issue of fundamental significance .

Visualizing landmark-based face morphing traces on digital images

This paper focuses on an identity sharing scheme known as face image morphing or simply morphing. Morphing is the process of creating a composite face image, a morph, by digitally manipulating face images of different individuals, usually two. Under certain circumstances, the composite image looks like both contributors and can be used by one of them (accomplice) to issue an ID document. The other contributor (criminal) can then use the ID document for illegal activities, which is a serious security vulnerability. So far, researchers have focused on automated morphing detection solutions. Our main contribution is the evaluation of the effectiveness and limitations of two image forensics methods in visualizing morphing related traces in digital images. Visualization of morphing traces is important as it can be used as hard evidence in forensic context (i.e., court cases) and lead to the development of morphing algorithm specific feature extraction strategies for automated detection. To evaluate the two methods, we created morphs using two state-of-the-art morphing algorithms, complying with the face image requirements of three currently existing online passport application processes. We found that complementary use of the visualization methods can reveal morphing related traces. We also show how some application process-specific requirements affect visualization results by testing three likely morphing attack scenarios with varied image processing parameters and propose application process amendments that would make forensic image analysis more reliable.

Функционально-грамматический максимализм инфинитивной поэзии Иосифа Бродского

This paper is focused on the phenomenon of functional-grammatical maximalism of the independent infinitive, exemplified by Joseph Brodsky’s poetic texts. The expression in them of three predicative categories of the infinitive – time, person and mood – is analysed. The specific features of this expression include: a unique temporal plane of action, expressed in a literary text by the infinitive through its internal time, i.e. the aspect; a composite image of the subject of an infinitive text, i.e. a meta-subject, in which the Self of the speaker merges with the Self of the listener, represented by both the second and the third person or by the seeming absence of the person; and meditative modality characteristic of Brodsky’s infinitive writing, gravitating towards semantic hyperbolism, i.e. the theme of another, unattainable existence. The author used the functional-grammatical, semantic and contextual analyses, generally applied in poetic grammar. The analysis found a combination of homogeneity, formal minimalism and functional maximalism of the infinitive. In addition, the following lexical-grammatical methods of expressing this phenomenon were identified in Brodsky’s poems: the category of aspect and semantics of infinitives, which contribute to changing the nature of time in a poetic text and to expressing the subject’s attitude to the action of debitive, epistemic and meditative modality; additional temporal indicators in the form of secondary parts of the sentence helping to create a time cycle in the text; personal pronouns indicating the presence of persons in contact with the subject of the infinitive action. The phenomenon analysed in this paper can be explained by the nature of the infinitive, which is indefinite and ambiguous and, therefore, has high functional potential. This makes the topic under discussion interesting for linguists and calls for an in-depth study of this linguistic unit using different types of texts.

Impact of land use land cover change on the land surface temperature: A case study of Shillong

Shillong has been experiencing changes in land use land cover over the past years especially in the town areas where a lot of built-ups are emerging. In response to this, there has been some impacts in the distribution of temperature. In this study, remote sensing techniques have been used. The maximum likelihood method has been used for supervised classification of land use land cover from false color composite images. Landsat 5 TM and Landsat 8 OLI/ TIRS has been used in this study. The study period is 30 years from 1992 - 2022 with an interval of 15 years. The emergence and increase of more built-up and impervious land cover type surfaces has made the land surface temperature to increase. The correlation between NDVI and LST shows a negative relationship while that of NDBI and LST shows a positive relationship. The temperature has been increasing with an average of 2° C each interval of study. The hotspot analysis has been used to demarcate the heat zones. The highest temperature is found in the crowded and high built-up areas while the lowest temperature is found in the vegetated lands which further cause heat island effect in these highly built-up areas. Key Words: Land use land cover, Land surface temperature, Change detection, Hotspot analysis, Correlation analysis, Heat-island effect.

Integration of remote sensing and geophysical methods for structural and lithological mapping in a part of Precambrian Basement Rocks, northern Nigeria

Landsat 8 OLI data, Aeromagnetic data, and Radial Vertical Electrical Sounding (RVES) survey data were integrated to map lithology, delineate structures and their trends, and delineate possible mineralized zones in the area. Landsat 8 OLI data yielded a color composite image, and surface lineaments map of the area. The aeromagnetic maps were utilized to map lithology, and subsurface structures. A radial survey confirmed the fractures derived from the structural maps. Results classified the area into three geological units viz: migmatite, banded gneiss, and quartzite. The banded gneiss is the most deformed and contains series of structures that are significant for mineral and groundwater explorations. Clay alteration is the most dominant in the area, followed by iron oxide alterations. Lineament alignments are N-S, NNW-SSE, NNE-SSW, NE-SW, and E-W directions.

Separation of Isobaric Lipids in Imaging Mass Spectrometry Using Gas-Phase Charge Inversion Ion/Ion Reactions.

The diverse array of chemical compounds present in tissue samples results in many isobaric (i.e., same nominal mass) compounds in imaging mass spectrometry experiments. Adequate separation and differentiation of these compounds is necessary to ensure accurate analyte identification and avoid composite images comprising multiple compounds. High-resolution accurate mass (HRAM) measurements are able to resolve these compounds in some instances, but HRAM measurements are not always feasible depending on the instrument platform and the desired experimental time scale. Alternatively, tandem mass spectrometry (MS/MS) can be used to perform gas-phase transformations that improve molecular specificity. While conventional MS/MS methods employ collision induced dissociation (CID) to fragment compounds of interest and then analyze the product masses, gas-phase ion/ion reactions can be used to instead selectively react with desired classes of analytes. Herein, we have used gas-phase charge inversion ion/ion reactions to selectively resolve phosphatidylcholines (PCs) in isobaric lipid mixtures. A 1,4-phenylenedipropionic acid (PDPA) reagent dianion readily reacts with [M + H]+, [M + Na]+, and [M + K]+ ion types to produce demethylated product anions for each PC, [PC - CH3]-. These product anions are no longer isobaric and now differ in mass by 22 Da (protonated versus sodiated) and 16 Da (sodiated versus potassiated), respectively. This reaction has been used to differentiate isobaric lipids in the imaging mass spectrometry analysis of rat brain tissue.

Neural network modeling of titanium alloy composition-microstructure-property relationships based on multimodal data

The composition and microstructure of metallic materials determine their mechanical properties. Establishing quantitative “composition-microstructure-property” relationships is essential for material design and optimization. However, materials data are complex and heterogeneous, often containing multi-modal information such as “point-image-curve”. Traditional mathematical models and simulations can no longer meet the needs of advanced applications. Big data technology provides a new approach to materials research, and machine learning tools can offer high-dimensional information in a low-order form. In this study, a neural network-based titanium alloy microstructure identification model and a performance prediction model have been designed. A model for heterogeneous data of titanium alloy composition (data), microstructure (image), and performance (curve) has been established through the fusion of the two models. The cross-dimensional prediction of the material stress-strain curve using composition and microstructure image data has been successfully achieved. A four-step workflow of dataset preparation, pre-processing, model building, and fusion is determined. The task of identifying 1215 images of 14 titanium alloys and predicting 604 performance curves has been completed. This work demonstrates the process of establishing the “composition-microstructure-property” relationship of materials by big data technology.

MRI enhances the understanding of critical anatomy during primary laparoscopic port placement.

Despite the majority of laparoscopic visceral injuries occurring with primary entry, high-fidelity training models are lacking. Three healthy volunteers underwent non-contrast 3T MRI at Edinburgh Imaging. A direct entry 12mm trocar was filled with water to improve MR visibility, placed on the skin at entry points, then images were acquired in the supine position. Composite images were created, and distances from the trocar tip to the viscera were measured, demonstrating anatomical relationships during laparoscopic entry. With a BMI of 21 kg/m2, gentle downward pressure during skin incision or trocar entry reduced the distance to the aorta to less than the length of a No. 11 Scalpel blade (22mm). The need for counter-traction and stabilisation of the abdominal wall during incision and entry is demonstrated. With a BMI of 38 kg/m2, deviating from the vertical angle for trocar insertion can result in the entire trocar shaft being placed within the abdominal wall without entering the peritoneum, creating a 'failed entry.' At Palmer's point distance between the skin and bowel is only 20mm. Ensuring the stomach is not distended will minimise gastric injury risk. The use of MRI to provide visualisation of the critical anatomy during primary port entry allows the surgeon to gain better understanding of textually described best practice techniques.

Assessment of myelin in Alzheimer’s by a R1 and the relationship to amyloid burden

AbstractBackgroundAlzheimer’s disease (AD) is well known as a grey matter disorder, characterized by substantial and progressive neurodegeneration. However, less is known about myelin loss in AD, particularly as the disease progresses in severity. To gain a better insight on the role of myelin over the clinical and biologic progression of AD, we tested for differences in quantitative relaxometry R1, a metric sensitive to brain myelin, among individuals who were cognitively unimpaired (CU), diagnosed with mild cognitive impairment (MCI), and diagnosed with AD, in addition to testing for associations between R1 and amyloid PET. Understanding the course of myelin injury in disease progression is expected to address the gap in knowledge about the trajectory of myelin alterations in AD.MethodsParticipants from the Alzheimer’s Disease Connectome Project underwent T1‐weighted (T1w) MPnRAGE with motion‐correction and amyloid imaging with [11C]PiB PET. Composite T1w images and R1 maps were reconstructed at 1mm isotropic resolution. T1w scans were bias‐field corrected and denoised before undergoing tissue segmentation. R1 was sampled and bilaterally averaged across cortical‐adjacent white matter (WM). An average WM‐R1 was generated for each participant by calculating the mean of cortical regions R1. MANOVA and regression analyses were used to analyze the difference and relationship between each group’s WM‐R1 and PiB‐Index.ResultThe three groups showed an association with PiB‐Index and WM‐R1, Pillai’s Trace = 0.28, F(4, 110) = 4.56, p<0.01. Participants with AD showed higher amyloid and lower WM‐R1. In contrast, UC and MCI participants showed lower amyloid and WM‐R1. A significant correlation was found in individuals with AD wherein amyloid burden was positively correlated with WM‐R1 across the whole brain, R2 = 0.64, F(1, 6) = 10.99, p = 0.016.ConclusionOur findings indicate myelin differences by clinical status, in addition to relationships between amyloid and R1 indices of myelin. While AD was associated with lower myelin as indexed by R1, we unexpectedly found that higher cortical amyloid in AD was associated with higher myelin indexed by R1, suggesting an increase in myelination as AD progresses. Overall, lower myelin in clinical AD, accompanied by the higher myelin with greater amyloid burden suggests possible reparative mechanisms in response to neurodegeneration in AD.

Multispectral Satellite Imagery Products for Fire Weather Applications

Abstract Two multispectral satellite imagery products are presented that were developed for use within the fire management community. These products, which take the form of false color red–green–blue composites, were designed to aid fire detection and characterization, and for assessment of the environment surrounding a fire. The first, named the Fire Temperature RGB, uses spectral channels near 1.6, 2.2, and 3.9 μm for fire detection and rapid assessment of the range of fire intensity through intuitive coloration. The second, named the Day Fire RGB, uses spectral channels near 0.64, 0.86, and 3.9 μm for rapid scene assessment. The 0.64 μm channel provides information on smoke, the 0.86 μm channel provides information on vegetation health and burn scars, and the 3.9 μm channel provides active fire detections. Examples of these red–green–blue composite images developed from observations collected by three operational satellite imagers (VIIRS on the polar-orbiting platform and the Advanced Baseline Imager and Advanced Himawari Imager on the geostationary platform) demonstrate that both red–green–blue composites are useful for fire detection and contain valuable information that is not present within operational fire detection algorithms. In particular, it is shown that Fire Temperature RGB and Day Fire RGB images from VIIRS have similar utility for fire detection as the operational VIIRS Active Fire products, with the added benefit that the imagery provides context for more than just the fires themselves. Significance Statement The current generation of operational polar-orbiting weather satellites that began with the launch of Suomi NPP offers new capabilities with regard to fire detection and monitoring. In particular, false color red–green–blue composite imagery is now being used by fire managers, incident meteorologists, and others in the fire management community to visualize a fire’s behavior and the context in which it occurs. This paper outlines two of these red–green–blue composites that have gained widespread use throughout the U.S. National Weather Service and the Alaska Fire Service. These red–green–blue composites have been applied to the current generation of geostationary and polar-orbiting satellites to great effect and have changed how incident management teams respond to wildland fires.

Multicenter comparison of head shape outcomes for three minimally invasive strip craniectomy techniques for sagittal craniosynostosis.

A literature gap exists comparing whole head shape outcome following correction of sagittal craniosynostosis. The objective of this multicenter study was to provide an analysis of long-term results following three different endoscopic strip craniectomy techniques for correction of sagittal craniosynostosis: 1) spring-assisted strip craniectomy, 2) wide-strip craniectomy with biparietal and bitemporal barrel-stave wedge osteotomies plus helmet orthosis, and 3) narrow-strip craniectomy plus orthosis without barrel staves. Pre- and postoperative 3D stereophotogrammetric images were collected from patients who underwent craniosynostosis surgery. Procedures were divided among institutions as follows: spring-assisted strip craniectomies were performed at Atrium Health Wake Forest Baptist Hospital; narrow-strip craniectomies were performed at St. Louis Children's Hospital by one craniofacial surgeon; and wide-vertex craniectomies were performed at St. Louis Children's Hospital prior to 2010, and then continued at Children's Medical Center Dallas. Pre- and postoperative 3D whole-head composite images were generated for each procedure to visually represent outcomes at final follow-up and compared with age-matched normal controls. Patients in the spring-assisted strip craniectomy group showed normalization of frontal bossing and skull height compared with age-matched controls, whereas patients undergoing wide-strip craniectomy showed greater correction of occipital protrusion. Patients in the narrow-strip craniectomy cohort had intermediate results between these outcomes. Nested aggregate head shapes showed good correction of head shapes from all techniques. This large, retrospective, multicenter study illustrated whole head shape outcomes from three different craniectomy procedures. Although each procedure showed some differences in loci of primary correction, all three surgical methods demonstrated good correction of primary scaphocephalic deformity.

Image super-resolution reconstruction algorithm based on EDSR and improved SRGAN

Nowadays Image super-resolution reconstruction algorithm has become the dominant research aspect of computer vision, which is about generate high resolution composite image using low resolution original image. SRCNN(Super-Resolution Convolutional Neural Network) algorithm is the pioneer to apply deep learning to image super-resolution reconstruction, while SRGAN(Super-Resolution using Generative Adversarial Network) further explores the fusion of deep learning domain and image super-resolution reconstruction on grounds of SRCNN algorithm, and uses generative adversarial network to fit the extra information after image resolution improvement. Aiming at the defects of SRResNet generation network in SRGAN algorithm, this paper tries to improve the generator in its generation adversarial network by referring to the idea of ENSR(Enhanced Deep Residual Networks for Single Image Super-Resolution) algorithm, and obtains obvious effect. The improved algorithm has a significant improvement in performance.

PROPOSAL OF RATIOMETRIC INDEX FOR THE DIFFERENTIATION OF CELL PAINTED SUBORGANELLES USING DEEP CNN-BASED SEMANTIC SEGMENTATION

Cell painting technique provides large amount of potential information for applications such as drug discovery, bioactivity prediction and cytotoxicity assessment. However, its utility is restricted due to the requirement of advanced, costly and specific instrumentation protocols. Therefore, creating cell painted images using simple microscopic data can provide a better alternative for these applications. This study investigates the applicability of deep network-based semantic segmentation to generate cell painted images of nuclei, endoplasmic reticulum (ER) and cytoplasm from a composite image. For this, 3456 composite images from a public dataset of Broad Bioimage Benchmark collection are considered. The corresponding ground truth images for nuclei, ER and cytoplasm are generated using Otsu’s thresholding technique and used as labeled dataset. Semantic segmentation network is applied to these data and optimized using stochastic gradient descent with momentum algorithm at a learning rate of 0.01. The segmentation performance of the trained network is evaluated using accuracy, loss, mean Boundary [Formula: see text] (BF) score, Dice Index, Jaccard Index and structural similarity index. Gradient weighted Class Activation Mapping (Grad-CAM) is employed to visualize significant image regions identified by the model. Further, a cellular index is proposed as a geometrical measure which is capable of differentiating the segmented cell organelles. The trained model yields 96.52% accuracy with a loss of 0.07 for 50 epochs. Dice Index of 0.93, 0.76 and 0.75 is achieved for nuclei, ER and cytoplasm respectively. It is observed that nuclei to cytoplasm provides comparatively higher percentage change (74.56%) in the ratiometric index than nuclei to ER and ER to cytoplasm. The achieved results demonstrate that the proposed study can predict the cell painted organelles from a composite image with good performance measures. This study could be employed for generating cell painted organelles from raw microscopy images without using specific fluorescent labeling.

Semantic‐aware visual consistency network for fused image harmonisation

AbstractWith a focus on integrated sensing, communication, and computation (ISCC) systems, multiple sensor devices collect information of different objects and upload it to data processing servers for fusion. Appearance gaps in composite images caused by distinct capture conditions can degrade the visual quality and affect the accuracy of other image processing and analysis results. The authors propose a fused‐image harmonisation method that aims to eliminate appearance gaps among different objects. First, the authors modify a lightweight image harmonisation backbone and combined it with a pretrained segmentation model, in which the extracted semantic features were fed to both the encoder and decoder. Then the authors implement a semantic‐related background‐to‐foreground style transfer by leveraging spatial separation adaptive instance normalisation (SAIN). To better preserve the input semantic information, the authors design a simple and effective semantic‐aware adaptive denormalisation (SADE) module. Experimental results demonstrate that the authors’ proposed method achieves competitive performance on the iHarmony4 dataset and benefits from the harmonisation of fused images with incompatible appearance gaps.