Multi Images Research Articles

Swin-Caption: Swin Transformer-Based Image Captioning with Feature Enhancement and Multi-Stage Fusion

The objective of image captioning is to empower computers to generate human-like sentences autonomously, describing a provided image. To tackle the challenges of insufficient accuracy in image feature extraction and underutilization of visual information, we present a Swin Transformer-based model for image captioning with feature enhancement and multi-stage fusion (Swin-Caption). Initially, the Swin Transformer is employed in the capacity of an encoder for extracting images, while feature enhancement is adopted to gather additional image feature information. Subsequently, a multi-stage image and semantic fusion module is constructed to utilize the semantic information from past time steps. Lastly, a two-layer LSTM is utilized to decode semantic and image data, generating captions. The proposed model outperforms the baseline model in experimental tests and instance analysis on the public datasets Flickr8K, Flickr30K, and MS-COCO.

Enhanced breast cancer diagnosis through integration of computer vision with fusion based joint transfer learning using multi modality medical images

Breast cancer (BC) is a type of cancer which progresses and spreads from breast tissues and gradually exceeds the entire body; this kind of cancer originates in both sexes. Prompt recognition of this disorder is most significant in this phase, and it is measured by providing patients with the essential treatment so their efficient lifetime can be protected. Scientists and researchers in numerous studies have initiated techniques to identify tumours in early phases. Still, misperception in classifying skeptical lesions can be due to poor image excellence and dissimilar breast density. BC is a primary health concern, requiring constant initial detection and improvement in analysis. BC analysis has made major progress recently with combining multi-modal image modalities. These studies deliver an overview of the segmentation, classification, or grading of numerous cancer types, including BC, by employing conventional machine learning (ML) models over hand-engineered features. Therefore, this study uses multi-modality medical imaging to propose a Computer Vision with Fusion Joint Transfer Learning for Breast Cancer Diagnosis (CVFBJTL-BCD) technique. The presented CVFBJTL-BCD technique utilizes feature fusion and DL models to effectively detect and identify BC diagnoses. The CVFBJTL-BCD technique primarily employs the Gabor filtering (GF) technique for noise removal. Next, the CVFBJTL-BCD technique uses a fusion-based joint transfer learning (TL) process comprising three models, namely DenseNet201, InceptionV3, and MobileNetV2. The stacked autoencoders (SAE) model is implemented to classify BC diagnosis. Finally, the horse herd optimization algorithm (HHOA) model is utilized to select parameters involved in the SAE method optimally. To demonstrate the improved results of the CVFBJTL-BCD methodology, a comprehensive series of experimentations are performed on two benchmark datasets. The comparative analysis of the CVFBJTL-BCD technique portrayed a superior accuracy value of 98.18% and 99.15% over existing methods under Histopathological and Ultrasound datasets.

Precision Ice Detection on Power Transmission Lines: A Novel Approach with Multi-Scale Retinex and Advanced Morphological Edge Detection Monitoring.

Line icings on the power transmission lines are dangerous risks that may lead to situations like structural damage or power outages. The current techniques used for identifying ice have certain drawbacks, particularly when used in complex environments. This paper aims to detect lines on the top and bottom in PTLI with low illumination and complex backgrounds. The proposed method integrates multistage image processing techniques, including image enhancement, filtering, thresholding, object isolation, edge detection, and line identification. A binocular camera is used to capture images of PTLI. The effectiveness of the method is evaluated through a series of metrics, including accuracy, sensitivity, specificity, and precision, and compared with existing methods. It is observed that the proposed method significantly outperforms the existing methods of ice detection and thickness measurement. This paper uses average accuracy of detection and isolation of ice formations under various conditions at a percentage of 98.35, sensitivity at 91.63%, specificity at 99.42%, and precision of 96.03%. Furthermore, the accuracy of the ice thickness based on the thickness measurements is shown with a much smaller RMSE of 1.20 mm, MAE of 1.10 mm, and R-squared of 0.95. The proposed scheme for ice detection provides a more accurate and reliable method for monitoring ice formation on power transmission lines.

Self-learning based joint multi image super-resolution and sub-pixel registration

Multi Image Super-resolution (MISR) refers to the task of enhancing the spatial resolution of a stack of low-resolution (LR) images representing the same scene. Although many deep learning-based single image super-resolution (SISR) technologies have recently been developed, deep learning has not been widely exploited for MISR, even though it can achieve higher reconstruction accuracy because more information can be extracted from the stack of LR images. One of the primary obstacles encountered by deep networks when addressing the MISR problem is the variability in the number of LR images that act as input to the network. This impedes the feasibility of adopting an end-to-end learning approach, because the varying number of input images makes it difficult to construct a training dataset for the network. Another challenge arises from the requirement to align the LR input images to generate high-resolution (HR) image of high quality, which requires complex and sophisticated methods.In this paper, we propose a self-learning based method that can simultaneously perform super-resolution and sub-pixel registration of multiple LR images. The proposed method trains a neural network with only the LR images as input and without any true target HR images; i.e., the proposed method requires no extra training dataset. Therefore, it is easy to use the proposed method to deal with different numbers of input images. To our knowledge this is the first time that a neural network is trained using only LR images to perform a joint MISR and sub-pixel registration. Experimental results confirmed that the HR images generated by the proposed method achieved better results in both quantitative and qualitative evaluations than those generated by other deep learning-based methods.

Convolutional neural network-based multimodal image information fusion for moisture damage assessment of cultural heritage buildings

Water is an essential factor causing the deterioration of historical and cultural heritage, and identifying and quantifying moisture damage is essential for conserving cultural heritage buildings. This study combines infrared thermography (IRT) and microwave technology to investigate moisture-induced diseases and the variation of the spatial distribution of water content in the Yungang Grottoes of China, a World Heritage Site. Information on building humidity distribution was extracted and combined from IRT and microwave information. Furthermore, multi-feature image fusion combines different types of monitoring information through the Convolutional Neural Network (CNN), which is used to diagnose moisture distribution and content accurately. The features obtained from different non-destructive methods were then jointly analyzed using statistical analysis methods such as absolute contrast, noise ratio (SNR), and spatial stability to quantify the artifact moisture defect data, thus improving the accuracy and range of features for detecting water-damaged pathologies. The results show that the combination of IRT and microwave information technology not only qualitatively but also quantitatively analyses the moisture in the building, significantly using microwave moisture meters to study water distribution at different depths and surface characterization by IRT. The use of non-destructive monitoring tools helps to guide the diagnosis of moisture damage, restoration of heritage buildings, and the planning of intervention strategies.

Blind non-linear spectral unmixing with spatial coherence for hyper and multispectral images

Multi and hyperspectral images have become invaluable sources of information, revolutionizing various fields such as remote sensing, environmental monitoring, agriculture and medicine. In this expansive domain, the multi-linear mixing model (MMM) is a versatile tool to analyze spatial and spectral domains by effectively bridging the gap between linear and non-linear interactions of light and matter. This paper introduces an upgraded methodology that integrates the versatility of MMM in non-linear spectral unmixing, while leveraging spatial coherence (SC) enhancement through total variation theory to mitigate noise effects in the abundance maps. Referred to as non-linear extended blind end-member and abundance extraction with SC (NEBEAE-SC), the proposed methodology relies on constrained quadratic optimization, cyclic coordinate descent algorithm, and the split Bregman formulation. The validation of NEBEAE-SC involved rigorous testing on various hyperspectral datasets, including a synthetic image, remote sensing scenarios, and two biomedical applications. Specifically, our biomedical applications are focused on classification tasks, the first addressing hyperspectral images of in-vivo brain tissue, and the second involving multispectral images of ex-vivo human placenta. Our results demonstrate an improvement in the abundance estimation by NEBEAE-SC compared to similar algorithms in the state-of-the-art by offering a robust tool for non-linear spectral unmixing in diverse application domains.

Self-adaptive 2D[sbnd]3D image fusion for automated pixel-level pavement crack detection

Current 2D and 3D image-based crack detection methods in transportation infrastructure often struggle with noise robustness and feature diversity. To overcome these challenges, the paper use CSF-CrackNet, a self-adaptive 2D3D image fusion model utilizes channel and spatial modules for automated pavement crack segmentation. CSF-CrackNet consists of four parts: feature enhanced and field sensing (FEFS) module, channel module, spatial module, and semantic segmentation module. A multi-feature image dataset was established using a vehicle-mounted 3D imaging system, including color images, depth images, and color-depth overlapped images. Results show that the mean intersection over union (mIOU) of most models under the CSF-CrackNet framework can be increased to above 80 %. Compared with original RGB and depth images, the average mIOU increases with image fusion by 10 % and 5 %, respectively. The ablation experiment and weight significance analysis further demonstrate that CSF-CrackNet can significantly improve semantic segmentation performance by balancing information between 2D and 3D images.

Wearable damaged clothing fabric image recognition system based on image restoration algorithm

With the improvement of people's living standards and the rapid growth of national technological strength, wearable devices have gradually integrated into people's daily lives, making them convenient for people's work and lives. Traditional fabric detection methods often rely on manual visual inspection, which has low efficiency and accuracy that does not meet the requirements. In view of this, the research has innovatively integrated GAN, CNN, field Programmable gate array (FPGA), multi-stage image recovery algorithm based on graph neural network, for high frequency and low frequency fusion, and combined it with wearable devices to develop a new wearable fabric recovery and recognition system. The results showed that in the COCO dataset, after 250 iterations, the WFR2S algorithm achieved a damage identification accuracy of 95.5 %, a mean absolute error of 0.011, a real-time processing speed of 30 fps, and a resource consumption of 2.4 W. These data demonstrate the significant performance advantages of WFR2S in image recovery and damage detection. The newly developed image recovery algorithm improves the accuracy, availability and stability of damage recognition in garment fabrics. This study not only has theoretical value but also has broad application prospects in practical application. The aim is to contribute to the intelligent development of the garment industry.

Enhanced low-light image fusion through multi-stage processing with Bayesian analysis and quadratic contrast function

This manuscript introduces an innovative multi-stage image fusion framework that adeptly integrates infrared (IR) and visible (VIS) spectrum images to surmount the difficulties posed by low-light settings. The approach commences with an initial preprocessing stage, utilizing an Efficient Guided Image Filter for the infrared (IR) images to amplify edge boundaries and a function for the visible (VIS) images to boost local contrast and brightness. Utilizing a two-scale decomposition technique that incorporates Lipschitz constraints-based smoothing, the images are effectively divided into distinct base and detail layers, thereby guaranteeing the preservation of essential structural information. The process of fusion is carried out in two distinct stages: firstly, a method grounded in Bayesian theory is employed to effectively combine the base layers, so effectively addressing any inherent uncertainty. Secondly, a Surface from Shade (SfS) method is utilized to ensure the preservation of the scene's geometry by enforcing integrability on the detail layers. Ultimately a Choose Max principle is employed to determine the most prominent textural characteristics, resulting in the amalgamation of the base and detail layers to generate an image that exhibits a substantial enhancement in both clarity and detail. The efficacy of our strategy is substantiated by rigorous testing, showcasing notable progressions in edge preservation, detail enhancement, and noise reduction. Consequently, our method presents significant advantages for real-world applications in image analysis.

Construction of Practical Teaching Mode of Law Course Based on Multi-Mode and Low-Resource Language Learning

China's legal profession is a new profession emerging in the process of China's rule of law. Its development and refinement are increasing in accordance with the rapid social, economic and political development. However, as a lawyer, people need to learn real kung fu. For complex and changeable legal practice problems, people should use the simplest and most effective way to solve them. Therefore, it is necessary to study the curriculum of law specialty to achieve its purpose. It is also necessary to reform the traditional education methods, introduce modern scientific and technological means, and improve their effectiveness, so as to construct a practical teaching mode of Professional Courses in Law (PCL). For practical legal courses, it can be conducted through multi-mode low-resource language learning method. This paper aimed to study how to study the ability of students to obtain information, process information and analyze and process information based on intelligent multi-image feature fusion. Based on the investigation of teaching objectives and teachers' teaching requirements, and the analysis and arrangement of the investigation results, a multi-task learning system model based on image intelligent features was established. In this experiment, 494 valid questionnaires were used to analyze the current situation of PCL practical teaching. Among them, 10.32% and 65.59% of the students were very satisfied with the results of PCL practical teaching, and 18.02% and 59.31% of the students liked PCL practical teaching very much. In the survey on the influencing factors of PCL practical teaching activities, 61.34% of the students thought that the students did not know enough and were not enthusiastic enough. On the whole, the students are satisfied with the practical teaching arrangement of PCL, but from the actual situation, the effect of practical teaching is not very good, and the expected goal has not been achieved.

Underwater image enhancement via color correction and multi-feature image fusion

Abstract The light attenuation underwater causes the actual underwater images to suffer from color cast, low contrast, and weak illumination. To address these issues, an effective fusion-based method is proposed, which realizes color correction, brightness adjustment, contrast, and detail enhancement of underwater images. Concretely, we first design an adaptive color correction method via dominant color channel judgment and lower color channel compensation. Then, we detect the brightness of each input image and propose a gamma correction function based on the gradient of the cumulative histogram to adjust the brightness of the low-light images. Subsequently, global histogram stretching and adaptive fractional differentiation techniques are employed to process the brightness-adjusted image, and then the global contrast-enhanced version and detail-enhanced version are generated respectively. To integrate the advantages of both versions, a channel fusion method based on the Lab color space is used to fuse the luminance and color of the two versions separately. The experimental results demonstrate the effectiveness of the proposed method in improving the color and illumination of underwater images, as well as enhancing the clarity of images. Moreover, the testing results on multiple datasets validate the excellent stability of this method.

Multi wavelength optical image feature fusion method for improving the reconstruction effect of cerebral hematoma

Diffuse optical tomography (DOT) under medical image guidance can be used to reconstruct images of subdural hematomas for monitoring purposes. However, when the shape of hematoma is irregular or the signal to noise ratio(SNR) of input signal is low, the accuracy and robustness of this method both decrease. In order to alleviate this situation and improve the reconstruction effect of cerebral hematoma images, a multi wavelength optical image feature fusion method was proposed in this paper. Experimental results show that the mean relative volume error (VRE) of the hematoma reconstruction model based on the multi-wavelength optical image feature fusion method is only 0.66 %, and the mean value of the average Manhattan distance (AMD) between the reconstructed absorption coefficient and the true absorption coefficient is 0.0052 mm−1. Compared with the model using single-wavelength optical image mapping feature extraction method, the multi-wavelength optical image feature fusion method reduces the average VRE of the model by 16.5 %, and the average AMD by 16.1 %. And compared with the model directly inputting single-wavelength optical information, its average VRE and average AMD decreased by 80.0 % and 27.8 %. This method has clinical application significance for the monitoring of subdural hematomas.

A multi-label image classification method combining multi-stage image semantic information and label relevance

A multi-label image classification method combining multi-stage image semantic information and label relevance

Highlight Model of Underwater Target Acoustic Scattering in the Bistatic System

<p>A highlight model for the target signal in the bistatic system is proposed based on the calculation results of the acoustic field, which means that the target can be equivalent to several highlights according to certain characteristics of the target when calculating the scattering acoustic field. The parameters of the highlight model were given based on the calculation results. Two sets of experiments on the highlight model of the bistatic system were conducted with a fixed transmitter and a target, one being the receiver rotating around the target, and the other being the target rotating around itself at different receiver angles. The comparison of multi angle pseudo color images through experiments and theoretical simulations shows that the scattering highlight model can describe the scattering characteristics of the target to a certain extent. Only geometric highlights were simulated in this article, and it can be foreseen that bistatic system highlight model of scattering waves can be accurately described with a full understanding of the target.</p> <p> </p>

Multi-view Image Fusion Using Ensemble Deep Learning Algorithm For MRI And CT Images

Medical image fusions are crucial elements in image-based health care diagnostics or therapies and generic applications of computer visions. However, the majority of existing methods suffer from noise distortion that affects the overall output. When pictures are distorted by noises, classical fusion techniques perform badly. Hence, fusion techniques that properly maintain information comprehensively from multiple faulty pictures need to be created. This work presents Enhanced Lion Swarm Optimization (ESLO) with Ensemble Deep Learning (EDL) to address the aforementioned issues. The primary steps in this study include image fusions, segmentation, noise reduction, feature extraction, picture classification, and feature selection. Adaptive Median Filters are first used for noise removal in sequence to enhance image quality by eliminating noises. The MRIs and CT images are then segmented using the Region Growing–based k -Means Clustering (RKMC) algorithm to separate the images into their component regions or objects. Images in black and white are divided into image. In the white image, the RKMC algorithm successfully considered the earlier tumour probability. The next step is feature extraction, which is accomplished by using the Modified Principal Component Analysis (MPCA) to draw out the most informative aspects of the images. Then the ELSO algorithm is applied for optimal feature selection, which is computed by best fitness values. After that, multi-view image fusions of multi modal images derive lower-, middle-, and higher-level image contents. It is done by using Deep Convolution Neural Network (DCNN) and the Tissue-Aware Conditional Generative Adversarial Network (TAcGAN) algorithm, which fuses the multi-view features and relevant image features, and it is used for real-time applications. ELSO +EDL algorithm gives better results in terms of accuracy, Peak Signal-To-Noise Ratio (PSNR), and lower Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE) when compared to other existing algorithms.

Multi image super resolution of MRI images using generative adversarial network

Multi image super resolution of MRI images using generative adversarial network

DeepmdQCT: A multitask network with domain invariant features and comprehensive attention mechanism for quantitative computer tomography diagnosis of osteoporosis

In the medical field, the application of machine learning technology in the automatic diagnosis and monitoring of osteoporosis often faces challenges related to domain adaptation in drug therapy research. The existing neural networks used for the diagnosis of osteoporosis may experience a decrease in model performance when applied to new data domains due to changes in radiation dose and equipment. To address this issue, in this study, we propose a new method for multi domain diagnostic and quantitative computed tomography (QCT) images, called DeepmdQCT. This method adopts a domain invariant feature strategy and integrates a comprehensive attention mechanism to guide the fusion of global and local features, effectively improving the diagnostic performance of multi domain CT images. We conducted experimental evaluations on a self-created OQCT dataset, and the results showed that for dose domain images, the average accuracy reached 91%, while for device domain images, the accuracy reached 90.5%. our method successfully estimated bone density values, with a fit of 0.95 to the gold standard. Our method not only achieved high accuracy in CT images in the dose and equipment fields, but also successfully estimated key bone density values, which is crucial for evaluating the effectiveness of osteoporosis drug treatment. In addition, we validated the effectiveness of our architecture in feature extraction using three publicly available datasets. We also encourage the application of the DeepmdQCT method to a wider range of medical image analysis fields to improve the performance of multi-domain images.

Multi visual images fusion approach for metro tunnel defects based on saliency optimization of pixel level defect image features

The multi-vision defect sensing system, lining composed primarily of IRT and RGB cameras, allows for automatic identification and extraction of small surface ailments, greatly enhancing detection efficiency. However, the presence of various issues like train vibration, inconsistent lighting, fluctuations in temperature and humidity leads to the images showing inadequate uniformity in illumination, blurriness, and a decrease in the level of detail. The above issues have led to unsatisfactory fusion processing results for multiple visual images and increased missed detection rates. To address the above-mentioned issue, multi visual images fusion approach for subway tunnel defects based on saliency optimization of pixel level defect image features is proposed. The approach initially analyses the train’s motion status and image blurring conditions. It then eliminates the dynamic blurring in the image. Secondly, Image weights are allocated based on the uniformity of visible light image illumination in the tunnel, as well as real-time temperature and humidity. Finally, image feature extraction and fusion are performed by a U-Net network that integrates channel attention mechanisms. The entire experiment was carried out on a dataset consisting of leakage data from the tunnel lining of Shanghai Metro and tunnel defect data from Beijing Metro. The experimental results demonstrate that this approach improves the image pixel value variation rate by 39.7%, enhances the edge quality by 23%, and outperforms similar approach in terms of average gradient, gradient quality, and sum of difference correlation with improvements of 15.9%, 7.3%, and 26.6% respectively.

Exploration on Classification of Vocal Music Theme Based on Intelligent Multi Image Feature Fusion

Exploration on Classification of Vocal Music Theme Based on Intelligent Multi Image Feature Fusion

Decoding motor imagery based on dipole feature imaging and a hybrid CNN with embedded squeeze-and-excitation block

Motor imagery (MI) decoding is the core of an intelligent rehabilitation system in brain computer interface, and it has a potential advantage by using source signals, which have higher spatial resolution and the same time resolution compared to scalp electroencephalography (EEG). However, how to delve and utilize the personalized frequency characteristic of dipoles for improving decoding performance has not been paid sufficient attention. In this paper, a novel dipole feature imaging (DFI) and a hybrid convolutional neural network (HCNN) with an embedded squeeze-and-excitation block (SEB), denoted as DFI-HCNN, are proposed for decoding MI tasks. EEG source imaging technique is used for brain source estimation, and each sub-band spectrum powers of all dipoles are calculated through frequency analysis and band division. Then, the 3D space information of dipoles is retrieved, and by using azimuthal equidistant projection algorithm it is transformed to a 2D plane, which is combined with nearest neighbor interpolation to generate multi sub-band dipole feature images. Furthermore, a HCNN is designed and applied to the ensemble of sub-band dipole feature images, from which the importance of sub-bands is acquired to adjust the corresponding attentions adaptively by SEB. Ten-fold cross-validation experiments on two public datasets achieve the comparatively higher decoding accuracies of 84.23% and 92.62%, respectively. The experiment results show that DFI is an effective feature representation, and HCNN with an embedded SEB can enhance the useful frequency information of dipoles for improving MI decoding.