Information In Fused Image Research Articles

Semi-supervised learning advances species recognition for aquatic biodiversity monitoring

Aquatic biodiversity monitoring relies on species recognition from images. While deep learning (DL) streamlines the recognition process, the performance of these method is closely linked to the large-scale labeled datasets, necessitating manual processing with expert knowledge and consume substantial time, labor, and financial resources. Semi-supervised learning (SSL) offers a promising avenue to improve the performance of DL models by utilizing the extensive unlabeled samples. However, the complex collection environments and the long-tailed class imbalance of aquatic species make SSL difficult to implement effectively. To address these challenges in aquatic species recognition within the SSL scheme, we propose a Wavelet Fusion Network and the Consistency Equilibrium Loss function. The former mitigates the influence of data collection environment by fusing image information at different frequencies decomposed through wavelet transform. The latter improves the SSL scheme by refining the consistency loss function and adaptively adjusting the margin for each class. Extensive experiments are conducted on the large-scale FishNet dataset. As expected, our method improves the existing SSL scheme by up to 9.34% in overall classification accuracy. With the accumulation of image data, the improved SSL method with limited labeled data, shows the potential to advance species recognition for aquatic biodiversity monitoring and conservation.

Multiple feature fusion transformer for modeling penicillin fermentation process with unequal sampling intervals.

The quality prediction of batch processes is an important task in the field of biological fermentation. However, dynamic nonlinearity, unequal sampling intervals, uneven duration, and multiple features of a batch process make this task challenging. Thus, the multiple-feature fusion transformer (MFFT) model is proposed for the time series quality prediction of a batch process. First, the application of sequence-to-sequence architecture enables MFFT to perform a wide range of sequence prediction tasks. Second, the transformer parallel operation model imposes no rigid requirement for the order of sequence input, allowing the model to deal with problems of unequal interval sampling and utilize the sequence information. Third, MFFT integrates a pretrained ResNet50 as a mycelium status classifier for fusing image information into the features. Moreover, a multiple-feature encoding structure is proposed to integrate sampling time and mycelium status. Finally, multiple tasks in penicillin fermentation have shown that MFFT significantly outperforms existing methods for time series prediction.

MCnet: Multiscale visible image and infrared image fusion network

In both civil and military fields, remote sensing image fusion is a popular method for improving images. Multimodal image fusion is typically employed in remote sensing image fusion. Multimodal image fusion acquires synthetic images containing rich image information by fusing image information from different wavebands. Current fusion networks concentrate on fusing features at a single image scale, and the resulting image lacks either spatial or texture features. To address these problems and achieve high-quality fusion, we propose MCnet (multiscale network). MCnet is a new method for multimodal image fusion of visible remote sensing images and infrared remote sensing images in a multiscale framework. Specifically, MCnet first deals with the fusion of image features at different scales in a coarse-to-fine manner. Second, MCnet adaptively provides the amount of information in each image at different scales and images for fine fusion feature supplementation in the coarse fusion stage. In the step of fine fusion, the outcomes of the previous stage will be supplied with the missing characteristics. Finally, we design an objective function with three components: structure loss, region loss, and image quality loss. Structure loss and region loss maintain convergence on overall image similarity and region feature similarity. The image quality constraint partially mitigates the effect of low-quality results on model convergence. MCnet emphasizes the texture features and edge contours of the results, which not only boost the quality of the fusion results but also cause the images to show better discriminative properties. We conduct sufficient experiments based on VIS (visible) and IR (infrared) datasets. The results demonstrate that our proposed model achieves state-of-the-art performance. We also conduct generalizability studies on the proposed method, which likewise yield positive results, demonstrating that MCnet is successful and applicable in a variety of situations.

3D Object Detection from Point Cloud Based on Deep Learning

In order to study the modern 3D object detection algorithm based on deep learning, this paper studies the point-based 3D object detection algorithm, that is, a 3D object detection algorithm that uses multilayer perceptron to extract point features. This paper proposes a method based on point RCNN. A three-stage 3D object detection algorithm improves the accuracy of the algorithm by fusing image information. The algorithm in this paper integrates the information and image information of the three stages well, which improves the information utilization of the whole algorithm. Compared with the traditional 3D target detection algorithm, the structure of the algorithm in this paper is more compact, which effectively improves the utilization of information.

融合图片信息的“标题党”新闻识别研究

融合图片信息的“标题党”新闻识别研究

Maximum a posteriori fusion method based on gradient consistency constraint for multispectral/panchromatic remote sensing images

Multispectral (MS) images with high spatial resolution (HR) can be obtained by fusing MS images and panchromatic (PAN) image, the HR MS images have an important significance in image interpretation and classification, etc. In the present paper, a new image fusion method based on gradient consistency constraint for MS/PAN images is developed. The method is based on maximum a posteriori (MAP) framework. The relationship of desired HR MS images and PAN image is formulated by gradient consistency constraint. Observation model of MS images and the Huber-Markov priori are combined to solve the fused image by gradient descent algorithm. In the proposed method, gradient consistency constraint is introduced, and defect of band number restriction is overcomed in conventional model-based fusion methods. Iterative step for every band is solved adaptively, and spectral characteristics of each band are fully taken into account, so it not only ensures the spectral information fidelity, but also improves the integration degree of spatial information of fused image. The proposed method has been tested using IKONOS and WorldView-2 images. It is compared with GS, AIHS and AMBF fusion methods from both qualitative and quantitative aspects. Experimental results show that the proposed method can better preserve spectral information while enhance spatial resolution, and it has broader applicability and better fusion result than other methods.

A probabilistic framework for image information fusion with an application to mammographic analysis

The recent increased interest in information fusion methods for solving complex problem, such as in image analysis, is motivated by the wish to better exploit the multitude of information, available from different sources, to enhance decision-making. In this paper, we propose a novel method, that advances the state of the art of fusing image information from different views, based on a special class of probabilistic graphical models, called causal independence models. The strength of this method is its ability to systematically and naturally capture uncertain domain knowledge, while performing information fusion in a computationally efficient way. We examine the value of the method for mammographic analysis and demonstrate its advantages in terms of explicit knowledge representation and accuracy (increase of at least 6.3% and 5.2% of true positive detection rates at 5% and 10% false positive rates) in comparison with previous single-view and multi-view systems, and benchmark fusion methods such as naïve Bayes and logistic regression.

Human Pose Regression Through Multiview Visual Fusion

We consider the problem of estimating 3-D human body pose from visual signals within a discriminative framework. It is challenging because there is a wide gap between complex 3-D human motion and planar visual observation, which makes this a severely ill-conditioned problem. In this paper, we focus on three critical factors to tackle human body pose estimation, namely, feature extraction, learning algorithm, and camera utilization. On the feature level, we describe images using the salient interest points represented by scale-invariant feature transform (SIFT)-like descriptors, in which the position, appearance, and local structural information are encoded simultaneously. On the learning algorithm level, we propose to use Gaussian processes and multiple linear (ML) regression to model the mapping between poses and features. Fusing image information from multiple cameras in different views is of great interest to us on the camera level. We make a comprehensive evaluation on the HumanEva database and get two meaningful insights into the three crucial aspects for human pose estimation: 1) although the choice of feature is very important to the problem, once the learning algorithm becomes efficient, the choice of feature is no longer critical, and 2) the impact of information combination from multiple cameras on pose estimation is closely related to not only the quantity of image information, but also its quality. In most cases, it is true that the more information is involved, the better results can be achieved. But when the information quantity is the same, the differences in quality will lead to totally different performance. Furthermore, dense evaluations demonstrate that our approach is an accurate and robust solution to the human body pose estimation problem.