Fusion Model Research Articles

Drug–drug interaction extraction based on multimodal feature fusion by Transformer and BiGRU

Understanding drug–drug interactions (DDIs) plays a vital role in the fields of drug disease treatment, drug development, preventing medical error, and controlling health care-costs. Extracting potential from biomedical corpora is a major complement of existing DDIs. Most existing DDI extraction (DDIE) methods do not consider the graph and structure of drug molecules, which can improve the performance of DDIE. Considering the different advantages of bi-directional gated recurrent units (BiGRU), Transformer, and attention mechanisms in DDIE tasks, a multimodal feature fusion model combining BiGRU and Transformer (BiGGT) is here constructed for DDIE. In BiGGT, the vector embeddings of medical corpora, drug molecule topology graphs, and structure are conducted by Word2vec, Mol2vec, and GCN, respectively. BiGRU and multi-head self-attention (MHSA) are integrated into Transformer to extract the local–global contextual DDIE features, which is important for DDIE. The extensive experiment results on the DDIExtraction 2013 shared task dataset show that the BiGGT-based DDIE method outperforms state-of-the-art DDIE approaches with a precision of 78.22%. BiGGT expands the application of multimodal deep learning in the field of multimodal DDIE.

Hierarchical Fusion of Infrared and Visible Images Based on Channel Attention Mechanism and Generative Adversarial Networks

In order to solve the problem that existing visible and infrared image fusion methods rely only on the original local or global information representation, which has the problem of edge blurring and non-protrusion of salient targets, this paper proposes a layered fusion method based on channel attention mechanism and improved Generative Adversarial Network (HFCA_GAN). Firstly, the infrared image and visible image are decomposed into a base layer and fine layer, respectively, by a guiding filter. Secondly, the visible light base layer is fused with the infrared image base layer by histogram mapping enhancement to improve the contour effect. Thirdly, the improved GAN algorithm is used to fuse the infrared and visible image refinement layer, and the depth transferable module and guided fusion network are added to enrich the detailed information of the fused image. Finally, the multilayer convolutional fusion network with channel attention mechanism is used to correlate the local information of the layered fusion image, and the final fusion image containing contour gradient information and useful details is obtained. TNO and RoadSence datasets are selected for training and testing. The results show that the proposed algorithm retains the global structure features of multilayer images and has obvious advantages in fusion performance, model generalization and computational efficiency.

Based on Deep Learning Model and Flink Streaming Computing Short Term Photovoltaic Power Generation Prediction for Suburban Distribution Network

With the advancement of global energy internet construction, accurate prediction of new energy generation power such as photovoltaic is an important foundation for ensuring the safety and economic working of new power systems. A short-term photovoltaic power generation prediction method for suburban distribution networks based on deep learning model fusion and Flink flow calculation is proposed to address the challenges of complex power grids, diversified disturbance factors, and isolated monitoring points. This method uses Bi directional Long Short Term Memory(BiLSTM) to extract cross sequential nonlinear characteristic of photovoltaic power generation time series data. Compared with standard LSTM, BiLSTM can consider both historical and future information simultaneously, thus extracting richer extracted features from power generation time series data. This method also integrates attention mechanism to capture the importance distribution of historical temporal features for power generation prediction, effectively solving the problem of long-term temporal dependence in standard LSTM models. The Flink streaming computing framework embeds a trained BiLSTM-Attention photovoltaic power generation prediction model, enabling real-time prediction and monitoring analysis of photovoltaic power generation at various monitoring points in the suburban distribution network. This article uses a dataset of a suburban photovoltaic power station for validation, and trains the model with historical power generation data, meteorological factors, weather types, seasons, and other information as inputs. The BiLSTM-Attention fusion model studys the temporal characteristics of power generation, and has high accuracy in predicting short-term photovoltaic power generation in different scenarios. The Flink streaming computing platform can not only process high throughput predicted power data, but also has low time delay.

Feasibility Analysis for Predicting Indian Ocean Bigeye Tuna (Thunnus obesus) Fishing Grounds Based on Temporal Characteristics of FY-3 Microwave Radiation Imager Data

Efficient and accurate fishery forecasting is of great significance in ensuring the efficiency of fishery operations. This paper proposes a fishery forecasting method using a brightness temperature (TB) time series spatial feature extraction and fusion model. Using Indian Ocean bigeye tuna fishery data from 2009 to 2021 as a reference, this paper discusses the feasibility of fishery forecasting using FY-3 Microwave Radiation Imager (MWRI) Level 1 TB data. For this paper, we designed a deep learning network model for radiometer TB time series feature extraction (TimeTB-FishNet) based on the Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), and Attention mechanism. After expanding the dimensions of TB features, the model uses them together with spatiotemporal feature factors (year, month, longitude, and latitude) as features. By adding the GRU and Attention to the CNN, the CNN-GRU-Attention model architecture is established and can extract deep time series spatial features from the data to achieve the best results. In the model validation experiments, the TimeTB-FishNet model performed optimally, with a coefficient of determination (R2) of 0.6643. In the generalization experiments, the R2 also reached 0.6261, with a root mean square error (RMSE) of 46.6031 kg/1000 hook. When the sea surface height (SSH) was introduced, the R2 further reached 0.6463, with a lower RMSE of 45.1318 kg/1000 hook. The experimental results show that the proposed method and model are feasible and effective. The proposed model can directly use enhanced radiometer TB data without relying on lagging ocean environmental product data, performing deep temporal and spatial feature extraction for fishery forecasting. This method can provide a reference for the fishing of bigeye tuna in the Indian Ocean.

Cross-material battery capacity estimation using hybrid-model fusion transfer learning

Evaluating battery health involves navigating the intricate interplay of physical, chemical, and electrochemical processes across multiple scales—a task that becomes even more complex with the introduction of new battery materials. This necessitates substantial development, modeling, and recalibration of boundary conditions. Our study introduces a hybrid fusion model that combines convolutional neural networks (CNNs) with self-attention mechanisms to enhance battery health assessments. In total, three datasets are involved—covering 77 LFP, 20 NMC, and 18 NCA batteries—encompassing over 170,000 cycles across a broad spectrum of battery materials and operational conditions for pre-training the base model and for transfer learning. Our findings reveal that, when transferring aging knowledge from LFP to ternary batteries (NMC and NCA) under diverse chemistries, temperatures, and operational strategies, the model achieved root mean square errors (RMSEs) of 7.47 mAh and 12.4 mAh, mean absolute percentage errors (MAPEs) of 0.67 % and 1.14 %, and coefficients of determination (R2) of 0.922 and 0.918, respectively. These results demonstrate the effectiveness of our hybrid fusion model, which uses deep transfer learning and combines CNNs with self-attention mechanisms to accurately diagnose battery capacity across various types by analyzing short cycle sequences and integrating insights throughout the cell operational history.

A novel endoscopic artificial intelligence system to assist in the diagnosis of autoimmune gastritis: a multicenter study.

Autoimmune gastritis (AIG), distinct from Helicobacter pylori-associated atrophic gastritis (HpAG), is underdiagnosed due to limited awareness. This multicenter study aims to develop a novel endoscopic artificial intelligence (AI) system assisting in AIG diagnosis. Patients diagnosed with AIG, as well as HpAG and non-atrophic gastritis (NAG), were retrospectively enrolled from six centers. Endoscopic images with relevant demographic and medical data, were collected for the development of AI-assisted system, SEER-SCOPE AI, based on multi-site feature fusion model. The diagnostic performance of SEER-SCOPE AI was evaluated in the internal and external datasets. Endoscopists' performance with and without AI support was tested and compared using Mann-Whitney U test. Heatmap analysis was performed to interpret SEER-SCOPE AI. 1 070 patients (294 AIG, 386 HpAG, 390 NAG) with 18 828 endoscopy images were collected. SEER-SCOPE AI achieved strong performance for identifying AIG, with 96.9% sensitivity, 92.2% specificity and an AUROC of 0.990 internally, and 90.3% sensitivity, 93.1% specificity and an AUROC of 0.973 externally. The performance of SEER-SCOPE AI (sensitivity 91.3%) was comparable to experts (87.3%) and significantly outperformed non-experts (70.0%). With AI support, the overall performance of endoscopists was improved (sensitivity: 90.3% [95% CI 86.0%-93.2%] vs. 78.7% [95% CI 73.6%-83.2%], p=0.008). Heatmap analysis revealed consistent focus of SEER-SCOPE AI on regions corresponding to atrophic areas. SEER-SCOPE AI demonstrated expert-level performance in identifying AIG, and enhanced the diagnostic ability of endoscopists. Its application holds promise as a potent endoscopy-assisted tool for guiding biopsy sampling and early detection of AIG.

An ensemble deep learning model for medical image fusion with Siamese neural networks and VGG-19.

Multimodal medical image fusion methods, which combine complementary information from many multi-modality medical images, are among the most important and practical approaches in numerous clinical applications. Various conventional image fusion techniques have been developed for multimodality image fusion. Complex procedures for weight map computing, fixed fusion strategy and lack of contextual understanding remain difficult in conventional and machine learning approaches, usually resulting in artefacts that degrade the image quality. This work proposes an efficient hybrid learning model for medical image fusion using pre-trained and non-pre-trained networks i.e. VGG-19 and SNN with stacking ensemble method. The model leveraging the unique capabilities of each architecture, can effectively preserve the detailed information with high visual quality, for numerous combinations of image modalities in image fusion challenges, notably improved contrast, increased resolution, and lower artefacts. Additionally, this ensemble model can be more robust in the fusion of various combinations of source images that are publicly available from Havard-Medical-Image-Fusion Datasets, GitHub. and Kaggle. Our proposed model performance is superior in terms of visual quality and performance metrics to that of the existing fusion methods in literature like PCA+DTCWT, NSCT, DWT, DTCWT+NSCT, GADCT, CNN and VGG-19.

Four-phase CT lesion recognition based on multi-phase information fusion framework and spatiotemporal prediction module

Multiphase information fusion and spatiotemporal feature modeling play a crucial role in the task of four-phase CT lesion recognition. In this paper, we propose a four-phase CT lesion recognition algorithm based on multiphase information fusion framework and spatiotemporal prediction module. Specifically, the multiphase information fusion framework uses the interactive perception mechanism to realize the channel-spatial information interactive weighting between multiphase features. In the spatiotemporal prediction module, we design a 1D deep residual network to integrate multiphase feature vectors, and use the GRU architecture to model the temporal enhancement information between CT slices. In addition, we employ CT image pseudo-color processing for data augmentation and train the whole network based on a multi-task learning framework. We verify the proposed network on a four-phase CT dataset. The experimental results show that the proposed network can effectively fuse the multi-phase information and model the temporal enhancement information between CT slices, showing excellent performance in lesion recognition.

Differential evolution optimization based ensemble framework for accurate cervical cancer diagnosis

Cervical cancer remains a significant global health concern, demanding accurate and efficient diagnostic approaches for early detection and treatment. However, there are situations where a single deep learning model might not be sufficient to adequately capture the pertinent information required for precise disease prediction from intricate datasets. In this paper, we introduce an innovative ensemble learning framework that leverages the Differential Evolution (DE) algorithm to enhance the detection of cervical cancer. Our innovative approach combines the collective strength of a diverse set of base models using feature fusion techniques to achieve precise and robust classification. We have developed three base models, denoted as Fusion 1, Fusion 2, and Fusion 3, where each fusion model incorporates two pre-trained models using a feature fusion technique. To capture sequential dependencies and improve feature attention, we have integrated ConvLSTM layers and Squeeze-and-Excitation (SE) blocks into these base models. Subsequently, we've amalgamated the predictions of each base model using a weighted aggregation scheme, with the weights optimized using the DE algorithm. Our ensemble framework introduces a novel method for dynamic weight adjustment, utilizing advanced fusion techniques to improve cervical cancer detection accuracy. This approach ensures robust diagnostic performance across cancer cells datasets, enhancing patient outcomes and community health. Our evaluation covers two datasets, SIPaKMeD and Mendeley LBC, demonstrating the remarkable performance and robustness of our ensemble model. It consistently outperforms individual base models and even surpasses state-of-the-art methods, achieving an impressive accuracy rate of 98.96 % on the Mendeley LBC dataset and 95.68 % on the SIPaKMeD dataset. The incorporation of ConvLSTM layers, SE blocks, and DE optimization significantly contributes to the model's exceptional accuracy, its ability to generalize effectively, and its adaptability to diverse scenarios. Additionally, we employ Grad-CAM to visualize how the base models focus on crucial regions for accurate predictions, providing insights into the model's decision-making process. Our ensemble model exhibits outstanding Area under the Curve (AUC) values in Receiver Operating Characteristic (ROC) analysis, further confirming its effectiveness. This research represents a significant advancement in cervical cancer detection, underscoring the potential of ensemble learning techniques optimized through DE for enhancing diagnostic accuracy and advancing patient care.

Bi-TAM-Net framework: fault diagnosis for insulated bearing based on new noise-resistant time-series framework

Abstract Insulated bearings are extensively employed in wind turbines and other applications as essential core parts of high-power frequency control motors. However, the influence of the wind turbine structure makes it difficult to define the wind turbine insulated bearing fault signal extraction, and there is no distinctive index or qualitative formula to describe the bearing fault. In order to solve the above challenges, Bi-TAM-Net framework is developed to diagnose the insulated bearing fault signals and achieve accurate identification of bearing faults. Firstly, the temporal information feature fusion model is created by the Bi-TAM-Net framework using the time-series bearing dataset as the model data input with recursive and chain linking rules in the direction of time-series evolution. By virtue of its parallel approach and deep stacking structure, it comprehensively portrays the bearing fault data and temporally fuses the global feature information, which can effectively improve the classification performance of insulated bearing fault data. Then the self-attention mechanism is introduced into the structure of the designed temporal information fusion model for optimization, which can be modeled in sequences of arbitrary length, extracting the correlation between the features, isolating the unimportant noise information, and strengthening the extraction ability of the proposed framework for important information. Finally, in order to verify the accuracy and superiority of the developed Bi-TAM-Net framework for bearing faults, based on the same dataset, the Bi-TAM-Net framework is compared and analyzed with seven methods such as the advanced TAM-Net model, and the results show that the Bi-TAM-Net framework has better superiority. This study offers the analytical approach with engineering application value for the design and maintenance of insulated bearings used in wind turbines.

Intelligent prediction of rail corrugation evolution trend based on self-attention bidirectional TCN and GRU

Analyzing the evolution trend of rail corrugation using signal processing and deep learning is critical for railway safety, as current traditional methods struggle to capture the complex evolution of corrugation. This present study addresses the challenge of accurately capturing this trend, which relies significantly on expert judgment, by proposing an intelligent prediction method based on self-attention (SA), a bidirectional temporal convolutional network (TCN), and a bidirectional gated recurrent unit (GRU). First, multidomain feature extraction and adaptive feature screening were used to obtain the optimal feature set. These features were then combined with principal component analysis (PCA) and the Mahalanobis distance (MD) method to construct a comprehensive health indicator (CHI) that reflects the evolution of rail corrugation. A bidirectional fusion model architecture was employed to capture the temporal correlations between forward and backward information during corrugation evolution, with SA embedded in the model to enhance the focus on key information. The outcome was a rail corrugation trend prediction network that combined a bidirectional TCN, bidirectional GRU, and SA. Subsequently, a multi-strategy improved crested porcupine optimizer (CPO) algorithm was constructed to automatically obtain the optimal network hyperparameters. The proposed method was validated with on-site rail corrugation data, demonstrating superior predictive performance compared to other advanced methods. In summary, the proposed method can accurately predict the evolution trend of rail corrugation, offering a valuable tool for on-site railway maintenance.

Lightweight Multi-Domain Fusion Model for Through-Wall Human Activity Recognition Using IR-UWB Radar

Impulse radio ultra-wideband (IR-UWB) radar, operating in the low-frequency band, can penetrate walls and utilize its high range resolution to recognize different human activities. Complex deep neural networks have demonstrated significant performance advantages in classifying radar spectrograms of various actions, but at the cost of a substantial computational overhead. In response, this paper proposes a lightweight model named TG2-CAFNet. First, clutter suppression and time–frequency analysis are used to obtain range–time and micro-Doppler feature maps of human activities. Then, leveraging GhostV2 convolution, a lightweight feature extraction module, TG2, suitable for radar spectrograms is constructed. Using a parallel structure, the features of the two spectrograms are extracted separately. Finally, to further explore the correlation between the two spectrograms and enhance the feature representation capabilities, an improved nonlinear fusion method called coordinate attention fusion (CAF) is proposed based on attention feature fusion (AFF). This method extends the adaptive weighting fusion of AFF to a spatial distribution, effectively capturing the subtle spatial relationships between the two radar spectrograms. Experiments showed that the proposed method achieved a high degree of model lightweightness, while also achieving a recognition accuracy of 99.1%.

Feasibility study of opportunistic osteoporosis screening on chest CT using a multi-feature fusion DCNN model

SummaryA multi-feature fusion DCNN model for automated evaluation of lumbar vertebrae L1 on chest combined with clinical information and radiomics permits estimation of volumetric bone mineral density for evaluation of osteoporosis.PurposeTo develop a multi-feature deep learning model based on chest CT, combined with clinical information and radiomics to explore the feasibility in screening for osteoporosis based on estimation of volumetric bone mineral density.MethodsThe chest CT images of 1048 health check subjects were retrospectively collected as the master dataset, and the images of 637 subjects obtained from a different CT scanner were used for the external validation cohort. The subjects were divided into three categories according to the quantitative CT (QCT) examination, namely, normal group, osteopenia group, and osteoporosis group. Firstly, a deep learning–based segmentation model was constructed. Then, classification models were established and selected, and then, an optimal model to build bone density value prediction regression model was chosen.ResultsThe DSC value was 0.951 ± 0.030 in the testing dataset and 0.947 ± 0.060 in the external validation cohort. The multi-feature fusion model based on the lumbar 1 vertebra had the best performance in the diagnosis. The area under the curve (AUC) of diagnosing normal, osteopenia, and osteoporosis was 0.992, 0.973, and 0.989. The mean absolute errors (MAEs) of the bone density prediction regression model in the test set and external testing dataset are 8.20 mg/cm3 and 9.23 mg/cm3, respectively, and the root mean square errors (RMSEs) are 10.25 mg/cm3 and 11.91 mg/cm3, respectively. The R-squared values are 0.942 and 0.923, respectively. The Pearson correlation coefficients are 0.972 and 0.965.ConclusionThe multi-feature fusion DCNN model based on only the lumbar 1 vertebrae and clinical variables can perform bone density three-classification diagnosis and estimate volumetric bone mineral density. If confirmed in independent populations, this automated opportunistic chest CT evaluation can help clinical screening of large-sample populations to identify subjects at high risk of osteoporotic fracture.

DMCCT: Dual-Branch Multi-Granularity Convolutional Cross-Substitution Transformer for Hyperspectral Image Classification

In the field of hyperspectral image classification, deep learning technology, especially convolutional neural networks, has achieved remarkable progress. However, convolutional neural network models encounter challenges in hyperspectral image classification due to limitations in their receptive fields. Conversely, the global modeling capability of Transformers has garnered attention in hyperspectral image classification. Nevertheless, the high computational cost and inadequate local feature extraction hinder its widespread application. In this study, we propose a novel fusion model of convolutional neural networks and Transformers to enhance performance in hyperspectral image classification, namely the dual-branch multi-granularity convolutional cross-substitution Transformer (DMCCT). The proposed model adopts a dual-branch structure to separately extract spatial and spectral features, thereby mitigating mutual interference and information loss between spectral and spatial data during feature extraction. Moreover, a multi-granularity embedding module is introduced to facilitate multi-scale and multi-level local feature extraction for spatial and spectral information. In particular, the improved convolutional cross-substitution Transformer module effectively integrates convolution and Transformer, reducing the complexity of attention operations and enhancing the accuracy of hyperspectral image classification tasks. Subsequently, the proposed method is evaluated against existing approaches using three classical datasets, namely Pavia University, Kennedy Space Center, and Indian Pines. Experimental results demonstrate the efficacy of the proposed method, achieving significant classification results on these datasets with overall classification accuracies of 98.57%, 97.96%, and 96.59%, respectively. These results establish the superiority of the proposed method in the context of hyperspectral image classification under similar experimental conditions.

International Air Cargo Volume Forecasting based on the Stacking Fusion Model

Accurate forecasting of international air cargo volumes is crucial for air cargo carriers, which optimizes operational efficiency and profitability and strengthens market competitiveness and risk response capabilities. This study focuses on the open-source cargo data of over 100 international airlines in India from January 2015 to March 2017. Firstly, the missing data must be dealt with. Then, histograms and probability density plots are used to depict the distribution of the cargo volume and analyze the dynamics of the cargo volume of each company at annual and quarterly levels. Subsequently, key factors affecting cargo volumes are analyzed, with a special focus on the cargo performance of the Top 5 airlines, which is visualized in box plots. Given the time-series nature of cargo data, this study adopts the data sequential partitioning method. It innovatively utilizes the Stacking ensemble learning framework, integrating regression models such as Random Forest, Decision Tree, Extreme Gradient Boosting, and K-Nearest Neighbor as the base learners. The meta-learner is constructed to integrate the outputs of each base model, and the hyperparameters of the model are optimized by grid search and cross-validation. Comparison results show that the Stacking model has a higher R² value and lower MSE than any single base learners, indicating that it is more effective in international air cargo volume prediction. The prediction model based on stacking ensemble learning provides an effective strategy for improving air cargo prediction accuracy.

Dual Stream Encoder–Decoder Architecture with Feature Fusion Model for Underwater Object Detection

Underwater surveillance is an imminent and fascinating exploratory domain, particularly in monitoring aquatic ecosystems. This field offers valuable insights into underwater behavior and activities, which have broad applications across various domains. Specifically, underwater surveillance involves detecting and tracking moving objects within aquatic environments. However, the complex properties of water make object detection a challenging task. Background subtraction is a commonly employed technique for detecting local changes in video scenes by segmenting images into the background and foreground to isolate the object of interest. Within this context, we propose an innovative dual-stream encoder–decoder framework based on the VGG-16 and ResNet-50 models for detecting moving objects in underwater frames. The network includes a feature fusion module that effectively extracts multiple-level features. Using a limited set of images and performing training in an end-to-end manner, the proposed framework yields accurate results without post-processing. The efficacy of the proposed technique is confirmed through visual and quantitative comparisons with eight cutting-edge methods using two standard databases. The first one employed in our experiments is the Underwater Change Detection Dataset, which includes five challenges, each challenge comprising approximately 1000 frames. The categories in this dataset were recorded under various underwater conditions. The second dataset used for practical analysis is the Fish4Knowledge dataset, where we considered five challenges. Each category, recorded in different aquatic settings, contains a varying number of frames, typically exceeding 1000 per category. Our proposed method surpasses all methods used for comparison by attaining an average F-measure of 0.98 on the Underwater Change Detection Dataset and 0.89 on the Fish4Knowledge dataset.

An efficient data fusion model based on Bayesian model averaging for robust water quality prediction using deep learning strategies

Accurate monitoring of dissolved oxygen (DO) levels is critical for stakeholders to effectively safeguard water resources and aquatic ecosystem health. This research presents an innovative data fusion framework based on Bayesian model averaging (BMA) by the combination of several neuroscience models (deep learning methodologies) including multilayer perceptron neural network (MLPNN), recurrent neural network (RNN), convolutional neural network (CNN), gated recurrent unit (GRU), long short-term memory (LSTM), and seasonal autoregressive integrated moving average with exogenous variables (SARIMAX). BMA has the capacity to greatly enhance the results attained by standalone approaches. In this study, two feature selection methods such as mutual information (MI) and recursive feature elimination (RFE) applied to select effective predictors and investigate the importance of each input parameters. The techniques were evaluated based on four different metrics and, finally, to demonstrate the usefulness and effectiveness of the newly implemented strategy, it was applied proficiently at the USGS stations, 01427510 and 02336152 within the USA. The findings from analyzing data based on two stations confirmed that the suggested approach (BMA) outperformed other methods such as MLPNN, RNN, CNN, LSTM, and SARIMAX when it came to predicting the levels of DO on a daily basis. In terms of RMSE, MAE and R2, BMA yielded 0.272 mg/L, 0.216 mg/L, and 0.975 using MI technique and 0.320 mg/L, 0.261 mg/L, and 0.965 using RFE method at 01427510 USGS station, respectively. Similarly, based on RMSE, MAE and R2, BMA produced DO prediction by RMSE = 0.352 mg/L, 0.264 mg/L, and 0.968 by MI approach and RMSE = 0.378 mg/L, 0.282 mg/L, and 0.963 via RFE process at 02336152 USGS station, respectively. After analyzing different combinations of input that obtained from feature selection paradigms, it was observed that the variables with the greatest impact on daily dissolved oxygen levels are the dissolved oxygen levels in the previous time period and the water temperature. On the other hand, the pH and turbidity have minimal influence on the daily DO. Finally, the results of this study confirmed that the BMA tool can be efficiently applied to predict DO concentration based on deep learning model outputs.

Panoramic image quality assessment based on multi-viewport adaptive fusion

Existing panoramic image quality assessment models are relatively independent when extracting local features of each viewport, resulting in high computational complexity and difficulty in characterizing the correlation between viewports using an end-to-end fusion model. To address the above problems, a quality assessment method based on feature sharing and multi-viewport adaptive fusion is proposed. Using a shared backbone network, the viewport segmentation and calculation tasks that are independent of each other in the existing method are converted to the feature domain, so that the local features of the entire image can be extracted with only one feedforward calculation. On this basis, a feature domain viewport segmentation method based on spherical uniform sampling is introduced to ensure that the pixel density of the observation space and the representation space is consistent, and semantic information is used to guide the adaptive fusion of local quality features of each viewport. The linear correlation coefficient and rank correlation coefficient on the CVIQ and OIQA datasets are both above 0.96, which is the best compared with the existing mainstream evaluation methods. Compared with the traditional evaluation method SSIM, its average linear correlation coefficient and average rank correlation coefficient on the two datasets are improved by 9.52%and respectively 8.69%; compared with the latest evaluation method MPFIQA, its average linear correlation coefficient and average rank correlation coefficient are improved by 1.71%and respectively 1.44%.

The severity assessment and nucleic acid turning-negative-time prediction in COVID-19 patients with COPD using a fused deep learning model

BackgroundPrevious studies have shown that patients with pre-existing chronic obstructive pulmonary diseases (COPD) were more likely to be infected with coronavirus disease (COVID-19) and lead to more severe lung lesions. However, few studies have explored the severity and prognosis of COVID-19 patients with different phenotypes of COPD.PurposeThe aim of this study is to investigate the value of the deep learning and radiomics features for the severity evaluation and the nucleic acid turning-negative time prediction in COVID-19 patients with COPD including two phenotypes of chronic bronchitis predominant patients and emphysema predominant patients.MethodsA total of 281 patients were retrospectively collected from Hohhot First Hospital between October 2022 and January 2023. They were divided to three groups: COVID-19 group of 95 patients, COVID-19 with emphysema group of 94 patients, COVID-19 with chronic bronchitis group of 92 patients. All patients underwent chest computed tomography (CT) scans and recorded clinical data. The U-net model was pretrained to segment the pulmonary involvement area on CT images and the severity of pneumonia were evaluated by the percentage of pulmonary involvement volume to lung volume. The 107 radiomics features were extracted by pyradiomics package. The Spearman method was employed to analyze the correlation of the data and visualize it through a heatmap. Then we establish a deep learning model (model 1) and a fusion model (model 2) combined deep learning with radiomics features to predict nucleic acid turning-negative time.ResultsCOVID-19 patients with emphysema was lowest in the lymphocyte count compared to COVID-19 patients and COVID-19 companied with chronic bronchitis, and they have the most extensive range of pulmonary inflammation. The lymphocyte count was significantly correlated with pulmonary involvement and the time for nucleic acid turning negative (r=-0.145, P < 0.05). Importantly, our results demonstrated that model 2 achieved an accuracy of 80.9% in predicting nucleic acid turning-negative time.ConclusionThe pre-existing emphysema phenotype of COPD severely aggravated the pulmonary involvement of COVID-19 patients. Deep learning and radiomics features may provide more information to accurately predict the nucleic acid turning-negative time, which is expected to play an important role in clinical practice.

Enhanced Infant Movement Analysis Using Transformer-Based Fusion of Diverse Video Features for Neurodevelopmental Monitoring

Neurodevelopment is a highly intricate process, and early detection of abnormalities is critical for optimizing outcomes through timely intervention. Accurate and cost-effective diagnostic methods for neurological disorders, particularly in infants, remain a significant challenge due to the heterogeneity of data and the variability in neurodevelopmental conditions. This study recruited twelve parent–infant pairs, with infants aged 3 to 12 months. Approximately 25 min of 2D video footage was captured, documenting natural play interactions between the infants and toys. We developed a novel, open-source method to classify and analyse infant movement patterns using deep learning techniques, specifically employing a transformer-based fusion model that integrates multiple video features within a unified deep neural network. This approach significantly outperforms traditional methods reliant on individual video features, achieving an accuracy of over 90%. Furthermore, a sensitivity analysis revealed that the pose estimation contributed far less to the model’s output than the pre-trained transformer and convolutional neural network (CNN) components, providing key insights into the relative importance of different feature sets. By providing a more robust, accurate and low-cost analysis of movement patterns, our work aims to enhance the early detection and potential prediction of neurodevelopmental delays, whilst providing insight into the functioning of the transformer-based fusion models of diverse video features.