Multilevel Fusion Research Articles

Research on application of multimodal data fusion in intelligent building environment perception

With the rapid development of the building industry, intelligent buildings benefit from its safety, energy saving, environmental protection and integration and other advantages have been widely loved by people, most operators also realize the importance of intelligent buildings to bring people humanized and customized services, and in order to realize the personalized service of the building, multi-modal data fusion is an effective method. On the other hand, in today’s Internet of Things society, many practical applications need to deploy a large number of sensing equipment for data collection and processing, so as to carry out high-quality monitoring of the physical world, but due to the inherent limitations of these hardware equipment and the influence of factors such as the environment, single mode data often cannot be completely and comprehensively monitored to the physical world’s changing characteristics. In this development context, multi-modal data fusion has become a research hotspot in the field of machine learning. Based on this, this paper proposes a one-stage fast object detection model with multi-level fusion of multi-modal features and end-to-end characteristics for building indoor environment perception, and conducts experimental analysis on the performance of the model. The verification results show that the accuracy of the proposed method is 50.7% and the running speed is 0.107 s, which has better performance than the existing detection methods.

Query-Adaptive Late Fusion for Hierarchical Fine-Grained Video-Text Retrieval.

Recently, a hierarchical fine-grained fusion mechanism has been proved effective in cross-modal retrieval between videos and texts. Generally, the hierarchical fine-grained semantic representations (video-text semantic matching is decomposed into three levels including global-event representation matching, action-relation representation matching, and local-entity representation matching) to be fused can work well by themselves for the query. However, in real-world scenarios and applications, existing methods failed to adaptively estimate the effectiveness of multiple levels of the semantic representations for a given query in advance of multilevel fusion, resulting in a worse performance than expected. As a result, it is extremely essential to identify the effectiveness of hierarchical semantic representations in a query-adaptive manner. To this end, this article proposes an effective query-adaptive multilevel fusion (QAMF) model based on manipulating multiple similarity scores between the hierarchical visual and text representations. First, we decompose video-side and text-side representations into hierarchical semantic representations consisting of global-event level, action-relation level, and local-entity level, respectively. Then, the multilevel representation of the video-text pair is aligned to calculate the similarity score for each level. Meanwhile, the sorted similarity score curves of the good semantic representation are different from the inferior ones, which exhibit a "cliff" shape and gradually decline (see Fig. fig1 as an example). Finally, we leverage the Gaussian decay function to fit the tail of the score curve and calculate the area under the normalized sorted similarity curve as the indicator of semantic representation effectiveness, namely, the area of good semantic representation is small, and vice versa. Extensive experiments on three public benchmark video-text datasets have demonstrated that our method consistently outperforms the state-of-the-art (SoTA). A simple demo of QAMF will soon be publicly available on our homepage: https://github.com/Lab-ANT.

A classification framework with multi-level fusion of object-based analysis and convolutional neural network: a case study for land use classification in mining areas

ABSTRACT To enhance the accuracy of land use classification in mining areas, the Object-based Convolutional Neural Network (OCNN) method has been widely used. However, existing researches tend to neglect the importance of decision-level fusion, focusing only on feature-level fusion. This study proposes a new classification framework with Multi-level Fusion of Object-based analysis and CNN (MFOCNN) to achieves high-accuracy land use classification in mining areas. First, Simple Linear Iterative Cluster (SLIC) is employed to generate image objects, which serve as the basic unit for classification. Second, an improved DenseNet is proposed to extract deep features from image patches, which represent image objects, and provide the classification result. Third, handcrafted features including spectral, textural and geometric of the image objects are extracted and fused with the deep features to obtain the classification result with random forest classifier. Finally, the Dempster-Shafer (DS) evidence theory is applied to fuse the two previously described classification results at the decision-level to obtain the final result. Experiments conducted in the mining areas of Erdos using Gaofen-6 images demonstrate that the proposed MFOCNN achieves the best visual performance and accuracy among all tested methods. The MFOCNN, with its feature-level fusion and decision-level fusion, significantly improves the accuracy of land use classification in mining areas. The results suggest that the proposed MFOCNN is a promising method for achieving high-accuracy land use classification in mining areas.

Proximity Between Screw and Endplate of Upper Instrumented Vertebra Using a Cortical Bone Trajectory Screw can Increase Radiographic Adjacent Segment Degeneration Risks in Patients With Lumbar Spinal Stenosis.

Retrospective observational study. To determine the proximity between screw and endplate of the upper instrumented vertebra (UIV) using a cortical bone trajectory (CBT) screw as a predictive factor for radiographic adjacent segment degeneration (ASD) in patients surgically treated with transforaminal lumbar interbody fusion (TLIF) with CBT screws (CBT-TLIF) with lumbar spinal stenosis. The risk factors for radiographic ASD after CBT-TLIF remain unknown. Among patients surgically treated with CBT-TLIF at a single institute, 239 consecutive patients (80 males and 159 females) were enrolled. ASD was defined by the presence of one or more of the following three radiologic criteria on the adjacent segment: >3mm anteroposterior translation, >10° segmental kyphosis, or >50% loss of disc height comparing immediate postoperative and 1-year follow-up radiographs. Clinical and radiological features associated with the development of ASD were retrospectively measured. Univariate and multivariate analyses were performed to identify risk factors associated with radiographic ASD. Radiographic ASD was observed in 71 (29.7%) cases at 1-year postoperative follow-up. The preoperative Pfirrmann grade of the adjacent segment (>grade 2), multi-level fusion (>2 levels), and proximity between the tip of CBT screws and endplate on the UIV were significantly associated with radiographic ASD (OR = 3.98, 95% CI [1.06-15.05], P=0.042 versus OR = 3.03, 95% CI [1.00-9.14], P=0.049 versus OR = 0.53, 95% CI [0.40-0.72], P<0.001). The cut-off value of the distance between the tip of the screw and endplate on UIV for radiographic ASD was approximately 2.5 mm (right-sided CBT screw; cut-off value 2.48mm/ left-sided CBT screw; cut-off value 2.465mm). Radiographic adjacent segment degeneration progression can occur when the cortical trajectory bone screw is close to the endplate of the upper instrumented vertebrae in patients with lumbar spinal stenosis undergoing fusion surgery.

1085 Adjunctive Use of Bone Growth Stimulation in Cervical Spine Fusion in Patients at Risk for Nonunion

INTRODUCTION: Prior failed fusion, multi-level disease, diabetes, nicotine usage, and osteoporosis are factors associated with nonunion, leading to prolonged pain, reduced function, and overall decreased quality of life. Previous studies have shown that pulsed electromagnetic field (PEMF) stimulation can improve fusion rates, particularly in subjects at risk for nonunion. METHODS: Data from a multi-center prospective study of subjects that underwent cervical spine fusion and were treated with PEMF was compared to a “matched” retrospective cohort that had fusion but were not treated with PEMF. The prospective cohort was enrolled based on having 1 or more of the following risk factors for nonunion: prior failed cervical spine fusion, multilevel fusion, diabetes, osteoporosis, or nicotine use. Fusion was assessed at 12 months postop using x-rays and/or CT. A cohort of cervical spine fusion patients that were prescribed PEMF, but did not receive PEMF treatment (e.g., insurance denial) were compared to PEMF treated subjects. RESULTS: At baseline, subjects in the PEMF treated group had a significantly higher average number of risk factors for nonunion (1.8 for treated group vs. 1.1 for control, p &lt; 0.001), and significantly higher average number of surgical levels (2.8 for treated group vs. 2.0 for control, p &lt; 0.005). At 12 months post-op, subjects in the PEMF group (n = 161) had a 90.1% fusion rate, compared to a 65.4% for the control group (n = 26) (p &lt; 0.001). CONCLUSIONS: PEMF stimulation provides significant improvements in cervical spine fusion rates in subjects having risk factors for nonunion. Compared to control subjects, PEMF treated subjects had improved fusion outcomes despite being older, having more risk factors for pseudarthrosis, and undergoing more complex surgeries.

173 Electrical Stimulation Used as an Adjunct to Cervical Spine Fusion in Patients at Risk for Pseudarthrosis

INTRODUCTION: Prior studies show that pulsed electromagnetic field (PEMF) stimulation promotes bone healing and can be a valuable tool to help overcome biological deficiencies in patients at risk. The current study further evaluates the adjunct effect of PEMF in subjects undergoing cervical spinal surgery presenting with risk factors for pseudarthrosis. METHODS: A prospective multicenter clinical trial (NCT 03177473) was conducted at eleven clinical sites. Subjects undergoing cervical spinal fusion surgery with one or more risk factors for pseudarthrosis were identified for enrollment. Risk factors for pseudarthrosis included multilevel fusion, prior failed cervical spine fusion, diabetes, osteoporosis, or smoking. Subjects undergoing cervical spinal fusion were required to wear the PEMF device for 4 hours/day for 6 months post-op. Fusion status was determined at the 12-month visit using anterior/posterior, lateral, and flexion/extension radiographs and computed tomography (without contrast). NDI, EQ-5D, SF-36, and VAS for neck and arm pain were collected as secondary outcome measures. RESULTS: Out of 160 subjects, 144 (90.0%) were graded as fused (all levels) at the 12-month visit. Fusion success was 91.7% (n = 55/60), 89.0% (n = 89/100), and 90.9% (n = 20/22) for subjects with 1, 2+, or 3+ risk factors, respectively. Significant improvements in NDI, SF-36, EQ-5D, VAS-arm and VAS-neck were observed compared to baseline (p &lt; 0.001). A device compliance rate of 85.0% was observed at 3-months and 78.3% at 6-months in subjects who had successful cervical fusion. CONCLUSIONS: The PEMF bone growth stimulator device is a valuable tool to aid in fusion healing, especially in challenging populations. Subjects in this study experienced favorable fusion rates of 90% and significant improvement in patient reported outcomes despite having elevated comorbidity and/or complex surgeries.

The Incidence of Screw Failure in Fenestrated Polyaxial Pedicle Screws vs. Conventional Pedicle Screws in the Treatment of Adolescent Idiopathic Scoliosis (AIS).

Background: Adolescent idiopathic scoliosis (AIS) is a spinal pathology affecting 0.47-5.2% of the population, often requiring surgical intervention to control deformity progression. Posterior spinal instrumentation and fusion with pedicle screw fixation are standard procedures for AIS curve correction; however, implant failure remains a significant complication, especially in multi-level fusions. This retrospective cohort study aims to compare the failure rates between conventional pedicle screws (CPSs) and fenestrated pedicle screws (FPSs) in AIS treatment, with a focus on investigating potential causes of these failures. Methods: This study, conducted from January 2016 to December 2020, involves a two-center retrospective analysis of AIS patients undergoing posterior instrumented fusion. Results: Data from a total of 162 patients (122 females and 40 males) revealed a mean age of 14.95 years (range: 11-18). The CPS group consisted of 80 patients (56 females and 24 males), whereas the FPS group consisted of 82 patients (66 females and 16 males) stratified by Risser grade and Lenke Classification. Radiological assessments, clinical outcomes, and SRS-22 scores were evaluated pre-operatively, at 6 months, and post-operatively (minimum follow-up of 2 years). Conclusions: Fenestrated pedicle screws (FPSs) pose concerns due to their lower mechanical strength compared to solid screws. Understanding their limitations and optimizing their application in AIS treatment is essential.

DeepFake detection based on high-frequency enhancement network for highly compressed content

The DeepFake, which generates synthetic content, has sparked a revolution in the fight against deception and forgery. However, most existing DeepFake detection methods mainly focus on improving detection performance with high-quality data while ignoring low-quality synthetic content that suffers from high compression. To address this issue, we propose a novel High-Frequency Enhancement framework, which leverages a learnable adaptive high-frequency enhancement network to enrich weak high-frequency information in compressed content without uncompressed data supervision. The framework consists of three branches, i.e., the Basic branch with RGB domain, the Local High-Frequency Enhancement branch with Block-wise Discrete Cosine Transform, and the Global High-Frequency Enhancement branch with Multi-level Discrete Wavelet Transform. Among them, the local branch utilizes the Discrete Cosine Transform coefficient and channel attention mechanism to indirectly achieve adaptive frequency-aware multi-spatial attention, while the global branch supplements the high-frequency information by extracting coarse-to-fine multi-scale high-frequency cues and cascade-residual-based multi-level fusion by Discrete Wavelet Transform coefficients. In addition, we design a Two-Stage Cross-Fusion module to effectively integrate all information, thereby greatly enhancing weak high-frequency information in low-quality data. Experimental results on FaceForensics++, Celeb-DF, and OpenForensics datasets show that the proposed method outperforms the existing state-of-the-art methods and can effectively improve the detection performance of DeepFakes, especially on low-quality data. The code is available here.11https://github.com/Gina-YUE/HiFE.

5G Indoor Positioning Error Correction Based on 5G-PECNN.

With the development of the mobile network communication industry, 5G has been widely used in the consumer market, and the application of 5G technology for indoor positioning has emerged. Like most indoor positioning techniques, the propagation of 5G signals in indoor spaces is affected by noise, multipath propagation interference, installation errors, and other factors, leading to errors in 5G indoor positioning. This paper aims to address these issues by first constructing a 5G indoor positioning dataset and analyzing the characteristics of 5G positioning errors. Subsequently, we propose a 5G Positioning Error Correction Neural Network (5G-PECNN) based on neural networks. This network employs a multi-level fusion network structure designed to adapt to the error characteristics of 5G through adaptive gradient descent. Experimental validation demonstrates that the algorithm proposed in this paper achieves superior error correction within the error region, significantly outperforming traditional neural networks.

Low-light images enhancement via a dense transformer network

This paper proposes a dense network composed of an improved Transformer network, which successfully restores low-light images to high-quality normal-light images, alleviating issues such as low brightness, high noise, and missing critical information in low-light images. The entire network architecture is based on the improved Transformer network and builds a dense network with a combination of long and short connections. While retaining the self-attention mechanism of the Transformer network, it achieves multi-level fusion and utilization of shallow and deep features, providing the network with rich image features and enabling the restoration of low-light images to high-quality normal-light images. Additionally, a spatial-domain and frequency-domain combined loss function is designed, considering both pixel-level and frequency domain losses, effectively constraining the image restoration process and avoiding spectral biases. Lastly, a multi-scale hybrid gate feedforward network is designed to replace the traditional feedforward network in the Transformer, facilitating feature selection and forward propagation. These designs effectively enhance the richness of meaningful image features, alleviate spectral biases, and improve the visual quality of low-light images. Experimental results demonstrate the superiority of our method over state-of-the-art networks on various typical image enhancement datasets. Taking the most commonly used low-light dataset LOLv1 as an example, our method achieves improvements of 1.3% and 3.07% in PSNR and SSIM, respectively, compared to the best-performing network, showing favorable qualitative and quantitative evaluation results. The proposed method effectively addresses the issue of insufficiently realistic results in low-light image restoration, providing a reliable reference for practical applications.

Integrating image and gene-data with a semi-supervised attention model for prediction of KRAS gene mutation status in non-small cell lung cancer.

KRAS is a pathogenic gene frequently implicated in non-small cell lung cancer (NSCLC). However, biopsy as a diagnostic method has practical limitations. Therefore, it is important to accurately determine the mutation status of the KRAS gene non-invasively by combining NSCLC CT images and genetic data for early diagnosis and subsequent targeted therapy of patients. This paper proposes a Semi-supervised Multimodal Multiscale Attention Model (S2MMAM). S2MMAM comprises a Supervised Multilevel Fusion Segmentation Network (SMF-SN) and a Semi-supervised Multimodal Fusion Classification Network (S2MF-CN). S2MMAM facilitates the execution of the classification task by transferring the useful information captured in SMF-SN to the S2MF-CN to improve the model prediction accuracy. In SMF-SN, we propose a Triple Attention-guided Feature Aggregation module for obtaining segmentation features that incorporate high-level semantic abstract features and low-level semantic detail features. Segmentation features provide pre-guidance and key information expansion for S2MF-CN. S2MF-CN shares the encoder and decoder parameters of SMF-SN, which enables S2MF-CN to obtain rich classification features. S2MF-CN uses the proposed Intra and Inter Mutual Guidance Attention Fusion (I2MGAF) module to first guide segmentation and classification feature fusion to extract hidden multi-scale contextual information. I2MGAF then guides the multidimensional fusion of genetic data and CT image data to compensate for the lack of information in single modality data. S2MMAM achieved 83.27% AUC and 81.67% accuracy in predicting KRAS gene mutation status in NSCLC. This method uses medical image CT and genetic data to effectively improve the accuracy of predicting KRAS gene mutation status in NSCLC.

DDParcel: Deep Learning Anatomical Brain Parcellation From Diffusion MRI.

Parcellation of anatomically segregated cortical and subcortical brain regions is required in diffusion MRI (dMRI) analysis for region-specific quantification and better anatomical specificity of tractography. Most current dMRI parcellation approaches compute the parcellation from anatomical MRI (T1- or T2-weighted) data, using tools such as FreeSurfer or CAT12, and then register it to the diffusion space. However, the registration is challenging due to image distortions and low resolution of dMRI data, often resulting in mislabeling in the derived brain parcellation. Furthermore, these approaches are not applicable when anatomical MRI data is unavailable. As an alternative we developed the Deep Diffusion Parcellation (DDParcel), a deep learning method for fast and accurate parcellation of brain anatomical regions directly from dMRI data. The input to DDParcel are dMRI parameter maps and the output are labels for 101 anatomical regions corresponding to the FreeSurfer Desikan-Killiany (DK) parcellation. A multi-level fusion network leverages complementary information in the different input maps, at three network levels: input, intermediate layer, and output. DDParcel learns the registration of diffusion features to anatomical MRI from the high-quality Human Connectome Project data. Then, to predict brain parcellation for a new subject, the DDParcel network no longer requires anatomical MRI data but only the dMRI data. Comparing DDParcel's parcellation with T1w-based parcellation shows higher test-retest reproducibility and a higher regional homogeneity, while requiring much less computational time. Generalizability is demonstrated on a range of populations and dMRI acquisition protocols. Utility of DDParcel's parcellation is demonstrated on tractography analysis for fiber tract identification.

Multi-Level fusion graph neural network: Application to PET and CT imaging for risk stratification of head and neck cancer

Background and objectiveRisk stratification of head and neck cancer (HNC) patients is important to settle individualized treatment strategies. This study proposed a multi-level fusion graph neural network (MLF-GNN) that fuses PET/CT image features and clinical data for improved prognosis prediction of HNC patients. MethodsThis study exploited 642 HNC patients from 7 different centers collected from The Cancer Imaging Archive (TCIA), which were arbitrarily separated into 507 patients from 4 centers for training and internal validation, and 135 patients from 3 centers for external testing. We constructed a population graph with edges representing the similarity between patients and the vertices representing radiomics features. Single-modality GNN models and MLF-GNN models by using feature-fusion and network-fusion strategies were constructed for progression-free survival (PFS) prediction, respectively. Model performance was evaluated by the concordance index (C-index), area under the receiver operator characteristic curve (ROC-AUC), and Kaplan–Meier curves. ResultsCompared with the single-modality GNN models and feature-fusion based MLF-GNN model, the network-fusion based MLF-GNN model achieved the highest C-index of 0.788 (95 % CI: 0.727–0.848) and AUC of 0.807 (95 % CI: 0.731–0.870) for PFS prediction in the external testing set. Besides, it also showed good performance on the secondary endpoints of overall survival (OS), recurrence-free survival (RFS), and metastasis-free survival (MFS) in the external testing set, with C-index of 0.800 (95 % CI: 0.729–0.871), 0.823 (95 % CI: 0.763–0.884) and 0.758 (95 % CI: 0.673–0.844), respectively. Subgroup analysis stratified with pathogenic site and treatment type showed that the proposed method has good prediction performance on subgroup risk stratification. ConclusionsThe proposed MLF-GNN model could capture the topological relationships among patients by taking full advantage of multi-modality imaging data and clinical data, which achieved improved prognostic performance and was beneficial to guide individual treatment.

Reduced Bone Density Based on Hounsfield Units After Long-Segment Spinal Fusion with Harrington Rods

BackgroundLong-segment instrumentation, such as Harrington rods, offload vertebrae within the construct, which may result in significant stress shielding of the fused segments. The present study aimed to determine the effects of spinal fusion on bone density by measuring Hounsfield units (HU) throughout the spine in patients with a history of Harrington rod fusion. MethodsPatients with a history of Harrington rod fusion treated at a single academic institution were identified. Mean HU were calculated at five spinal segments for each patient—cranial adjacent mobile segment, cranial fused segment, mid-construct fused segment, caudal fused segment, and caudal adjacent mobile segment. Mean HU for each level were compared using paired-samples t-test with statistical significance defined by p<0.05. Hierarchical multiple regression, including age, gender, body mass index, and time since original fusion, was used to determine predictors of mid-fused segment HUs. ResultsOne hundred patients were included (mean age 55 ±12 years, 62% female). Mean HU for the mid-construct fused segment (110, 95%CI [100–121]) were significantly lower than both the cranial and caudal fused segments (150 and 118, respectively; both p<0.05), as well as both the cranial and caudal adjacent mobile segments (210 and 130, respectively; both p<0.001). Multivariable regression showed mid-construct HU were predicted only by patient age (−2.6HU/year; 95%CI [−3.4, −1.9]; p<0.001) and time since original surgery (−1.4HU/year; 95%CI [−2.6, −0.2]; p=0.02). ConclusionsHU were significantly decreased in the middle of prior long-segment fusion constructs, suggesting multilevel fusion constructs lead to vertebral bone density loss within the construct, potentially from stress shielding.

Individual Tree Species Identification for Complex Coniferous and Broad-Leaved Mixed Forests Based on Deep Learning Combined with UAV LiDAR Data and RGB Images

Automatic and accurate individual tree species identification is essential for the realization of smart forestry. Although existing studies have used unmanned aerial vehicle (UAV) remote sensing data for individual tree species identification, the effects of different spatial resolutions and combining multi-source remote sensing data for automatic individual tree species identification using deep learning methods still require further exploration, especially in complex forest conditions. Therefore, this study proposed an improved YOLOv8 model for individual tree species identification using multisource remote sensing data under complex forest stand conditions. Firstly, the RGB and LiDAR data of natural coniferous and broad-leaved mixed forests under complex conditions in Northeast China were acquired via a UAV. Then, different spatial resolutions, scales, and band combinations of multisource remote sensing data were explored, based on the YOLOv8 model for tree species identification. Subsequently, the Attention Multi-level Fusion (AMF) Gather-and-Distribute (GD) YOLOv8 model was proposed, according to the characteristics of the multisource remote sensing forest data, in which the two branches of the AMF Net backbone were able to extract and fuse features from multisource remote sensing data sources separately. Meanwhile, the GD mechanism was introduced into the neck of the model, in order to fully utilize the extracted features of the main trunk and complete the identification of eight individual tree species in the study area. The results showed that the YOLOv8x model based on RGB images combined with current mainstream object detection algorithms achieved the highest mAP of 75.3%. When the spatial resolution was within 8 cm, the accuracy of individual tree species identification exhibited only a slight variation. However, the accuracy decreased significantly with the decrease of spatial resolution when the resolution was greater than 15 cm. The identification results of different YOLOv8 scales showed that x, l, and m scales could exhibit higher accuracy compared with other scales. The DGB and PCA-D band combinations were superior to other band combinations for individual tree identification, with mAP of 75.5% and 76.2%, respectively. The proposed AMF GD YOLOv8 model had a more significant improvement in tree species identification accuracy than a single remote sensing sources and band combinations data, with a mAP of 81.0%. The study results clarified the impact of spatial resolution on individual tree species identification and demonstrated the excellent performance of the proposed AMF GD YOLOv8 model in individual tree species identification, which provides a new solution and technical reference for forestry resource investigation combined multisource remote sensing data.

Learning Hierarchical Fingerprints via Multi-Level Fusion for Video Integrity and Source Analysis

Learning Hierarchical Fingerprints via Multi-Level Fusion for Video Integrity and Source Analysis

Accurate leukocyte detection based on deformable-DETR and multi-level feature fusion for aiding diagnosis of blood diseases

In standard hospital blood tests, the traditional process requires doctors to manually isolate leukocytes from microscopic images of patients’ blood using microscopes. These isolated leukocytes are then categorized via automatic leukocyte classifiers to determine the proportion and volume of different types of leukocytes present in the blood samples, aiding disease diagnosis. This methodology is not only time-consuming and labor-intensive, but it also has a high propensity for errors due to factors such as image quality and environmental conditions, which could potentially lead to incorrect subsequent classifications and misdiagnosis. Contemporary leukocyte detection methods exhibit limitations in dealing with images with fewer leukocyte features and the disparity in scale among different leukocytes, leading to unsatisfactory results in most instances. To address these issues, this paper proposes an innovative method of leukocyte detection: the Multi-level Feature Fusion and Deformable Self-attention DETR (MFDS-DETR). To tackle the issue of leukocyte scale disparity, we designed the High-level Screening-feature Fusion Pyramid (HS-FPN), enabling multi-level fusion. This model uses high-level features as weights to filter low-level feature information via a channel attention module and then merges the screened information with the high-level features, thus enhancing the model’s feature expression capability. Further, we address the issue of leukocyte feature scarcity by incorporating a multi-scale deformable self-attention module in the encoder and using the self-attention and cross-deformable attention mechanisms in the decoder, which aids in the extraction of the global features of the leukocyte feature maps. The effectiveness, superiority, and generalizability of the proposed MFDS-DETR method are confirmed through comparisons with other cutting-edge leukocyte detection models using the private WBCDD, public LISC and BCCD datasets. Our source code and private WBCCD dataset are available at https://github.com/JustlfC03/MFDS-DETR.

Multi-Level Fusion for Enhanced Host-based Malware Detection in ICT-Enabled Smart Cities

In smart cities, the widespread adoption of Information and Communication Technologies (ICTs) presents both opportunities and challenges for security. While ICTs enable increased productivity, data sharing, and improved citizen services, they also create new vulnerabilities for malicious actors to exploit. This necessitates robust host-based security solutions to protect critical infrastructure and data. This paper proposes a novel multi-level fusion approach for enhanced host-based malware detection in ICT-enabled smart cities. By leveraging diverse data sources and employing advanced fusion techniques, our approach achieves significant improvements in malware detection accuracy, network evaluation, and security analysis compared to existing methods. Specifically, our proposed approach demonstrates a 72.1% malware detection rate across various attack scenarios, 69.7% accuracy in host network evaluation, 82.8% reduction in security analysis error, 75.4% accuracy in network probability detection, and an overall accuracy of 67.2%. These results showcase the potential of multi-level fusion for strengthening host-based security in smart cities. This approach offers several advantages over traditional host-based security solutions. Firstly, it provides more comprehensive threat detection by utilizing multiple data sources. Secondly, it reduces the burden on IT administrators by automating security analysis and decision-making. Finally, it enables continuous improvement through adaptive learning and feedback mechanisms. Overall, our multi-level fusion approach represents a promising advancement in host-based security for ICT-enabled smart cities. It offers significant improvements in accuracy and efficiency, paving the way for a more secure and resilient urban environment.

BEVEFNet: A Multiple Object Tracking Model Based on LiDAR-Camera Fusion

As a crucial task in the field of computer vision, object tracking models are widely used in various application domains, such as autonomous driving. However, existing multiple object tracking methods still face challenges in accurately and efficiently tracking moving multi-targets in real time. This paper presents BEVEFNet, a camera-LiDAR multi-target tracking model based on multistage fusion, which effectively utilizes the semantic information from optical images and the spatial and geometric information from LiDAR data to unify multi-modal features in a shared Bird’s Eye View(BEV) representation space. By leveraging LiDAR data to complement optical images, multi-level fusion is achieved at both the feature and decision levels. The proposed efficient sparse 3D feature extraction network significantly enhances the speed of multiple object tracking by incorporating sparse convolution. Experiments conducted on the nuSences dataset demonstrate that BEVEFNet achieves an AMOTA of 69.7, improving the accuracy of multiple object tracking.

Leveraging Advanced Machine Learning Methods to Enhance Multilevel Fusion Score Level Computations

This research introduces a novel technique for determining numerous fusion score levels that works with many datasets and purposes. Each of the four system pieces works together. These are Feature Engineering, Ensemble Learning, deep neural networks (DNNs), and Transfer Learning. In feature engineering, raw data is totally transformed. This stage stresses the importance of PCA and MI for predictive power. AdaBoost is added during ensemble learning. It repeatedly teaches weak learners and adjusts weights depending on errors to create a strong ensemble model. Weighted input processing, ReLU activation, and dropout layers smoothly integrate DNNs. These reveal minor data patterns and correlations. In transfer learning (fine-tuning), a trained model is modified for the feature-engineered dataset. In comparative testing, the recommended technique had greater accuracy, precision, recall, F1 score, AUC-ROC, and training duration. Efficiency measures reduce reasoning time, memory, parameter count, model size, and energy utilization. Visualizations demonstrate resource consumption, method scores, and reasoning time distribution in research. This mathematical framework improves multilayer fusion score level computations, performs well, and is versatile in many scenarios, making it a good choice for large and diverse datasets.