Distinctive Features Research Articles

Detection of real-time deep fakes and face forgery in video conferencing employing generative adversarial networks

As facial modification technology advances rapidly, it poses a challenge to methods used to detect fake faces. The advent of deep learning and AI-based technologies has led to the creation of counterfeit photographs that are more difficult to discern apart from real ones. Existing Deep fake detection systems excel at spotting fake content with low visual quality and are easily recognized by visual artifacts. The study employed a unique active forensic strategy Compact Ensemble-based discriminators architecture using Deep Conditional Generative Adversarial Networks (CED-DCGAN), for identifying real-time deep fakes in video conferencing. DCGAN focuses on video-deep fake detection on features since technologies for creating convincing fakes are improving rapidly. As a first step towards recognizing DCGAN-generated images, split real-time video images into frames containing essential elements and then use that bandwidth to train an ensemble-based discriminator as a classifier. Spectra anomalies are produced by up-sampling processes, standard procedures in GAN systems for making large amounts of fake data films. The Compact Ensemble discriminator (CED) concentrates on the most distinguishing feature between the natural and synthetic images, giving the generators a robust training signal. As empirical results on publicly available datasets show, the suggested algorithms outperform state-of-the-art methods and the proposed CED-DCGAN technique successfully detects high-fidelity deep fakes in video conferencing and generalizes well when comparing with other techniques. Python tool is used for implementing this proposed study and the accuracy obtained for proposed work is 98.23 %.

Epistemic Logics for Relevant Reasoners

We present a neighbourhood-style semantic framework for modal epistemic logic modelling agents who process information using relevant logic. The distinguishing feature of the framework in comparison to relevant modal logic is that the environment the agent is situated in is assumed to be a classical possible world. This framework generates two-layered logics combining classical logic on the propositional level with relevant logic in the scope of modal operators. Our main technical result is a general soundness and completeness theorem.

Fusobacterium sphaericum sp. nov. , isolated from a human colon tumor, adheres to colonic epithelial cells and induces IL-8 secretion.

Cancerous tissue is a largely unexplored microbial niche that provides a unique environment for the colonization and growth of specific bacterial communities, and with it, the opportunity to identify novel bacterial species. Here, we report distinct features of a novel Fusobacterium species, F. sphaericum sp. nov. ( Fs ), isolated from primary colon adenocarcinoma tissue. We acquire the complete closed genome and associated methylome of this organism and phylogenetically confirm its classification into the Fusobacterium genus, with F. perfoetens as its closest neighbor. Fs is phenotypically and genetically distinct, with morphological analysis revealing its coccoid shape, that while similar to F. perfoetens is rare for most Fusobacterium members. Fs displays a metabolic profile and antibiotic resistance repertoire consistent with other Fusobacterium species. In vitro, Fs has adherent and immunomodulatory capabilities, as it intimately associates with human colon cancer epithelial cells and promotes IL-8 secretion. Analysis of the prevalence and abundance of Fs in >20,000 human metagenomic samples shows that it is a low-prevalence member within human stool with variable relative abundance, found in both healthy controls and patients with colorectal cancer (CRC). Our study sheds light on a novel bacterial species isolated directly from the human CRC tumor niche, and given its interaction with cancer epithelial cells suggests that its role in human health and disease warrants further investigation.

Development of Extemporaneous Compounding as an Initiating Factor in the Transformation of Pharmaceutical Education: International Experience and Current Trends

INTRODUCTION. Currently, stakeholders across the Russian medical and pharmaceutical community are focusing on the restoration of national pharmaceutical compounding. The Russian pharmaceutical compounding market, with its great potential for growth, requires highly qualified professionals able to ensure the operation of compounding pharmacies and the proper quality of extemporaneous medicinal products. The effectiveness of training for such specialists can be improved by analysing the experience of educational institutions in countries with developed pharmaceutical compounding systems.AIM. This study aimed to analyse the distinguishing features of training for pharmaceutical compounding specialists, as well as determine the main trends in pharmaceutical compounding education in leading international academic institutions.MATERIALS AND METHODS. The study used methods of logical, comparative, structural, and content analysis. The analysis covered 106 pharmacy and pharmacology curricula of the top-25 higher education providers from the QS World University Rankings by Quacquarelli Symonds. The curricula were analysed using open data published online, particularly, on the official websites of the selected universities.RESULTS. The article presents the distinguishing features of pharmaceutical compounding education at different academic qualification levels; these features include the requirements for applicants, time to degree, competence standards, structure of exams, and specifics of students’ professional engagement, with consideration of the national context. The key trends in the education of specialists in pharmaceutical compounding include a decrease in training time for mid-level specialists; a tendency towards concentrating on clinical disciplines due to the expansion of the roles of pharmacy professionals; a growing focus on specialisation in the areas of paediatric, dental, sterile, veterinary, radiopharmaceutical, and orphan drug compounding; increasing use of simulated practice experiences and online learning technologies; and the development of continuing professional education systems.CONCLUSIONS. Pharmaceutical education programmes of leading international educational organisations include disciplines related to extemporaneous compounding. Current international trends—pharmaceutical education with further specialisation in extemporaneous compounding, integration of simulation technologies and online learning into the educational process, and development of continuing professional education—coincide with the development directions of pharmaceutical education in the Russian Federation. Whereas international pharmacy specialists complete additional professional education programmes to gain an additional specialisation in the extemporaneous preparation of paediatric medicinal products, sterile dosage forms, chemotherapy products, radiopharmaceuticals, veterinary products, and dental products, Russian specialists can acquire these additional competencies through further education programmes at the levels of advanced training and residency.

Preparing the Electrical Signal Data of the Heart by Performing Segmentation Based on the Neural Network U-Net

Research on the automated extraction of essential data from an electrocardiography (ECG) recording has been a significant topic for a long time. The main focus of digital processing processes is to measure fiducial points that determine the beginning and end of the P, QRS, and T waves based on their waveform properties. The presence of unavoidable noise during ECG data collection and inherent physiological differences among individuals make it challenging to accurately identify these reference points, resulting in suboptimal performance. This is done through several primary stages that rely on the idea of preliminary processing of the ECG electrical signal through a set of steps (preparing raw data and converting them into files that are read and then processed by removing empty data and unifying the width of the signal at a length of 250 in order to remove noise accurately, and then performing the process of identifying the QRS in the first place and P-T implicitly, and then the task stage is determining the required peak and making a cut based on it. The U-Net pre-trained model is used for deep learning. It takes an ECG signal with a customisable sampling rate as input and generates a list of the beginning and ending points of P and T waves, as well as QRS complexes, as output. The distinguishing features of our segmentation method are its high speed, minimal parameter requirements, and strong generalization capabilities, which are used to create data that can be used in diagnosing diseases or biometric systems.

The anatomy of mastication in a toothless mammal: Morphological characteristics of the temporomandibular joint in the platypus (Ornithorhynchus anatinus)

AbstractThe temporomandibular joint (TMJ) is a distinguishing feature of mammals, and in most mammals includes an articular disc that buffers the loads placed on it by mastication. The disc is well developed in mammals with significant lateral masticatory jaw movements but is absent in toothless mammals, including extant monotremes, although histological studies of developing monotremes have shown rudimentary discs that fail to mature. Platypus (Ornithorhynchus anatinus) grind their food between keratinous pads in the maxillae and lower jaws and are the only edentulate mammals that masticate their food. In this study, we characterize the anatomy of the TMJ of the adult platypus to see if we can reconcile the anatomy, including the absence of the articular disc, with the mandibular movements observed in video recordings. We studied the gross anatomy of the maxillofacial region and the microstructure using microcomputed tomography (micro‐CT) and histological examination. Platypuses had well‐developed masticatory muscles but lacked an articular disc between the mandibular condyle and glenoid fossa. The surface of the glenoid fossa was slightly concave than that of the condylar head was correspondingly slightly convex. The pre‐ and postglenoid processes were not well developed. Micro‐CT showed dense trabecular bone in the anterior part of the condyle, where the lateral pterygoid muscle attached. Histological analysis showed that the surfaces of the condyle and glenoid fossa consisted of dense, avascular and thickened fibrous connective tissue. In addition, well‐developed synovial folds were present. These anatomical characteristics are consistent with both anterior and lateral movements of the mandible, while the thick layer of connective tissue substitutes for a disc by absorbing the mechanical stresses associated with mastication. The failure of the disc primordium to develop cannot be attributed to a lack of muscle development, but the distribution of stresses in the toothless platypus jaw is likely to be different from those in a masticating eutherian.

Therapeutic Exercise Recognition Using a Single UWB Radar with AI-Driven Feature Fusion and ML Techniques in a Real Environment.

Physiotherapy plays a crucial role in the rehabilitation of damaged or defective organs due to injuries or illnesses, often requiring long-term supervision by a physiotherapist in clinical settings or at home. AI-based support systems have been developed to enhance the precision and effectiveness of physiotherapy, particularly during the COVID-19 pandemic. These systems, which include game-based or tele-rehabilitation monitoring using camera-based optical systems like Vicon and Microsoft Kinect, face challenges such as privacy concerns, occlusion, and sensitivity to environmental light. Non-optical sensor alternatives, such as Inertial Movement Units (IMUs), Wi-Fi, ultrasound sensors, and ultrawide band (UWB) radar, have emerged to address these issues. Although IMUs are portable and cost-effective, they suffer from disadvantages like drift over time, limited range, and susceptibility to magnetic interference. In this study, a single UWB radar was utilized to recognize five therapeutic exercises related to the upper limb, performed by 34 male volunteers in a real environment. A novel feature fusion approach was developed to extract distinguishing features for these exercises. Various machine learning methods were applied, with the EnsembleRRGraBoost ensemble method achieving the highest recognition accuracy of 99.45%. The performance of the EnsembleRRGraBoost model was further validated using five-fold cross-validation, maintaining its high accuracy.

Machine Learning and Wearable Technology: Monitoring Changes in Biomedical Signal Patterns during Pre-Migraine Nights.

Migraine is one of the most common neurological disorders, characterized by moderate-to-severe headache episodes. Autonomic nervous system (ANS) alterations can occur at phases of migraine attack. This study investigates patterns of ANS changes during the pre-ictal night of migraine, utilizing wearable biosensor technology in ten individuals. Various physiological, activity-based, and signal processing metrics were examined to train predictive models and understand the relationship between specific features and migraine occurrences. Data were filtered based on specified criteria for nocturnal sleep, and analysis frames ranging from 5 to 120 min were used to improve the diversity of the training sample and investigate the impact of analysis frame duration on feature significance and migraine prediction. Several models, including XGBoost (Extreme Gradient Boosting), HistGradientBoosting (Histogram-Based Gradient Boosting), Random Forest, SVM, and KNN, were trained on unbalanced data and using cost-sensitive learning with a 5:1 ratio. To evaluate the changes in features during pre-migraine nights and nights before migraine-free days, an analysis of variance (ANOVA) was performed. The results showed that the features of electrodermal activity, skin temperature, and accelerometer exhibited the highest F-statistic values and the most significant p-values in the 5 and 10 min frames, which makes them particularly useful for the early detection of migraines. The generalized prediction model using XGBoost and a 5 min analysis frame achieved 0.806 for accuracy, 0.638 for precision, 0.595 for recall, and 0.607 for F1-score. Despite identifying distinguishing features between pre-migraine and migraine-free nights, the performance of the current model suggests the need for further improvements for clinical application.

Advanced hybrid malware identification framework for the Internet of Medical Things, driven by deep learning

AbstractThe Internet of Things (IoT) effortlessly enables communication between items on the World Wide Web and other systems. This extensive use of IoTs has created new services and automation in numerous industries, enhancing the standard of living, especially in healthcare. Internet of Medical Things (IoMT) adoption has been beneficial during pandemic conditions by enabling remote patient monitoring and therapy. Nevertheless, the excessive use of IoMT has raised security concerns as it can compromise critical data. This breach in security can result in an inaccurate diagnosis or violate privacy. This research presents a novel approach to hybrid deep learning‐based detection of malware solutions for the IoT. This study uses RNN‐Bi‐LSTM to detect and extract significant features related to an already existing dataset. The proposed model exhibits a detection accuracy of 98.38% when evaluated using these existing datasets. Statistical tests like Mathew co‐relation and Log Loss also indicated reliability of proposed framework. The distinguished feature of our framework is its ability to combine complex deep learning models for IoMT security, which is of economic and scientific importance. It certainly offers a reliable solution for healthcare applications that rely on real‐time functionality and dependency on IoMT systems.

Non-Newtonian behaviors of ferrofluid Couette–Poiseuille flows in time-varying magnetic fields

We analyze the fully developed Couette–Poiseuille flows of ferrofluids between two parallel flat walls subject to three types of time-varying magnetic fields. In these scenarios, ferrofluids exhibit diverse non-Newtonian characteristics such as distinct flow velocity distribution, apparent viscosity and shear stress compared to ordinary Couette–Poiseuille flows. The influence of spin viscosity is explored first through the solution of the governing equations with zero and non-zero spin viscosities. It shows that although the value of the spin viscosity is very small, its inviscid limit would have great influence over the velocity and spin velocity distributions. The assumption of zero spin viscosity leads to an exaggerated non-Newtonian behavior induced by time-varying magnetic fields in the ferrofluid Couette–Poiseuille flows. Then the solutions of equations with non-zero spin viscosity are utilized to delve into non-Newtonian behaviors of ferrofluid Couette–Poiseuille flow under the application of the three time-varying magnetic fields. The results indicate that negative rotational viscosity will occur if the dimensionless frequency lies in the range 1–10, which is a distinguishing feature compared with Newtonian flows. At this point, non-Newtonian flow induced by magnetic field arises, although this effect is very tiny. Within the same frequency range, reversed tangential stress appears in strong uniform alternating magnetic fields. The minimum negative rotational viscosity may arrive at up to 20 % of the intrinsic viscosity in the rotating magnetic field when the magnetization relaxation time is 4 ms.

A reference learning network for fault diagnosis of rotating machinery under strong noise

The strong noise often masks the fault characteristics of equipment, which reduces the accuracy of fault diagnosis and even leads to the inability of intelligent fault diagnosis algorithms to be applied in industrial environments. This has always been a challenge in the field of mechanical fault diagnosis. As known that equipment failure results from the continuous degradation of the equipment’s state, with the failure state evolving from the healthy state. Considering that both healthy signals and fault signals contain similar noise, this paper proposes a Reference Learning Network (RLNet) model. The model aims to enhance the distinguishing features between healthy and faulty samples through reference units, thereby eliminating the influence of noise on feature distribution. Firstly, the impact of variable speed on the model’s robustness is mitigated using the computed order tracking method. Then, the difference features between healthy samples and a class of fault samples are extracted through the binary classification reference learning unit (RLU). Next, the extracted differential features are used to train the state classifier. Finally, membership weights are employed to effectively combine the feature recognition results, reducing the influence of fault features from mismatched RLUs. The robustness and superiority of the proposed method were verified by comparing it with five other intelligent fault diagnosis methods on the gear and bearing datasets. RLNet is of great significance for the engineering application of intelligent fault diagnosis methods in industrial noise environments.

The COVID-19 thrombus: distinguishing pathological, mechanistic, and phenotypic features and management.

A heightened risk for thrombosis is a hallmark of COVID-19. Expansive clinical experience and medical literature have characterized small (micro) and large (macro) vessel involvement of the venous and arterial circulatory systems. Most events occur in patients with serious or critical illness in the hyperacute (first 1-2weeks) or acute phases (2-4weeks) of SARS-CoV-2 infection. However, thrombosis involving the venous, arterial, and microcirculatory systems has been reported in the subacute (4-8weeks), convalescent (>8-12weeks) and chronic phases (>12weeks) among patients with mild-to-moderate illness. The purpose of the current focused review is to highlight the distinguishing clinical features, pathological components, and potential mechanisms of venous, arterial, and microvascular thrombosis in patients with COVID-19. The overarching objective is to better understand the proclivity for thrombosis, laying a solid foundation for screening and surveillance modalities, preventive strategies, and optimal patient management.

Electronic, optical, and thermoelectric efficiency of novel Li-based ternary chalcogenides: First-principles study

The distinguishing features of Li-based ternary chalcogenides are their high thermal stability and adjustable optical characteristics. The presentcomputational investigations focused on the complex interactions between the optical, thermal, and electronic properties of novel LiSmX2 (X = S, Se) ternary chalcogenides. The calculated formation energy and cohesive energy values confirm the stability of these materials. The phonon dispersion results confirm their thermodynamic stability. The predicted spin-polarized band structure calculation signifies an indirect energy gap in both materials. The negative trend in the ε1(ω), indicates that the response of these materials is more analogous to a metal within the given energy range. The intense peaks in the reflectivity spectra suggest that these materials can be employed as filter against UV radiations. Electronic band structures of LiSmS2 and LiSmSe2 may further improve the noticed linear rise in the thermal conductivity that helps electrons transmit heat more efficiently. Our present study may open the door to deeper investigations into these semiconductors’ optoelectronic and thermoelectric characteristics andpotential use in optoelectronic and thermoelectric devices.

Rationalizing the Mathematics Requirements for a Master of Biostatistics Program: A Case Study and Commentary

Within a 2-year Master of Biostatistics program, we reconsidered the mathematics requirements for admission, and also the premise that the most distinguishing feature of a top candidate is deep exposure to mathematics. Our assessment took place within a broad curriculum review intended to enhance alignment: aligning programmatic goals with job skills, aligning the use of mathematics within our curriculum with programmatic goals, aligning our admission criteria with our curriculum, and aligning our application materials with these admission criteria. We developed a specific list of mathematical skills required by the curriculum, and are revising the application materials to include self-report on applicant's exposure to and functional mastery of those skills. We illustrate how functional mastery is operationally defined. Our criteria for identifying top candidates was broadened to include those with especial skills in analytics, biology and/or communication (i.e., the three conceptual pillars of our program). Deep mathematical training is one of many ways to become a top candidate. Within STEM fields such as biostatistics, and despite the commonly held assumption that admission criteria should emphasize depth of mathematical training, a systematic analysis suggests that this assumption imposes a gratuitous requirement on applicants. Reconsidering this assumption can help remove unnecessary barriers, reduce challenges, and support success for aspiring or emerging biostatisticians from diverse or multifaceted backgrounds. It is one way to contribute to a more inclusive and equitable learning environment in our STEM discipline. We believe similar programs might benefit from performing this type of analysis and reflection.

Machine learning identifies clusters of the normal adolescent spine based on sagittal balance.

This study applied a machine learning semi-supervised clustering approach to radiographs of adolescent sagittal spines from a single pediatric institution to identify patterns of sagittal alignment in the normal adolescent spine. We sought to explore the inherent variability found in adolescent sagittal alignment using machine learning to remove bias and determine whether clusters of sagittal alignment exist. Multiple semi-supervised machine learning clustering algorithms were applied to 111 normal adolescent sagittal spines. Sagittal parameters for resultant clusters were determined. Machine learning analysis found that the spines did cluster into distinct groups with an optimal number of clusters ranging from 3 to 5. We performed an analysis on both 3 and 5-cluster groups. The 3-cluster groups analysis found good consistency between methods with 96 of 111, while the analysis of 5-cluster groups found consistency with 105 of 111 spines. When assessing for differences in sagittal parameters between the groups for both analyses, there were differences in T4-12 TK, L1-S1 LL, SS, SVA, PI-LL mismatch, and TPA. However, the only parameter that was statistically different for all groups was SVA. Based on machine learning, the adolescent sagittal spine alignments do cluster into distinct groups. While there were distinguishing features with TK and LL, the most important parameter distinguishing these groups was SVA. Further studies may help to understand these findings in relation to spinal deformities.

Automatic Face Detection of Farm Images Based on an Enhanced Lightweight Deep Learning Model

In the realm of precise management, artificial intelligence has garnered significant attention and adoption, particularly within the domain of smart agriculture. In modern animal husbandry, animal face detection is conducive to individual identification, expression detection and behavior analysis of animals, and this technological advancement holds immense importance in fostering the advancement of intelligent farming practices. In order to solve the challenge of face detection caused by similar appearance features (color, texture, etc.) and no obvious feature differences between the solid-color goats and sheep in the natural environment, this research introduces a novel approach for face detection by combining the capabilities of YOLOv5 and a convolutional block attention module (CBAM). First, datasets of goats and sheep with different angles, scales and densities were constructed. Second, the basic framework of YOLOv5 was used for object detection. To address the obstacle posed by the limited presence of distinguishing features on the faces of goats and sheep, this study aims to overcome the challenge of extracting informative facial characteristics. The CBAM block was introduced to construct the YOLOv5-CBAM model to improve the feature extraction ability. Finally, 2412 images were selected and divided into training set and verification set according to 8:1. The experimental results of this dataset show that the proposed YOLOv5-CBAM model yielded remarkable results with a precision rate of 0.970, a recall rate of 0.890, a mAP@0.5 score of 0.935, an frames per second (FPS) of 140.845, and a model size of 14.680[Formula: see text]MB. In comparison to other approaches such as Faster R-CNN, SSD, YOLOv3, and YOLOv5, the proposed model demonstrated superior performance in some aspects. In addition, it excelled in both lightweight design and overall effectiveness, and it is well-suited for real-time detection of animal faces in real-world farming settings, ensuring efficient identification and monitoring of animals within practical agricultural environments.

Post-Internet Music: Negotiating Queer Identity in the Digital Era

My paper investigates an electronic music genre that emerged in the 2010s, in conjunction with the rise of attention to identity politics on mainstream platforms. The genre does not have a definite name; however, attempts have been made by critics (see, for example, Waugh 2017) to categorise it under the umbrella term "post-internet music". Such a label encompasses different and often competing sub-genres, all of which nonetheless share an imbrication in internet discourse: these range from Hyperpop, to ‘Deconstructed Club, to ‘Conceptronica’ (Reynolds 2019). As such, the post-internet genre has three distinguishing features: a) many artists operating within it identify both themselves and their music as queer; b) on the sonic side, the genre often comprises complex sound design, loud mixes, a wide stereo field, organic drum samples, pitched-up vocals, and beats that elude conventional time-signatures, so as to sound "deconstructed"; c) the music often makes overt or subtle reference to internet culture, and the digital domain. I seek to place particular attention to the conjuncture between the genre's sonic properties and its ‘queerness’, arguing that the genre is both enabled by, but also enables, internet-mediated identity categories. In this sense, I question what it means to be queer in the contemporary neoliberal era, how this might historically differ from previous elaborations of queerness, and to what extent electronic music - a genre so entrenched in masculinist narratives of technology - can act as a vehicle for counter-normative expression. 

Hybrid attentive prototypical network for few-shot action recognition

Most previous few-shot action recognition works tend to process video temporal and spatial features separately, resulting in insufficient extraction of comprehensive features. In this paper, a novel hybrid attentive prototypical network (HAPN) framework for few-shot action recognition is proposed. Distinguished by its joint processing of temporal and spatial information, the HAPN framework strategically manipulates these dimensions from feature extraction to the attention module, consequently enhancing its ability to perform action recognition tasks. Our framework utilizes the R(2+1)D backbone network, coupling the extraction of integrated temporal and spatial features to ensure a comprehensive understanding of video content. Additionally, our framework introduces the novel Residual Tri-dimensional Attention (ResTriDA) mechanism, specifically designed to augment feature information across the temporal, spatial, and channel dimensions. ResTriDA dynamically enhances crucial aspects of video features by amplifying significant channel-wise features for action distinction, accentuating spatial details vital for capturing the essence of actions within frames, and emphasizing temporal dynamics to capture movement over time. We further propose a prototypical attentive matching module (PAM) built on the concept of metric learning to resolve the overfitting issue common in few-shot tasks. We evaluate our HAPN framework on three classical few-shot action recognition datasets: Kinetics-100, UCF101, and HMDB51. The results indicate that our framework significantly outperformed state-of-the-art methods. Notably, the 1-shot task, demonstrated an increase of 9.8% in accuracy on UCF101 and improvements of 3.9% on HMDB51 and 12.4% on Kinetics-100. These gains confirm the robustness and effectiveness of our approach in leveraging limited data for precise action recognition.

BCLNet: Boundary contrastive learning with gated attention feature fusion and multi-branch spatial-channel reconstruction for land use classification

The fusion of optical and synthetic aperture radar (SAR) images is not only a crucial method for enhancing land use classification tasks but also forms a fundamental basis for the interpretation of multimodal remote sensing imagery. However, the visual differences caused by the distinct nonlinear radiometric properties of optical and SAR images lead to challenges in effectively integrating multimodal features using current land use classification methods, resulting in lower classification accuracy. On one hand, these approaches may be affected by the heterogeneity among different source images, leading to an inability to fully utilize the semantic content of the fused features. On the other hand, the semantic embedding space does not consider the semantic relationships between different pixels from a global perspective (i.e., across the entire dataset). To address the aforementioned challenges, we propose the boundary contrastive learning classification network (BCLNet). In this framework, we introduce the gate attention fusion (GAF) module, which selectively considers distinctiveness and consistency features in optical and SAR imagery, facilitating effective modal representation fusion. Furthermore, we introduce the multi-branch spatial channel reconstruction (MSCR) module for enhanced feature augmentation, operating in both spatial and channel dimensions with a dual-dimensional feature reconstruction selection mechanism for the fused features. Furthermore, we adopt boundary contrast learning to address the global pixel semantic space embedding problem. Our method is evaluated on the WHU-OPT-SAR dataset and a multi-class scene classification dataset (MSD) that we constructed. Compared to state-of-the-art methods, our approach demonstrates improvements in pixel classification overall accuracy (OA) and mean intersection over union (mIOU).

Multi-level interactive fusion network based on adversarial learning for fusion classification of hyperspectral and LiDAR data

With the growing awareness of the limitations of unimodal data, research on joint classification using multimodal remote sensing data has garnered increasing attention. However, existing methods still have certain deficiencies in dealing with feature extraction and feature fusion of multimodal remote sensing data. In order to obtain more accurate classification results, it is crucial to fully exploit and utilize the complementary information in multimodal remote sensing data. In this paper, a multi-level interactive fusion network based on adversarial learning (MLIF-AL) is proposed for hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification. First, the feature extraction part based on generative adversarial network (GAN) utilizes adversarial training between the generator and the discriminator to guide the network to learn more discriminative and distinguished feature representations. Meanwhile, a cross-modal interactive information extraction (CMIIE) module is also designed for the generator encoding part to promote the interaction between multimodal data from a global perspective and realize the full utilization of multimodal complementary information. In addition, in the multi-level feature fusion classification (MLFFC) part, the complementarities between the features learned by the neural networks at each layer are comprehensively utilized to obtain higher-level multimodal fusion semantic information. Finally, the adversarial loss and classification loss are combined for network training. Extensive experiments on three commonly used HSI and LiDAR datasets and comparisons with state-of-the-art methods demonstrate the effectiveness and superiority of the model.