Joint Classification Research Articles

Optimising solder joint inspection in printed circuit boards through X-ray imaging and machine learning integration

ABSTRACT This study combines machine learning (ML) and X-ray imaging to evaluate the health of solder joints in printed circuit boards (PCBs). A convolutional neural network (CNN) served as the base framework, with CNN-LSTM and CNN-CapsNet models added to enhance performance. Pre-training with the CNN facilitated feature extraction, boosting the subsequent performance of LSTM and CapsNet models. The research focused on three objectives: identifying the best ML model for limited datasets, addressing class imbalance in defective solder samples with data augmentation, and using image manipulation to assess model strengths and limitations. Data augmentation significantly improved model accuracies, with CNN, LSTM, and CapsNet achieving 87.05%, 91.29%, and 94.65%, respectively, compared to 76.23%, 83.32%, and 88.05% without augmentation. CapsNet outperformed other models, leveraging its dynamic routing mechanism to preserve feature hierarchies and maintain stable performance. LSTM demonstrated rapid learning through memory cells, while CNNs were prone to overfitting. CapsNet also excelled in balancing classification across solder types, highlighting its ability to handle complex feature relationships. Robustness tests showed CapsNet’s resilience to image transformations like rotation, scaling, and flipping, though extreme deformations remained challenging. These results underscore CapsNet’s potential for accurate and reliable solder joint classification in diverse scenarios.

Enhancing Clinical Insights: New Radiographic Grading for Lumbar Facet Joint Degeneration, A Reliability Study.

Lumbar facet joint diseases can often lead to reduced work efficiency and increased medical costs. As a primary imaging tool in orthopedics, X-rays offer numerous advantages. However, there is no consensus on the classification of lumbar facet joints based on X-ray imaging. This study was conducted for 356 patients between November 2017 and September 2023. A senior radiologist and a senior orthopedic surgeon discussed and established a grading system for lumbar facet joints based on both X-ray and MRI findings. Two double-blind attending spinal surgeons were asked to evaluate and grade the included cases. The intra-rater reliability and inter-rater reliability were evaluated by assessing the weighted kappa (κw) coefficient. The level of inter-rater reliability of MRI for complete agreement was good for Doctor A (κw = 0.792) and Doctor B (κw = 0.869). The inter-rater reliability coefficient for grading lumbar facet joint degeneration based on the radiograph was as follows: for Doctor A (κw = 0.702, session 1; κw = 0.847, session 2) and Doctor B (κw = 0.714, session 1; κw = 0.828, session 2). Moreover, the level of intra-rater reliability based on X-ray for complete agreement was excellent for Doctor A (κw = 0.843) and Doctor B (κw = 0.836). We propose a new grading system for lumbar facet joint degeneration based on X-rays, which serves as a supplement to the Weishaupt and Pathria classifications. This grading system aims to provide clinicians with a more comprehensive understanding of lumbar facet joint degeneration, allowing for the use of a broader range of diagnostic tools to evaluate facet joint degeneration from multiple perspectives. We believe that this grading system can be valuable in assessing potential anatomical changes related to the facet joint, thereby informing modifications to surgical techniques and procedural steps.

Radon Exposure and Gestational Diabetes

Understanding environmental risk factors for gestational diabetes (GD) is crucial for developing preventive strategies and improving pregnancy outcomes. To examine the association of county-level radon exposure with GD risk in pregnant individuals. This multicenter, population-based cohort study used data from the Nulliparous Pregnancy Outcomes Study: Monitoring Mothers-to-Be (nuMoM2b) cohort, which recruited nulliparous pregnant participants from 8 US clinical centers between October 2010 and September 2013. Participants who had pregestational diabetes or were missing data on GD or county-level radon measurements were excluded from the current study. Data were analyzed from September 2023 to January 2024. County-level radon data were created by the Lawrence Berkeley National Laboratory based on the Environmental Protection Agency's short- and long-term indoor home radon assessments. Radon exposure was categorized into 3 groups: less than 1, 1 to less than 2, and 2 or more picocuries (pCi)/L (to convert to becquerels per cubic meter, multiply by 37). Because radon, smoking, and fine particulate matter air pollutants (PM2.5) may share similar biological pathways, participants were categorized by joint classifications of radon level (<2 and ≥2 pCi/L) with smoking status (never smokers and ever smokers) and radon level with PM2.5 level (above or below the median). The main outcome was GD, identified based on glucose tolerance testing and information from medical record abstraction. Multiple logistic regression models were used to assess the association between radon exposure and GD. Among the 9107 participants, mean (SD) age was 27.0 (5.6) years; 3782 of 9101 (41.6%) had ever used tobacco. The mean (SD) county-level radon concentration was 1.6 (0.9) pCi/L, and 382 participants (4.2%) had GD recorded. After adjusting for potential confounders, individuals living in counties with the highest radon level (≥2 pCi/L) had higher odds of developing GD compared with those living in counties with the lowest radon level (<1 pCi/L) (odds ratio [OR], 1.37; 95% CI, 1.02-1.84); after additional adjustment for PM2.5, the OR was 1.36 (95% CI, 1.00-1.86). Elevated odds of GD were also observed in ever smokers living in counties with a higher (≥2 pCi/L) radon level (OR, 2.09; 95% CI, 1.41-3.11) and participants living in counties with higher radon and PM2.5 levels (OR, 1.93; 95% CI, 1.31-2.83), though no statistically significant interactions were observed. This cohort study suggests that higher radon exposure is associated with greater odds of GD in nulliparous pregnant individuals. Further studies are needed to confirm the results and elucidate the underlying mechanisms, especially with individual-level residential radon exposure assessment.

MGMNet: Mutual-Guidance Mechanism for Joint Classification of Multisource Remote Sensing Data

MGMNet: Mutual-Guidance Mechanism for Joint Classification of Multisource Remote Sensing Data

Multi-level Feature Gated Fusion based Spatial and Frequency Domain Attention Network for Joint Classification of Hyperspectral and LiDAR Data

Multi-level Feature Gated Fusion based Spatial and Frequency Domain Attention Network for Joint Classification of Hyperspectral and LiDAR Data

AM2CFN: Assimilation Modality Mapping Guided Crossmodal Fusion Network for HSI and LiDAR Data Joint Classification

AM²CFN: Assimilation Modality Mapping Guided Crossmodal Fusion Network for HSI and LiDAR Data Joint Classification

Joint Classification of Hyperspectral and LiDAR Data Using Hierarchical Multimodal Feature Aggregation-Based Multihead Axial Attention Transformer

Joint Classification of Hyperspectral and LiDAR Data Using Hierarchical Multimodal Feature Aggregation-Based Multihead Axial Attention Transformer

Joint Classification of Hyperspectral and LiDAR Data via Multiprobability Decision Fusion Method

With the development of sensor technology, the sources of remotely sensed image data for the same region are becoming increasingly diverse. Unlike single-source remote sensing image data, multisource remote sensing image data can provide complementary information for the same feature, promoting its recognition. The effective utilization of remote sensing image data from various sources can enhance the extraction of image features and improve the accuracy of feature recognition. Hyperspectral remote sensing (HSI) data and light detection and ranging (LiDAR) data can provide complementary information from different perspectives and are frequently combined in feature identification tasks. However, the process of joint use suffers from data redundancy, low classification accuracy and high time complexity. To address the aforementioned issues and improve feature recognition in classification tasks, this paper introduces a multiprobability decision fusion (PRDRMF) method for the combined classification of HSI and LiDAR data. First, the original HSI data and LiDAR data are downscaled via the principal component–relative total variation (PRTV) method to remove redundant information. In the multifeature extraction module, the local texture features and spatial features of the image are extracted to consider the local texture and spatial structure of the image data. This is achieved by utilizing the local binary pattern (LBP) and extended multiattribute profile (EMAP) for the two types of data after dimensionality reduction. The four extracted features are subsequently input into the corresponding kernel–extreme learning machine (KELM), which has a simple structure and good classification performance, to obtain four classification probability matrices (CPMs). Finally, the four CPMs are fused via a multiprobability decision fusion method to obtain the optimal classification results. Comparison experiments on four classical HSI and LiDAR datasets demonstrate that the method proposed in this paper achieves high classification performance while reducing the overall time complexity of the method.

Microgrid energy optimization management considering consumer segmentation optimization and dynamic time intervals

ABSTRACT To prioritize critical loads and enhance microgrid energy management efficiency, this study introduces a method that combines consumer segmentation optimization and dynamic time intervals. A penalty-based approach is used to minimize unmet load demands, prioritizing energy for critical loads. A joint classification model for multi-type consumers is developed using a convolutional neural network autoencoder and hierarchical clustering. Dynamic time intervals are applied to reduce iterations and improve memory and processor efficiency in solving the energy management model. Simulation tests on a microgrid with distributed generation, battery storage, and consumption units confirm the method’s effectiveness. Results show that configuring dynamic time intervals and segmenting loads by priority yield solutions similar to fixed interval methods. Finally, the energy optimization management effects of the proposed method were compared with those of two latest methods. The comparison results demonstrate that if a microgrid underwent four different disconnection scenarios from the main distribution network, the proposed method saves 23.15%, 23.08%, 23.79%, and 34.61% time to achieve energy optimization management compared with that of the first latest method, and 24.20%, 23.87%, 25.11%, and 36.18% time than that of the second latest method.

A Hierarchical Coarse–Fine Adaptive Fusion Network for the Joint Classification of Hyperspectral and LiDAR Data

Hyperspectral image (HSI) and light detection and ranging (LiDAR) data have gained significant attention due to their excellent complementarity, which can be combined to improve the accuracy of land cover classification. However, there are still many challenges, such as unrelated physical characteristics, different data structures, and a lack of labeled samples. Many methods fail to leverage the full potential of multi-source data, particularly hierarchical complementary information. To address these problems, a hierarchical coarse–fine adaptive (HCFA) fusion network with dynamic convolution and a transformer is proposed for multi-source remote sensing land cover classification. Although fusing hierarchical information can improve the classification accuracy of the model, improper hierarchical feature selection and optimization may negatively affect the classification results. Therefore, a coarse–fine mutual learning strategy is proposed to dynamically fuse hierarchical information. Additionally, the disparity between multi-source data continues to prevent the realization of effective fusion. To tackle this challenge, cross-tokenization and cross-token attention are implemented to enhance information interaction. Furthermore, to improve the model representation with limited computational cost, we combine the advantages of dynamic convolution with a transformer. Validation on three standard datasets demonstrates that HCFA achieves high accuracy with just 1% of the training set while maintaining low computational costs.

AFA–Mamba: Adaptive Feature Alignment with Global–Local Mamba for Hyperspectral and LiDAR Data Classification

The joint classification of hyperspectral imagery (HSI) and LiDAR data is an important task in the field of remote sensing image interpretation. Traditional classification methods, such as support vector machine (SVM) and random forest (RF), have difficulty capturing the complex spectral–spatial–elevation correlation information. Recently, important progress has been made in HSI-LiDAR classification using Convolutional Neural Networks (CNNs) and Transformers. However, due to the large spatial extent of remote sensing images, the vanilla Transformer and CNNs struggle to effectively capture global context. Moreover, the weak misalignment between multi-source data poses challenges for their effective fusion. In this paper, we introduce AFA–Mamba, an Adaptive Feature Alignment Network with a Global–Local Mamba design that achieves accurate land cover classification. It contains two main core designs: (1) We first propose a Global–Local Mamba encoder, which effectively models context through a 2D selective scanning mechanism while introducing local bias to enhance the spatial features of local objects. (2) We also propose an SSE Adaptive Alignment and Fusion (A2F) module to adaptively adjust the relative positions between multi-source features. This module establishes a guided subspace to accurately estimate feature-level offsets, enabling optimal fusion. As a result, our AFA–Mamba consistently outperforms state-of-the-art multi-source fusion classification approaches across multiple datasets.

Advanced welding automation: Intelligent systems for multipass welding in Butt Double V-Groove and Tee Double Bevel configurations

The paper addresses the imperative shift towards automation in welding processes, leveraging advanced technologies such as industrial robotic systems. Focusing on the reconstruction and classification of weld joints, it introduces a methodology for automatic trajectory determination. Utilizing a laser profilometer mounted on the robot, weld joints are reconstructed in three dimensions, and spurious data is filtered out through signal processing. A classification algorithm, integrating signal processing and artificial intelligence, accurately categorizes joint profiles, including V-joints and single bevel T-joints. The proposed intelligent and adaptive system enhances welding automation by analyzing point cloud data from laser scanning to optimize welding trajectories. This study establishes a foundational framework for further refinement and broader application in welding automation.Key Points•Introduction of a methodology for automated trajectory determination in welding processes.•Utilization of laser scanning and signal processing for reconstruction and classification of weld joints.•Implementation of an intelligent and adaptive system to optimize welding trajectories.

Measurements of noise emissions for classification of highway expansion joints

In this paper, we classify expansion joints on Slovenian highways by their potential as noise sources. To measure this potential, we utilized a close proximity measurement chain with additional vibration measurements on the car's undercarriage. By combining these measurements, we were able to amass a large amount of noise relatable data specific to expansion joints. By correlating this data to expansion joint types present on a number of bridges and viaducts, we were able to statistically analyse and rank various joint types regarding their noise emission. Our previous research activities focused on noise complaints associated with specific cases that showed that the problem can be very annoying, even though it was quite localized. Our study opens up a new way to assess expansion joints during regular close proximity measurements on the road network, and simultaneously presents a quantifiable measurement data basis to exhibit the expansion joint noise emission problems near highways.

DMSCA: deep multiscale cross-modal attention network for hyperspectral and light detection and ranging data fusion and joint classification

DMSCA: deep multiscale cross-modal attention network for hyperspectral and light detection and ranging data fusion and joint classification

Predictors of Radial Head Dislocation in Patients with Multiple Hereditary Exostoses.

Background: Radial head dislocation in patients with multiple hereditary exostoses (MHE) is associated with loss of function and cosmetic problems. The treatment of the deformity with radial head dislocation is difficult and the timing of surgical intervention is important. The aim of this study was to evaluate the factors predictive of radial head dislocation in patients with MHE. Methods: Patients diagnosed with forearm deformity due to MHE between 1995 and 2021 were retrospectively evaluated. Radiographic parameters including radial bow (RB), ulnar bow (UB), total radial bow (TRB), total ulnar bow (TUB), percent ulnar length (PUL), ulnar shortening (US), radial articular angle (RAA), modified Masada classification and irregularity of proximal radioulnar joint (PRUJ) of the dislocated group (group D), that is subluxation or dislocation of the radial head, and the located group (group L) were compared. Results: A total of 18 patients and 25 limbs (5 girls and 13 boys) with a mean age of 10.5 years were included. There were significant differences in TUB (22.8° ± 5.6° vs. 10.7° ± 6.5°), PUL (97.5% ± 5.5% vs. 108.2% ± 7.7%) between group D and group L (p < 0.05). Moreover, irregularity of PRUJ on radiographs was more in group D (p < 0.05). Conclusions: It is possible that appropriate radiographic assessment in relation to radial head dislocation may prevent delayed surgical treatment of forearm deformities in MHE. Level of Evidence: Level IV (Diagnostic).

A cross-modal feature aggregation and enhancement network for hyperspectral and LiDAR joint classification

Advancements in Earth observation technologies have greatly enhanced the potential of integrating hyperspectral (HS) images with Light Detection and Ranging (LiDAR) data for land use and land cover classification. Despite this, most existing methods primarily focus on employing deep network layers to extract features from two heterogeneous data modalities, often overlooking a gradual modeling data representation approach from shallow to deep layers. Furthermore, excessive network layers can result in the deterioration of modality-specific features, therefore lowering the classification performance. The paper proposed a novel cross-modal feature aggregation and enhancement network for the joint classification of HSI and LiDAR data. Initially, a cross-modal feature fusion module is developed to exploit spatial scale consistency to complete the interchange and fusion of feature embedding at the pixel level, preserving the original information from the two heterogeneous modalities to a certain degree. Then two straightforward strategies (i.e., addition and concatenation) are employed in the shallow network layers before being sent to the transformer encoder. The former facilitates the model’s ability to discern more subtle distinctions and refine spatial location details. The latter ensures the preservation of information integrity, effectively mitigating the risk of feature loss. Moreover, invertible neural networks and a feature enhancement module are introduced, leveraging the complementary information of HSI and LiDAR data to enhance the detail and texture information extracted in deeper layers. Extensive experiments on Houston2013, Trento, and MUUFL datasets demonstrate that the proposed method outperforms several state-of-the-art models in three evaluation metrics, achieving an accuracy improvement of up to 2%. The proposed model brings new inspirations for HSI and LiDAR classification, which is critical for accurate environmental monitoring, urban planning, and precision agriculture. The source code is publicly accessible at https://github.com/zhangyiyan001/CMFAEN.

Multi-level interactive fusion network based on adversarial learning for fusion classification of hyperspectral and LiDAR data

With the growing awareness of the limitations of unimodal data, research on joint classification using multimodal remote sensing data has garnered increasing attention. However, existing methods still have certain deficiencies in dealing with feature extraction and feature fusion of multimodal remote sensing data. In order to obtain more accurate classification results, it is crucial to fully exploit and utilize the complementary information in multimodal remote sensing data. In this paper, a multi-level interactive fusion network based on adversarial learning (MLIF-AL) is proposed for hyperspectral image (HSI) and light detection and ranging (LiDAR) data fusion classification. First, the feature extraction part based on generative adversarial network (GAN) utilizes adversarial training between the generator and the discriminator to guide the network to learn more discriminative and distinguished feature representations. Meanwhile, a cross-modal interactive information extraction (CMIIE) module is also designed for the generator encoding part to promote the interaction between multimodal data from a global perspective and realize the full utilization of multimodal complementary information. In addition, in the multi-level feature fusion classification (MLFFC) part, the complementarities between the features learned by the neural networks at each layer are comprehensively utilized to obtain higher-level multimodal fusion semantic information. Finally, the adversarial loss and classification loss are combined for network training. Extensive experiments on three commonly used HSI and LiDAR datasets and comparisons with state-of-the-art methods demonstrate the effectiveness and superiority of the model.

Fruit and vegetable intake modifies the association between ultra-processed food and metabolic syndrome

BackgroundThis prospective cohort study aimed to investigate the association between ultra-processed food (UPF) and the risk of metabolic syndrome (MetS), as well as to assess whether fruit and vegetable intake and weight change modify this association.MethodsWe included 1915 healthy participants who participated in the Tehran Lipid and Glucose Study (TLGS), all of whom had complete demographic, anthropometric, and dietary measurements. A validated food frequency questionnaire was used to assess UPF consumption based on the NOVA classification system. MetS was defined according to the Joint Interim Statement. Multivariable adjusted Cox regression was used to estimate hazard ratios (HRs) for MetS events across tertiles of UPF. The effect of fruit and vegetable consumption and weight change on this association was assessed using joint classification by Cox regression.ResultsUFP consumption showed no association with MetS risk after adjusting for confounders. However, after adjustment for dietary fiber, fruits, and vegetables, the highest tertile of UPF consumption was positively linked to MetS risk, compared to the lowest tertile. There was a significant interaction between fruit, vegetable, and dietary fiber intake and UPF consumption concerning the risk of MetS (All P values < 0.05). Among individuals consuming less than 248 g/day of fruit, the risk of MetS increased by 54% (confidence interval: 1.13–2.10) in the highest UPF tertile. Consuming vegetables and dietary fiber below the median (258 g/day and 42.2 g/day, respectively) increased the risk of MetS in the third tertile of UPF. However, consuming vegetables and fiber ≥ median intake, reduced the risk of MetS among those with the lowest UPF consumption. Furthermore, the risk of MetS was observed in the third tertile of UPF consumption among individuals with fruit and vegetable consumption < 537 g/day. UPF consumption was not associated with the risk of MetS in different weight change statuses.ConclusionsConsuming more fruits and vegetables mitigated the adverse effect of UPF on the risk of developing MetS.

Cross-Scene Joint Classification of Multisource Data With Multilevel Domain Adaption Network.

Domain adaption (DA) is a challenging task that integrates knowledge from source domain (SD) to perform data analysis for target domain. Most of the existing DA approaches only focus on single-source-single-target setting. In contrast, multisource (MS) data collaborative utilization has been extensively used in various applications, while how to integrate DA with MS collaboration still faces great challenges. In this article, we propose a multilevel DA network (MDA-NET) for promoting information collaboration and cross-scene (CS) classification based on hyperspectral image (HSI) and light detection and ranging (LiDAR) data. In this framework, modality-related adapters are built, and then a mutual-aid classifier is used to aggregate all the discriminative information captured from different modalities for boosting CS classification performance. Experimental results on two cross-domain datasets show that the proposed method consistently provides better performance than other state-of-the-art DA approaches.

Emulation and detection of physical faults and cyber-attacks on building energy systems through real-time hardware-in-the-loop experiments

The increasing use of remote or mobile access, integrated wearable technologies, data exchange, and cloud-based data analytics in modern smart buildings is steering the building industry towards open communication technologies. The increased connectivity and accessibility could lead to more cyber-attacks in smart buildings. On the other hand, physical faults (e.g., HVAC − heating, ventilation, and air-conditioning faults) may have similar adverse impacts as those from the cyber-attacks on building energy systems, such as occupant discomfort, energy wastage, and equipment downtime. However, current physical behavior-based anomaly detection methods fail to differentiate between cyber-attacks and physical faults in building energy systems. Moreover, the challenge in collecting real-world threat data with ground truth has led researchers to rely on numerical models with user-defined assumptions, which may not accurately reflect real-world conditions due to the lack of in-situ experimental datasets. To address these challenges and gaps, this paper presents a flexible hardware-in-the-loop (HIL) testbed for generating cyber-attack and physical fault datasets and demonstrating threat detection algorithms in a real building automation system (BAS) environment. This testbed combines hardware (i.e., real BAS with local HVAC controllers and a physical network) with software (i.e., high-fidelity models to represent behaviors of building envelope and HVAC energy systems), enabling emulations of realistic threats. Five HIL experiments, including one baseline without any threats, two with physical faults, and two with cyber-attacks, were conducted to generate datasets containing detailed network traffic and system states. A joint classification framework, incorporating a network analyzer and a physical HVAC fault detector, was proposed to automatically detect cyber-physical abnormalities on BAS at both the network and the physical HVAC levels. The network analyzer comprises a conditional random fields (CRF) based command validator and a statistics-based detection strategy. The fault detector employs a weather and schedule-based pattern matching and feature-based principal component analysis (WPM-FPCA) method. Evaluation of the classification using four metrics from the multi-class confusion matrix revealed an average accuracy of 90.2 %, recall of 89.7 %, precision of 88.5 % and F1-score of 89.2 %. These results demonstrate that the proposed joint classification framework can effectively differentiate between specific types of cyber-attacks (e.g., device reinitialization attack, network Denial-of-Service attack) and physical faults (e.g., air handling unit operational fault, cooling coil valve stuck) in real time for improved building energy management.