While pre-trained models, such as large language models, can achieve high performance with minimal fine-tuning, the source datasets used for pre-training often contain irrelevant or blackundant data, which can degrade performance on target tasks. Domain Adaptation Information Gain (DAIG)-based source data selection improves performance by pre-training on source data selected based on rough prior knowledge obtained from target data in advance. However, DAIG’s key component, the transition matrix, lacks flexibility and is limited to handling only single-label classification tasks. To address this limitation, we propose the Generalized DAIG (GDAIG)-guided selection process, a novel framework that extends DAIG to support multi-label classification. GDAIG introduces a soft transition matrix to capture inter-label dependencies and employs binary cross-entropy loss to enable adaptation to multi-label data. By leveraging “rough prior knowledge” from initial training on target data, GDAIG actively selects informative and task-relevant source data for pre-training. Experiments on medical image and general object classification datasets demonstrate that GDAIG consistently outperforms baseline approaches, with particularly significant improvements in scenarios involving label mismatch between source and target domains (partial or no label overlap), where conventional transfer learning methods suffer from noise caused by irrelevant source labels. These results highlight GDAIG’s ability to enhance the effectiveness of pre-trained models through strategic source data selection, thereby optimizing performance for specific target tasks. Our framework goes beyond existing approaches that rely solely on pre-trained models, emphasizing the direct utilization of task-relevant source data. Furthermore, GDAIG provides a practical and effective solution for domains with scarce labeled data, such as medical image analysis. • A GDAIG-guided data selection strategy for multi-label classification is proposed. • GDAIG improves target model performance through task-relevant multi-label data selection. • A probabilistic transition matrix captures inter-label dependencies. • “Rough prior” from target data effectively guides source data pre-training. • GDAIG outperforms conventional baselines across diverse multi-label scenarios.

Neural ensembles, comprising multiple heterogeneous neural networks, show promise in machine learning tasks. Their efficacy depends on architecture design, traditionally relying on deep learning expertise. Neural ensemble architecture search methods aim to automate this process, but face challenges in balancing diversity, performance, and computational efficiency. This study introduces DENE, a differential evolution algorithm for NEAS, addressing these limitations. DENE employs a two-stage framework to generate neural ensembles with a multi-head structure, incorporating a surrogate model-based ranking strategy to reduce computational resources during performance evaluation. A novel diversity measurement function enables DENE to optimize for both accuracy and diversity through multi-objective optimization. Experiments on benchmark image classification datasets compare DENE against state-of-the-art evolutionary neural architecture search and NEAS algorithms. Results demonstrate that DENE efficiently generates highly competitive neural ensembles, outperforming existing methods in performance and search time. This research contributes to automated machine learning by providing a more efficient and effective approach to neural ensemble design, potentially broadening the applicability of these powerful models across various domains.

Intelligent transportation systems (ITS) have experienced an important development in the past decade because of developments in communication, control, and information technology deployed to roads, vehicles, and traffic controller systems. Vehicle form classification plays an essential role in applying ITS owing to its capability for collecting valuable traffic information, providing further development of transport structures, and improving human convenience. Nevertheless, the present service structure implements artificial intelligence (AI) methods with universal patterns for every vehicle. Still, the computational efficiency and needs of deep learning (DL) methods pose difficulties for real-time applications. DL is a useful device for classifying vehicle categories because it can take composite traffic data features and learns from larger data amounts. This manuscript develops a Secure Elliptic Galois Cryptography Framework for Vehicle Image Classification in Intelligent Transportation Systems (SEGCF-VICITS) method. ​The primary aim of the SEGCF-VICITS method is to ensure secure data transmission and intelligent decision-making in ITS environments. Initially, the SEGCF-VICITS model employs the elliptic galois cryptography (EGC) model to provide strong encryption for sensitive vehicular data, utilizing a key that is subsequently used for data decryption. Besides, the SE-DenseNet model is utilized for feature extraction. Additionally, the convolutional sparse autoencoder (CSAE) method is used for vehicle classification. The experimental validation of the SEGCF-VICITS method portrayed a superior accuracy value of 95.48% over existing models under the vehicle image classification dataset.

Coherent optical neural network chip with novel computing model for large-scale matrix-vector multiplication.

Comparative Evaluation of Attention Mechanisms Across CNN Architectures and Image Classification Datasets

CSCL: Bridging the plasticity-stability gap in continuous supervised contrastive learning.

Fine-tuning has emerged as a popular technique in the field of transfer learning, demonstrating remarkable achievements in various data-scarce tasks. The performance of fine-tuning in deep convolutional neural networks depends on the selection of which parameters to fine-tune and freeze. However, it is difficult to determine which parameters in the pre-trained model need to be fine-tuned for a new task. This article proposes a filter-level discrete optimization model to identify the filter subset for fine-tuning, a core step of filter selection coding optimization. Due to the huge search space of the filter fine-tuning problem, we propose a filter interactivity decomposition strategy to find a valid search subspace (a smaller search subspace containing the optimal solution) by dividing the entire filter fine-tuning problem into multiple suboptimization problems. Based on the decomposition strategy, we design a microscale-searching transfer optimization algorithm, which solves each subproblem by searching the valid search subspace instead of the original search space of the filter fine-tuning problem. To verify the validity of the proposed algorithm, extensive experiments are conducted on seven publicly available image classification datasets: Stanford Dogs, MIT Indoors, Caltech 256-30, Caltech 256-60, Aircraft, UCF-101, and Omniglot. Experimental results show that the proposed method significantly improves the fine-tuning accuracy while effectively reducing the filter fine-tuning problem scale. Moreover, the proposed algorithm outperforms the state-of-the-art fine-tuning methods on the fine-tuning problem for transfer learning.

Personalized federated learning (pFL) aims to address data heterogeneity by training client-specific models. However, it faces two critical challenges under few-shot conditions. First, existing methods often overlook the hierarchical structure of neural representations, limiting their ability to balance generalization and personalization. Second, recent approaches incorporate representation-level inductive biases that typically rely on rigid assumptions, such as fixed perturbation patterns or compact class clusters, making them vulnerable to distribution shifts in federated environments. To overcome these limitations, we propose pFedH2A, a novel hierarchical framework incorporating brain-inspired mechanisms, tailored for personalized federated learning in few-shot scenarios. First, we design a dual-branch hypernetwork (DHN) that employs two structurally distinct branches to generate aggregation weights. Each branch is biased toward capturing either low-level shared features or high-level personalized representations, enabling fine-grained personalization by mimicking the brain’s division of perceptual and representational processing. Second, we introduce a relation-aware module that learns an adaptive similarity function for each client, supporting few-shot classification by measuring whether a pair of samples belongs to the same class without relying on rigid prototype assumptions. Extensive experiments on public image classification datasets demonstrate that pFedH2A outperforms existing pFL baselines under few-shot scenarios, validating its effectiveness.

WideTopo: Improving foresight neural network pruning through training dynamics preservation and wide topologies exploration.

Remote sensing land-cover classification can provide valuable data support for natural resource management. Existing classification methods based on graph-neural networks rely mainly on the global features and non-Euclidean structural features of image objects without considering the local features that describe their internal structures and the raster-depth features in the form of Euclidean structures. To this end, this paper presents a multi-scale and deep-feature, remote sensing???image land cover???classification method that embeds raster-depth features into node features and captures multi-scale graph-embedding information from global graphs and subgraphs to fully express image information. The depth-feature map of the image is obtained through a visual geometry Group 16???layer network and integrated into the feature space. The fractal network evolution algorithm is adopted to obtain multi-scale image objects. Global-scale features such as spectral, texture, index, and raster-depth features of the image objects are extracted, and local-scale features (e.g., average degree, average path length, graph diameter, average clustering coefficient, small-world effect) of the subgraphs are extracted to construct multi-scale depth features. The composite global graph structure is constructed by adopting adaptive weights, the graph embeddings are extracted via the graph convolutional network, and the node categories are predicted via SoftMax. For the Gaofen Image Dataset (GID‐15, a public benchmark dataset for land cover classification) and the 2017 China Computer Federation Remote Sensing Image Classification Dataset (CCF 2017, released in the 2017 China Computer Federation Big Data and Computational Intelligence Contest), as compared with the traditional method that considers only the global scale and the single-graph structure, this method improves the overall accuracy by 3.83% and 3.46%, respectively, and increases the kappa coefficient by 0.0681 and 0.0637, respectively, which indicates its effectiveness.

Spiking neural networks (SNNs) have the merit of energy efficiency, and have been widely used for various real-world applications. Similar to other types of neural networks, the performance of SNN is also significantly decided by its architecture. In this paper, we propose an Evolutionary Multi-objective Spiking Neural Architecture Search (EMO-SNAS) method that completely enables the automatic design of SNN architectures with both high performance and low power consumption. To achieve this, we first design a variable-length encoding strategy for SNNs, addressing the issue that traditional encoding strategies need to manually set the depth in advance. Furthermore, we propose an exploitation operator focusing on the local search for the variable-length encoding, as well as an exploration operator focusing on the global search based on the temporal expansion. Based on NSGA-II, EMO-SNAS can greatly balance the performance and power consumption during the architecture design. Experiments on three widely used image classification datasets show that EMO-SNAS can achieve the best among the state-of-the-art methods. Specifically, EMO-SNAS gains 0.45%, 0.26%, and 7.52% in terms of classification accuracy, yet significantly contributes to 33%, 29%, and 12% fewer spike numbers on CIFAR10, CIFAR100, and TinyImageNet datasets. Ablation studies show that the temporal expansion can improve the performance of EMO-SNAS. Moreover, the measurement of power consumption and theoretical convergence of EMO-SNAS are also discussed to justify its component design. In addition, with EMO-SNAS, the impact of initial channels for SNNs is also systemically investigated, based on which a conclusion against existing consensus is achieved. The source code is available at https://github.com/songxt3/EMO-SNAS.

The rapid growth of model scale has necessitated substantial computational resources for fine-tuning. Existing approach such as low-rank adaptation (LoRA) has sought to address the problem of handling the large updated parameters in full fine-tuning (FT). However, LoRA utilize random initialization and optimization of low-rank matrices to approximate updated weights, which can result in suboptimal convergence and an accuracy gap compared to full fine-tuning (FT). To address these issues, we propose low-rank LDU (LoLDU), a parameter-efficient fine-tuning (PEFT) approach that significantly reduces trainable parameters by 2600 times compared to regular PEFT methods while maintaining comparable performance. LoLDU leverages lower-diag-upper (LDU) decomposition to initialize low-rank matrices for faster convergence and nonsingularity. We focus on optimizing the diagonal matrix for scaling transformations. To the best of our knowledge, LoLDU has the fewest parameters among all PEFT approaches. We conducted extensive experiments across 4 instruction-following datasets, six natural language understanding (NLU) datasets, eight image classification datasets, and image generation datasets with multiple model types [LLaMA2, RoBERTa, ViT, and stable diffusion (SD)], providing a comprehensive and detailed analysis. Our open-source code can be accessed at https://anonymous.4open.science/r/LoLDU-B5A6.

Federated Learning (FL), which facilitates collaborative model training and protects users’ privacy, has drawn great interest from the research community. With FL, participants train their models on local data and submit the corresponding updates for aggregation to a server. While concealing the identities of the participants, FL may attract adversaries in order to hamper the underlying model. In this paper, we propose an FL framework, FedDOT, to defend against adversaries performing <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">targeted attacks</i>. FedDOT incorporates two powerful defense algorithms, Maximum Spanning Tree based attacker detection (MSTAD) and Densest graph based attacker detection (Density-AD), which leverage correlation between weight updates and graph theory concepts, maximum spanning tree, and densest graph. With a goal to withstand an overwhelming number of attackers, our algorithms provide strong solutions to aid an FL server, even in overwhelming scenarios where adversaries constitute more than half of the participants. Along with theoretical bounds in correlation space, a rigorous experimental analysis using image classification datasets is carried out to validate the robustness of the FedDOT framework in non-iid settings, which ascertains the superiority of the models against the state-of-the-art methods using a variety of metrics evaluating the accuracy and attack detection rate. With an attack success rate of < 10% for targeted attacks like single-label flipping, multi-label flipping, and backdoor, FedDOT successfully defends against overwhelming adversaries with a marginal accuracy drop of less than 2%.

With camera-equipped phones becoming essential items carried by people, utilizing multi-perspective views (MPV) captured from the surrounding environment has emerged as a promising approach to achieve pervasive localization in indoor environments. This MPV-based localization requires extensive data, necessitating the use of crowdsourcing for data collection and training. In this distributed process, multiple clients can process and share results, raising concerns about privacy breaches. To address this challenge, this paper introduces federated learning (FL) into MPV-based localization, resulting in the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MPVFedLoc</i> algorithm, which facilitates distributed learning without exchanging raw local data. However, FL-based methods often experience reduced accuracy due to data heterogeneity. To overcome this, we propose a model self-supervised federated learning framework within <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MPVFedLoc</i>. This framework integrates self-supervised learning at the model level and incorporates a model self-supervised loss into the local training objective to mitigate the bias between the global and local models. To evaluate the performance of MPV-based localization, we construct a benchmark dataset named TJF_Building and conduct extensive experiments. Additionally, we compare the performance of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MPVFedLoc</i> with three state-of-the-art FL methods in both homogeneous and heterogeneous settings. Experimental results demonstrate the robustness and effectiveness of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MPVFedLoc</i>, particularly in handling heterogeneous scenarios. Further experiments on two typical image classification datasets also highlight the potential of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MPVFedLoc</i> for diverse tasks.

Leukemia is a type of cancer that originates in the bone marrow, causing uncontrolled production of abnormal white blood cells that disrupt normal blood function and weaken the immune system. Manual inspection is time-consuming and error-prone, relying heavily on the expertise and experience of medical professionals. The proposed study presents a hybrid model for classifying leukemia by integrating transfer learning and neutrosophic domain enhancement. Neutrosophic domain transformation splits the RGB channel image into Truth (T), Falsity (F), and Indeterminacy (I) components to address uncertainty, ambiguity, and poor contrast in blood cell representations. This enables the improvement of features more directly linked to leukemia identification. The images are augmented using wavelet sharpening and contrast-limited adaptive histogram equalization (CLAHE) on the T component, total variation minimization (TVM) on the F component, and wavelet shrinkage denoising on the I component. This framework was trained and tested on the Leukemia Blood Cell Image Classification dataset, which included 3,256 peripheral blood smear (PBS) images across 4 classes: Benign, Early, Pre, and Pro. A transfer learning architecture based on MobileNetV2 was used for classification, and training was conducted using a 70:15:15 split for training, validation, and testing, respectively. The proposed neutrosophic-enhanced MobileNetV2 model achieved an overall testing accuracy of 98.36% and a macro F1-score of 0.98, demonstrating significant enhancement in multi-class leukemia classification. The incorporation of the neutrosophic enhancement method significantly improves classifier performance, resulting in higher accuracy without increasing computational power.

Long-tailed image classification faces challenges of data imbalance and poor feature representation for tail classes, leading to biased predictions favoring head classes. While most existing methods rely on deep neural networks (DNNs), they typically require large amounts of training data and lack interpretability. Genetic Programming (GP) has shown promise in few-shot learning but has seldom been investigated in long-tailed image classification, primarily due to its limited ability to handle class imbalance and its tendency for fitness functions to be biased toward head classes. To fill this gap, this paper proposes a multi-expert GP method for long-tailed image classification.We develop three objective functions, each serving as an expert: 1) a longtailed expert focusing on head-class performance; 2) a balanced-class expert that promotes equal class representation; and 3) an inverse long-tailed expert emphasizing tail classes. This triexpert framework enables GP to jointly optimize complementary objectives and learn robust feature representations for both head and tail classes. To further improve classification performance, the evolved GP individuals from the final population are used to train base learners, and their outputs are integrated via a voting-based ensemble model. Experimental results demonstrate that the proposed method outperforms state-of-the-art GP and DNN approaches without pretraining across seven long-tailed image classification datasets.

Medical image analysis remains challenging due to inherent limitations in imaging modalities, where structural aliasing and noise artifacts persistently compromise diagnostic accuracy. While convolutional neural networks (CNNs) and vision transformers (ViTs) have achieved remarkable progress in feature extraction, their inherent sampling mechanisms and spectral biases often exacerbate these high-frequency distortions, leading to suboptimal lesion characterization. To address this critical limitation, we propose Spec-ViT, a novel wavelet-based anti-aliasing Transformer architecture that synergistically integrates adaptive spectral purification with hierarchical attentive learning. The Wavelet Antialiasing Module (WAM) first implements learnable smoothing factor in the wavelet domain to suppress highfrequency artifacts, while preserving clinically relevant lowfrequency structures and fine diagnostic details. Building upon this spectral foundation, the Lightweight Enhanced Attention (LEA) refines feature representations through a dual-path mechanism, coupling channel-spatial attention with global multi-head self-attention to enhance lesion context modeling. Finally, the Smoothed Convolutional Gate (SCG) further sharpens local discriminability through depthwise convolution and adaptive Swish gating, completing a coherent pipeline from frequency-aware purification to global-local attentive analysis. Extensive experiments on five benchmark medical image classification datasets demonstrate that Spec-ViT consistently outperforms both baseline and state-of-the-art methods, achieving up to 84.04% accuracy on the Pediatric Pneumonia Chest X-rays dataset in particular.

Spiking neural networks (SNNs) are the third generation of artificial neural networks, which, thanks to their energy efficiency and sparsity, are ideal for use in resource-constrained environments such as IoT or robotics. However, even they may not meet extreme requirements, leading to the need for inference optimization methods, such as quantization and pruning. Recent studies have already considered the practical application of these methods for spiking neural networks, but they have not focused on the impact of initial optimization parameters on the performance of the compressed model. The goal of this study is to systematize and empirically investigate the impact of quantization and pruning method parameters on the final performance of spiking neural networks. A convolutional SNN (CSNN) architecture based on the Leaky Integrate-and-Fire (LIF) neuron was used for the experiments. The model was tested on three image classification datasets: MNIST, FMNIST, and CIFAR10. Compression was performed using static k-bit quantization methods after training and structured pruning with different coefficients encountered in practical use. The results show that at low compression parameters, SNNs demonstrate insignificant accuracy loss while providing a significant reduction in model size and energy consumption. However, for a more complex dataset, a suboptimal trained model, and extreme compression settings, a sharp and significant deterioration in classification metrics is observed

Many studies have established that the attention mechanism has great potential in improving the performance of Convolutional Neural Networks (CNNs) in image classification problems in recent years. Combining channel and spatial attention modules is one of the different kinds of attention mechanisms that are inspired by the visual perception of the human brain. So far, no paper has considered both parallel and sequential states of combining channel-spatial attention modules, so that while comparing them comprehensively and accurately, it can be definitively said which of them is more optimal in terms of a better balance between efficiency and computational complexity of the model. In this paper, we introduced two new types of channel-spatial attention modules, the Parallel Channel-spatial Attention Module (PCSAM) and the Sequential Channel-spatial Attention Module (SCSAM), to embed in the architecture of any CNN. Each of the proposed attention modules is composed of a channel and spatial attention sub-modules. The Channel Attention Module (CAM) and Spatial Attention Module (SAM) help the network in extracting the channels related to the architecture of the Region of Interest (RoI) and its location in the input feature maps, respectively. We increase the representation power of the attention-based networks by extracting the features using Global Average Pooling (GAP) and Global Maximum Pooling (GMP) in the CAM and SAM. Also, the Dilation Convolution (DC) layer is employed in the structure of the SAM instead of the standard convolution to better focus on the RoI in the feature maps. The PCSAM and SCSAM are implemented in the architecture of the ResNet18 and MobileNetv4 to produce the ResNet18PCSAM, ResNet18SCSAM, MobileNetv4PCSAM, and MobileNetv4SCSAM. All networks are trained and evaluated on three general image classification datasets, the CIFAR-10, CIFAR-100, and Tiny-ImageNet, with the same experimental conditions for 50 epochs. The classification results in the test step show that the MobileNetv4SCSAM has a better efficiency than other architectures on all datasets. It also achieved higher performance than the previous existing channel-spatial attention modules.

This article introduces an automated and unbiased framework for validating architectural hypotheses for neural network models, with a particular focus on Neural Decision Trees (NDTs). The proposed methodology employs Neural Architecture Search (NAS) as an unbiased tool to explore architectural variations and empirically assess theoretical claims. To demonstrate this framework, we investigate a hypothesis found in the literature: that the complexity of decision nodes in NDTs decreases monotonically with tree depth. This assumption, initially motivated by the task of monocular depth estimation, suggests that deeper nodes in the tree require fewer parameters due to simpler split functions.To rigorously test this hypothesis, we conduct a series of NAS campaigns over the CIFAR-10 image classification dataset. The search space parameterizes each node by the number of convolutional blocks and fully connected layers, while all other architectural components are held constant to isolate the effect of node depth. By applying Tree-structured Parzen Estimator (TPE)-based NAS and evaluating over 300 architectures, we quantify complexity metrics across tree levels and analyze their correlations using Spearman’s rank coefficient.The results provide no statistical or visual evidence supporting the hypothesized trend: node complexity does not decrease with depth. Instead, complexity remains nearly constant across levels, regardless of tree depth or search space size. These results suggest that assumptions derived from specific applications may not generalize to other domains, underscoring the importance of empirical validation and careful searchspace design. The presented framework may serve as a foundation for verifying other structural assumptions across various neural network families and applications.