Training Costs Research Articles

Time-series water prediction based on feature clustering fusion and XGboost

Abstract. The time series prediction problem has a very wide range of applications in many fields. Most scholars use the LSTM class of algorithms for prediction. However, this method consumes a significant amount of computational power and presents several challenges. To address this issue, this paper proposes a time series forecasting method based on feature fusion and XGBoost. Specifically, we first utilize holiday information and the K-Means algorithm for feature extraction to expand the feature dimensions of the dataset, and then employ XGBoost as a model for training and prediction. Experiments demonstrate that the method proposed in this paper significantly reduces error compared to other traditional machine learning and deep learning methods, while the training time is much shorter than these methods. For example, compared with LSTM, the MSLE of this model decreases by 1.42%, while the training time is only 0.15% of that of LSTM. This greatly saves on training costs and computational power consumption. This confirms the effectiveness of using machine learning and clustering algorithms in time series prediction and provides new methods and practical application directions for future time series prediction models.

Review of physics-informed machine-learning inversion of geophysical data

We review five types of physics-informed machine-learning (PIML) algorithms for inversion and modeling of geophysical data. Such algorithms use the combination of a data-driven machine-learning (ML) method and the equations of physics to model or invert geophysical data (or both). By incorporating the constraints of physics, PIML algorithms can effectively reduce the size of the solution space for ML models, enabling them to be trained on smaller data sets. This is especially advantageous in scenarios in which data availability may be limited or expensive to obtain. In this review, we restrict the physics to be that from the governing wave equation, either as a constraint that must be satisfied or by using numerical solutions of the wave equation for modeling and inversion. This approach ensures that the resulting models adhere to physical principles while leveraging the power of ML to analyze and interpret complex geophysical data. There are several potential benefits of PIML compared to standard numerical modeling or inversion of seismic data computed by, for example, finite-difference solutions to the wave equation. 1) Empirical tests suggest that PIML algorithms constrained by the physics of wave propagation can sometimes resist getting stuck in a local minima compared with standard full-waveform inversion (FWI). 2) After the weights of the neural network are found by training, then the forward and inverse operations by PIML can be more than several orders of magnitude more efficient than FWI. However, the computational cost for general training can be enormous. 3) If the ML inversion operator [Formula: see text] is locally trained on a small portion of the recorded data [Formula: see text], then there is sometimes no need for millions of training examples that aim for global generalization of [Formula: see text]. The benefit is that the locally trained [Formula: see text] can be used to economically invert the remaining test data [Formula: see text] for the true velocity [Formula: see text], where [Formula: see text] can comprise more than 90% of the recorded data.

Cascade contour-enhanced panoptic segmentation for robotic vision perception

Panoptic segmentation plays a crucial role in enabling robots to comprehend their surroundings, providing fine-grained scene understanding information for robots' intelligent tasks. Although existing methods have made some progress, they are prone to fail in areas with weak textures, small objects, etc. Inspired by biological vision research, we propose a cascaded contour-enhanced panoptic segmentation network called CCPSNet, attempting to enhance the discriminability of instances through structural knowledge. To acquire the scene structure, a cascade contour detection stream is designed, which extracts comprehensive scene contours using channel regulation structural perception module and coarse-to-fine cascade strategy. Furthermore, the contour-guided multi-scale feature enhancement stream is developed to boost the discrimination ability for small objects and weak textures. The stream integrates contour information and multi-scale context features through structural-aware feature modulation module and inverse aggregation technique. Experimental results show that our method improves accuracy on the Cityscapes (61.2 PQ) and COCO (43.5 PQ) datasets while also demonstrating robustness in challenging simulated real-world complex scenarios faced by robots, such as dirty cameras and rainy conditions. The proposed network promises to help the robot perceive the real scene. In future work, an unsupervised training strategy for the network could be explored to reduce the training cost.

Simplify to the Limit! Embedding-less Graph Collaborative Filtering for Recommender Systems

The tremendous positive driving effect of Graph Convolutional Network (GCN) and Graph Contrastive Learning (GCL) for recommender systems has become a consensus. GCN encoders are extensively used in recommendation models for capturing high-order connectivities between users and items, whereas GCL accelerates the training of recommendation tasks by adding extra supervision signals from contrastive objectives. However, little attention has been paid on corresponding theories that are truly tailored to recommendation tasks. From the technical perspective, Collaborative Filtering (CF) is seen as an important factor in recommender systems. It is applied to measure user-user, item-item, and user-item similarities rather than to achieve better clustering or node classification results. Besides, heuristic-based data augmentation may not be hold true in the field of recommender systems as it requires additional training costs and introduces noises that will corrupt the interaction graph structure and the semantic information of nodes. To tackle these limitations, we propose a novel Embedding-less Graph Collaborative Filtering (EGCF) for recommendation, which is tailor-made for the problem mentioned for CF and further simplifies existing solutions. Structurally, it consists of two parts: embedding-less GCN and embedding-less GCL. The former improves user-item affinity by streamlining user-type embeddings and carrying out iterative graph convolution. And the latter utilizes three-type contrastive objectives to directly measure the alignment and the uniformity of users, items, and interaction pairs, respectively, avoiding any type of data augmentation or multi-view construction. Even though EGCF has been extremely streamlined, extensive experimental results on three classical datasets demonstrate the effectiveness of EGCF in terms of recommendation accuracy and training efficiency. The code and used datasets are released at https://github.com/BlueGhostYi/ID-GRec.

Efficient Parameter Optimisation for Quantum Kernel Alignment: A Sub-sampling Approach in Variational Training

Quantum machine learning with quantum kernels for classification problems is a growing area of research. Recently, quantum kernel alignment techniques that parameterise the kernel have been developed, allowing the kernel to be trained and therefore aligned with a specific dataset. While quantum kernel alignment is a promising technique, it has been hampered by considerable training costs because the full kernel matrix must be constructed at every training iteration. Addressing this challenge, we introduce a novel method that seeks to balance efficiency and performance. We present a sub-sampling training approach that uses a subset of the kernel matrix at each training step, thereby reducing the overall computational cost of the training. In this work, we apply the sub-sampling method to synthetic datasets and a real-world breast cancer dataset and demonstrate considerable reductions in the number of circuits required to train the quantum kernel while maintaining classification accuracy.

Knowledge Graph Embedding for Hierarchical Entities Based on Auto-Embedding Size

Knowledge graph embedding represents entities and relations as low-dimensional continuous vectors. Recently, researchers have attempted to leverage the potential semantic connections between entities with hierarchical relationships in the knowledge graph. However, existing knowledge embedding methods that model entity hierarchical structures face the following challenges: (1) Setting the same embedding dimension for entities at different hierarchical levels can lead to overfitting issues and the waste of storage and computational resources; (2) Manually searching the embedding dimension in discrete space makes it easy to miss the optimal parameters and increases training costs. Therefore, we propose a knowledge embedding method, HEAES, that automatically learns the embedding dimensions for entities based on their hierarchical structure. Specifically, we first propose a new modeling method to capture the hierarchical relationships of entities, and we then train the model on a triple classification task. Then, we adaptively learn the pruning threshold to trim the embedding vectors, automatically learning different embedding dimensions for entities at different hierarchical levels. Experiments on the YAGO26K and DB111K datasets verify that introducing entity hierarchical information helps boost the model performance.

MobileNet-CAL: Tomato pest and disease classification method based on transfer learning and attention mechanism

Based on the MobileNetV3 model, this paper introduces the Coordinate attention (CA) attention mechanism and transfer learning to propose the MobileNet-CAL model to solve the problems of insufficient attention to model channel information, long classification training time, high training cost, and weak generalization ability. The model combines data enhancement, attention mechanism and transfer learning technology to improve the classification effect of the model, and updates the fully connected layer at the same time. Finally, the model is optimized by combining the Auxiliary loss function and the ASGD optimizer to improve the classification accuracy of the model, speed up the collection speed, and enhance the robustness of the model. Finally, the model is evaluated from multiple angles such as Accuracy, F1-score, Precision, Recall, and Specificity in the tomato pest and disease leaf dataset. Its Accuracy reaches 98.59% an accuracy of, which is improved by, and by, 4.47%respectively, compared with the original model. Experiments show that MobileNet-CAL can improve the classification accuracy to a certain extent, verifying that the model proposed 4.05%、3.43%、16.57% in 4.13%this paper is effective in the classification task of tomato pest and disease leaves.

Data-Driven Dynamic Security Partition Assessment of Power Systems Based on Symmetric Electrical Distance Matrix and Chebyshev Distance

A rapid dynamic security assessment (DSA) is crucial for online preventive and restoration decision-making. The deep learning-based DSA models have high efficiency and accuracy. However, the complex model structure and high training cost make them hard to update quickly. This paper proposes a dynamic security partition assessment method, aiming to develop accurate and incrementally updated DSA models with simple structures. Firstly, the power grid is self-adaptively partitioned into several local regions based on the mean shift algorithm. The input of the mean shift algorithm is a symmetric electrical distance matrix, and the distance metric is the Chebyshev distance. Secondly, high-level features of operating conditions are extracted based on the stacked denoising autoencoder. The symmetric electrical distance matrix is modified to represent fault locations in local regions. Finally, DSA models are constructed for fault locations in each region based on the radial basis function neural network (RBFNN) and Chebyshev distance. An online incremental updating strategy is designed to enhance the model adaptability. With the simulation software PSS/E 33.4.0, the proposed dynamic security partition assessment method is verified in a simplified provincial system and a large-scale practical system in China. Test results demonstrate that the Chebyshev distance can improve the partition quality of the mean shift algorithm by approximately 50%. The RBFNN-based partition assessment model achieves an accuracy of 98.96%, which is higher than the unified assessment with complex models. The proposed incremental updating strategy achieves an accuracy of over 98% and shortens the updating time to 30 s, which can meet the efficiency of online application.

State-of-the-Art Results with the Fashion-MNIST Dataset

In September 2024, the Fashion-MNIST dataset will be 7 years old. Proposed as a replacement for the well-known MNIST dataset, it continues to be used to evaluate machine learning model architectures. This paper describes new results achieved with the Fashion-MNIST dataset using classical machine learning models and a relatively simple convolutional network. We present the state-of-the-art results obtained using the CNN-3-128 convolutional network and data augmentation. The developed CNN-3-128 model containing three convolutional layers achieved an accuracy of 99.65% in the Fashion-MNIST test image set. In addition, this paper presents the results of computational experiments demonstrating the dependence between the number of adjustable parameters of the convolutional network and the maximum acceptable classification quality, which allows us to optimise the computational cost of model training.

Fine Granularity Is Critical for Intelligent Neural Network Pruning.

Neural network pruning is a popular approach to reducing the computational costs of training and/or deploying a network and aims to do so while minimizing accuracy loss. Pruning methods that remove individual weights (fine granularity) can remove more total network parameters before reaching a given degree of accuracy loss, while methods that preserve some or all of a network's structure (coarser granularity, such as pruning channels from a CNN) take better advantage of hardware and software optimized for dense matrix computations. We compare intelligent iterative pruning using several different criteria sampled from the literature against random pruning at initialization across multiple granularities on two different architectures and three image classification tasks. Our work is the first direct and comprehensive investigation of the relationship between granularity and the efficacy of intelligent pruning relative to a random-pruning baseline. We find that the accuracy advantage of intelligent over random pruning decreases dramatically as granularity becomes coarser, with minimal advantage for intelligent pruning at granularity coarse enough to fully preserve network structure. For instance, at pruning rates where random pruning leaves ResNet-20 at 85.0% test accuracy on CIFAR-10 after 30,000 training iterations, intelligent weight pruning with the best-in-context criterion leaves it at about 90.0% accuracy (on par with the unpruned network), kernel pruning leaves it at about 86.5%, and channel pruning leaves it at about 85.5%. Our results suggest that compared to coarse pruning, fine pruning combined with efficient implementation of the resulting networks is a more promising direction for easing the trade-off between high accuracy and low computational cost.

An integrated simulation-data envelopment analysis approach for impact of line-seru conversion

This study aims to design an innovative method to evaluate the effects of line-seru conversion in terms of various environmental and economic performance criteria. For this purpose, an integrated approach using simulation and data envelopment analysis (DEA) assisted with various performance criteria is proposed for the first time. The simulation studies involve diverse scenarios for the production of a circuit breaker on assembly lines balanced with RPW and COMSOAL heuristic algorithms, and seru production systems. The simulation results indicate a significant increase in labour effectiveness and station/yatai utilization ratios, complete removal of WIP inventories, which led to a growth in productivity up to 43.29%. The number of workers, carbon-dioxide emission, required work-space, training costs of workers and equipment cost data were incorporated in the results to evaluate the applicability and relative efficiency of the developed scenarios by use of the CCR and BCC models of DEA. The calculated scale efficiency shows that the scenarios for assembly lines balanced with COMSOAL and the seru-based scenarios were both fully effective. The seru production system scenario with 6 yatais was the most effective scenario due to reduced task times after conversion. Also, line-seru conversion proved to be advantageous despite the high training costs.

Super-resolution enhancement and segmentation for digital rock based on multi-task joint deep neural network

Reliable segmentation of pores and minerals from high-resolution (HR) digital rock data is the fundamental prerequisite for accurately characterizing the digital rock's physical properties. Limited by the complexity of the super-resolution (SR) issues and the user bias for segmentation, conventional SR enhancement and segmentation methods are generally brutal in satisfying the research demands. In recent years, with the introduction of deep learning technology into digital rock research, deep learning-based approaches for SR enhancement or segmentation methods have emerged and achieved significantly better effects than conventional approaches. Most deep learning-based digital rock SR enhancement and segmentation processing steps are currently separate. Still, preliminary studies have demonstrated that an end-to-end approach that integrates the two steps could achieve better performance. However, the training cost of 3D deep neural networks that can be directly applied to 3D digital rock processing is usually expensive. In contrast, 2D networks cannot be effectively applied to SR segmentation of 3D digital rocks. Here, we present an end-to-end SR segmentation framework for 3D digital rock based on a 2D multi-task joint deep neural network. The multi-task joint networks utilize a parallel architecture that integrates the SegNet for the segmentation task and the EDSR for the SR task. We also improved the loss function to address the issue of category imbalance and proposed an approach to utilize the 2D network for 3D digital rock processing. We demonstrate the effectiveness of the proposed network framework, the improved loss function, and the 3D digital rock processing strategies through ablation experiments on the high and low-resolution (LR) CT image datasets captured by imaging devices. The results show that the evaluation metrics and physical properties of the SR-segmented 2D\\3D digital rocks align with the HR-segmented results (ground truth). It indicates that the proposed framework can improve the performance of digital rock SR enhancement and segmentation and that it is essential to integrate deep learning frameworks into digital rock analysis.

DOL3: Distilled OpenL3 audio embeddings for lightweight audio classification

Deep audio representations, also known as embeddings, recently became a popular alternative to conventional features like spectrograms for a wide range of audio classification tasks because of their domain-agnostic character and reduced training costs. Still, the usage is often limited to rather computationally intensive system due to the nature of their extraction from large networks. This paper aims to minimize the computational costs of embedding extraction by distilling the knowledge of the OpenL3 audio network to a smaller student network. Results show that the student network maintains comparable performance as the teacher network on various music and ambient noise classification tasks, while reducing the network size by over 90\% and the computational load by five times.

A joint vehicular device scheduling and uncertain resource management scheme for Federated Learning in Internet of Vehicles

Federated learning (FL) offers an effective framework for the efficient process in vehicular edge computing. However, FL encompasses the process of distributing and uploading model parameters, which are inevitably transmitted in a wireless network environment. Some challenges in FL-assisted Internet of Vehicles (IoV) sceneries gradually emerging, such as data heterogeneity, concerned device resources, and unstable communication environment, which necessitate intelligent vehicle selection schemes that accelerate training efficiency. Based on these, we consider a new scenario, specifically an FL-assisted IoV system under uncertain communication conditions, and develop an interval many-objective vehicle selection and bandwidth allocation (IMoVSBA) joint optimization scheme. This scheme takes into account computation latency, energy consumption, server utilization, and data quality, while meeting multi-criteria resource optimization requirements. Among these, server utilization is a new objective designed specifically for this joint optimization problem. For the proposed problem, a novel interval many-objective evolutionary algorithm with individual comprehensive indicator to control the evolution direction (IMaOEACI) is designed. Simulation results demonstrate that this method outperforms other schemes in terms of accuracy, training cost, and server utilization, effectively improving training efficiency in wireless channel environments and reasonably utilizing bandwidth resources. It provides significant scientific value and application potential in the field of the IoVs.

Robust function guided color encoded single fringe pattern and unwrapping method

Fringe Projection Profilometry (Fringe Projection Profilometry，FPP), renowned for its speed, precision, and automation advantages, has found extensive applications in industries such as mechanical processing and quality inspection. A critical challenge within fringe projection techniques involves ensuring robust unwrapping while minimizing the necessary fringe images. While various unwrapping methods exist to reduce the number of projected images, there is a deficiency in efficient unwrapping methods developed under the guidance of error analysis theory. In this study, we derive a robustness function from FPP's error theory to assess the unwrapping robustness of different encoded images. Using this robustness function, we select highly robust single-color encoded fringe images and achieve unwrapping robustness with minimal training cost based on a single fringe pattern. Furthermore, this unwrapping process does not necessitate the establishment of complex training databases, as the K-nearest neighbor (K-Nearest Neighbor，KNN) algorithm learns the correspondence between image tricolor grayscale values and fringe orders from multi-frequency unwrapping method. Finally, experimental validation through a checkerboard measurement confirms that the proposed method can achieve robust unwrapping with low training costs using only one encoded fringe, thereby affirming the effectiveness of the introduced robustness function and the feasibility of the designed color encoded single fringe pattern unwrapping method.

The PREFIT Study: Bone-Targeted Pre-conditioning Reduces Musculoskeletal Injury and Associated Costs in Australian Army Recruit Training

The PREFIT Study: Bone-Targeted Pre-conditioning Reduces Musculoskeletal Injury and Associated Costs in Australian Army Recruit Training

Training in Transplantation and Cellular Therapy in Latin America: A Cross-Sectional Study of the LABMT

Hematopoietic cell transplantation (HCT) is a complex and resource-intensive procedure that has become a critical treatment for certain hematologic conditions. However, in Latin America, access to HCT is limited compared to high-income countries, in part due to a lack of standardized training programs for HCT professionals. To address this gap, the Latin-American Bone Marrow Transplantation Group conducted a cross-sectional study to assess the current state of training programs in HCT and cellular therapy across the region. This study aimed to describe and analyze the availability, characteristics, and challenges of HCT training programs in Latin America, with a focus on identifying barriers and proposing solutions for improvement. A cross-sectional survey was sent to 127 recognized HCT centers across 14 Latin-American countries in December 2022. The survey collected data on institutional characteristics, training program structure, costs, and barriers to program development. Descriptive statistics were used to summarize the data, and comparative analyses were performed using Chi-square and Mann-Whitney tests. Of the 127 centers surveyed, 50 (39%) responded, with the majority located in Brazil (34%) and Mexico (30%). Among the respondents, 64% (n = 32) offered formal training programs lasting 6 months or longer. The most significant barriers reported were lack of funding (n = 21), limited number of transplant procedures (n = 15), and a shortage of qualified professors (n = 11). Proposed solutions included increasing student mobility opportunities (n = 28), enhancing program quality (n = 27), and improving access to funding (n = 15). Only 6% of programs offered exposure to CAR-T therapy, and fewer than half of the centers provided international rotations. This study highlights significant disparities in HCT training programs across Latin America, with most countries lacking access to formalized training. While Brazil and Mexico serve as regional hubs, other nations have limited or no training opportunities. Addressing these gaps through increased funding, international collaborations, and standardized curricula is essential to improving HCT training and ultimately patient care in the region.

Smart Farming for Sustainable Agriculture: A Case Study through Korean Public TVET

This study aims to present implications for sustainable education and training of the agricultural workforce. It introduces the case of a smart farm learning factory as an educational facility at Korea Polytechnics. Learning factory refers to the practical experience gained using the knowledge acquired in engineering education to solve problems in the actual work field. For this study, we analyzed the smart farm case of Hongseong Campus of Korea Polytechnics, a government-funded vocational education institution in Korea. The research questions are as follows. First, what is the purpose of establishing a smart farm education and training facility? Secondly, what is the performance of the facility? Thirdly, what are the implications of the Korean smart farm for sustainable TEVT? As a result of analyzing the case, the purpose of establishing the facility was to improve the practical capabilities of convergence technologies used in machinery, automation, industrial facilities, and electrical departments. Additionally, the performance has been found to serve as the foundation for project-based learning subjects that incorporate convergence education, and companies have been able to reduce training costs and contribute to sales profits by using this facility to train employees. Finally, the implication of sustainable TVET through smart farm cases is that it has provided a way to enhance the effectiveness of teaching and learning guidance at agricultural sites, create educational resources that can utilize advanced agricultural technologies for sustainable development, and actively cooperate and collaborate with educational institutions, governments, and local companies.

Fine-grained detoxification framework via instance-level prefixes for large language models

Large Language Models (LLMs) have demonstrated remarkable performance across various natural language processing (NLP) tasks. However, their practical usability is often compromised by a propensity to generate toxic content, such as insults, threats, and profanity, particularly in response to adversarial prompts. Several fine-tuning and decoding approaches have been employed to address this challenge to mitigate toxicity. Nevertheless, these methods typically necessitate additional resources, such as high-quality training data or auxiliary models, thereby incurring extra costs. In this paper, we propose a novel detoxification framework, Fine-Grained Detoxification via Instance-Level Prefixes (FGDILP), which effectively mitigates the generation of toxic text without incurring additional training costs. Specifically, FGDILP leverages contextualized representations in attention space by contrasting a positive prefix-prepended prompt against multiple negative prefix-prepended prompts at the instance level. This methodology facilitates the construction of fine-grained subtoxicity vectors, which are subsequently fused to adjust the original generation pathway when responding to raw prompts. Our results demonstrate that FGDILP enables controlled text generation concerning detoxification at both the utterance and context levels. While our method slightly impacts generation fluency and diversity, it significantly outperforms prompt-based baselines regarding detoxification effectiveness. Our code is available at https://github.com/xinykou/FGDILP.

Accelerating one-shot neural architecture search via constructing a sparse search space

Neural Architecture Search (NAS) has garnered significant attention for its ability to automatically design high-quality deep neural networks (DNNs) tailored to various hardware platforms. The major challenge for NAS is the time-consuming network estimation process required to select optimal networks from a large pool of candidates. Rather than training each candidate from scratch, recent one-shot NAS methods accelerate the estimation process by only training a supernet and sampling sub-networks from it, inheriting partial network architectures and weights. Despite significant acceleration, the supernet training with a large search space (i.e., the number of candidate sub-networks) still requires thousands of GPU hours to support high-quality sub-network sampling. In this work, we propose SparseNAS, an approach for one-shot NAS acceleration by reducing the redundancy of the search space. We observe that many sub-networks in the space are underperforming, with significant performance disparity to high-performance sub-networks. Crucially, this disparity can be observed early in the beginning of the supernet training. Therefore, we train an early predictor to learn this disparity and filter out high-quality networks in advance. Then, the supernet training will be conducted in this space sub-space. Compared to the state-of-the-art one-shot NAS, our SparseNAS reports a 3.1× training speedup with comparable network performance on the ImageNet dataset. Compared to the state-of-the-art acceleration method, SparseNAS reports a maximum of 1.5% higher Top-1 accuracy and 28% training cost reduction with a 7× bigger search space. Extensive experiment results demonstrated that SparseNAS achieves better trade-offs between efficiency and performance than state-of-the-art one-shot NAS.