Large-scale Machine Research Articles

Sustainability: Mechanized Handling of Large-Scale Livestock's Cow Manure Factors Into a Greener Economy

This research objective is to develop a large-scale livestock cow manure processing machine to add economic value and be environmentally friendly. The population was 4,994 dairy farmers in Nakhonratchasima province, Thailand. The sample group was the dairy farmers group in Pak Chong district Nakhonratchasima province, Thailand, with a total of 357 people. By evaluating the rating scale (Rating Scale) five levels of Likert. and there was a confidence value with an alpha coefficient (Cronbach's Alpha = 0.703), which was analyzed by SLR systematic literature review and scamper analysis by synthesizing the data to design new products as three main issues were 1) product creation guidelines from SLR literature review 2) problem diagnosis to the solution with SCAMPER technique 3) new product design in accordance with GOAL data were analyzed by multiple regression analysis and confirmatory factor analysis which found that 1) factors machine features 2) the efficiency factor of the machine and 3) image factors of the machine that resulted positively on farmer satisfaction and explained the farmer satisfaction variance of 71.9%. The standard score regression equation was Z= .453(X_1 )+ .384(X_2 )+ .465(X_3 ) and the regression equation in raw score y ?=-1.20+.460(X_1 )+.385(X_2 )+.466(X_3 ). When evaluated with CFA, the satisfaction of the sample group of dairy farmers group with the developed cow dung separate machine was found. That is the result of 3 factors according to the model concordance index: ?^2= .874, df = 4, relative ?^2= .219, p-value = .928, RMSEA = .000, RMR = .004, GFI = .996, AGFI = .956, NFI = .998, TLI = 1.078, PGFI = .089, RFI = .980, AIC = 82.874
 
 Received: 18 August 2023 / Accepted: 25 October 2023 / Published: 5 November 2023

Accelerated Distributed Approximate Newton Method.

Distributed second-order optimization, as an effective strategy for training large-scale machine learning systems, has been widely investigated due to its low communication complexity. However, the existing distributed second-order optimization algorithms, including distributed approximate Newton (DANE), accelerated inexact DANE (AIDE), and statistically preconditioned accelerated gradient (SPAG), are all required to precisely solve an expensive subproblem up to the target precision. Therefore, this causes these algorithms to suffer from high computation costs and this hinders their development. In this article, we design a novel distributed second-order algorithm called the accelerated distributed approximate Newton (ADAN) method to overcome the high computation costs of the existing ones. Compared with DANE, AIDE, and SPAG, which are constructed based on the relative smooth theory, ADAN's theoretical foundation is built upon the inexact Newton theory. The different theoretical foundations lead to handle the expensive subproblem efficiently, and steps required to solve the subproblem are independent of the target precision. At the same time, ADAN resorts to the acceleration and can effectively exploit the objective function's curvature information, making ADAN to achieve a low communication complexity. Thus, ADAN can achieve both the communication and computation efficiencies, while DANE, AIDE, and SPAG can achieve only the communication efficiency. Our empirical study also validates the advantages of ADAN over extant distributed second-order algorithms.

A Study of Factors Influencing the Adoption of Artificial Intelligence in Crisis Management

This paper presents a study on the Factors Influencing the Adoption of Artificial Intelligence (AI) in Crisis Management. The research identifies 28 AI usage factors categorized into seven groups: Large-Scale Machine Learning, Deep Learning, Reinforcement Learning, Robotics, Computer Vision, Natural Language Processing, and Internet of Things. The study conducted a questionnaire survey among 281 employees at the UAE National Crisis and Emergency Management Authority, using purposive sampling to assess theiropinions regarding the impact of these usage factors on the adoption of AI in crisis management. The collected data underwent descriptive analysis to determine the ranking of AI usage factors within each of the seven groups. In terms of group rankings, Robotic emerged as the top-ranking factor, followed by Reinforcement Learning. Large-Scale Machine Learning occupied the next position, succeeded by Natural Language Processing, Deep Learning, Internet of Things, and Computer Vision, which held the lowest rank. Furthermore, when examining the correlation between these usage factor groups, it was discovered that most of them exhibited strong positive correlations, with correlation coefficients ranging from 0.634 to 0.934. This indicates that changes in one variable are associated with predictable changes in another variable. While this information can be instrumental in understanding relationships and making predictions, it does not establish a causal relationship.

SARAH-M: A fast stochastic recursive gradient descent algorithm via momentum

As a simple but effective way, the momentum method has been widely adopted in stochastic optimization algorithms for large-scale machine learning problems and the success of stochastic optimization with the momentum term for many applications in machine learning and other related areas has been reported everywhere. However, the understanding of how the momentum improves the performance of modern variance reduced stochastic gradient algorithms, e.g., the stochastic dual coordinate ascent average gradient (SDCA) method, the stochastically controlled stochastic gradient (SCSG) method, the stochastic recursive gradient algorithm (SARAH), etc., is still limited. To tackle this issue, this work studies the performance of SARAH with the momentum term theoretically and empirically, and develops a novel variance reduced stochastic gradient algorithm, termed as SARAH-M. We rigorously prove that SARAH-M attains a linear rate of convergence for minimizing the strongly convex function. We further propose an adaptive SARAH-M method (abbreviated as AdaSARAH-M) by incorporating the random Barzilai–Borwein (RBB) technique into SARAH-M, which provides an easy way to determine the step size for the original SARAH-M algorithm. The theoretical analysis that shows AdaSARAH-M with a linear convergence speed is also provided. Moreover, we show that the complexity of the proposed algorithms can outperform modern stochastic optimization algorithms. Finally, the numerical results, compared with state-of-the-art algorithms on benchmarking machine learning problems, verify the efficacy of the momentum in variance reduced stochastic gradient algorithms.

Powered stochastic optimization with hypergradient descent for large-scale learning systems

Stochastic optimization (SO) algorithms based on the Powerball function, namely powered stochastic optimization (PoweredSO) algorithms, have been confirmed, effectively, and demonstrated great potential in the context of large-scale optimization and machine learning tasks. Nevertheless, the issue of how to determine the learning rate for PoweredSO is a challenge and still unsolved problem. In this paper, we propose a class of adaptive PoweredSO approaches that are efficient, scalable and robust. It takes advantage of the hypergradient descent (HD) technique to automatically acquire an online learning rate for PoweredSO-like methods. In the first part, we study the behavior of the canonical PoweredSO algorithm, the Powerball stochastic gradient descent (pbSGD) method, with HD. The existing PoweredSO algorithms also suffer from the high variance because they take the similar algorithmic framework to SO algorithms, arising from sampling tactics. Therefore, the second portion develops an adaptive powered variance-reduced optimization method via utilizing both variance-reduced technique and HD. Moreover, we present the convergence analysis of the proposed algorithms and explore their iteration complexity on non-convex cases. Numerical experiments are conducted on machine learning tasks, verifying the superior performance over modern SO algorithms.

A Lyapunov Theory for Finite-Sample Guarantees of Markovian Stochastic Approximation

The stochastic approximation (SA) method stands as the foundational mathematical tool for modern large-scale optimization and machine learning. Therefore, gaining a theoretical understanding of SA algorithms is of fundamental interest. In their paper titled “A Lyapunov Theory for Finite-Sample Guarantees of Markovian Stochastic Approximation,” Chen et al. present a unified Lyapunov framework for the finite-sample analysis of a Markovian SA algorithm under a contractive operator with respect to an arbitrary norm. The key novelty lies in the construction of a smooth Lyapunov function called the generalized Moreau envelope. The authors demonstrate the effectiveness of their SA results in the context of reinforcement learning (RL), specifically through popular algorithms such as variants of temporal difference (TD) learning and Q-learning. As byproducts, the results provide theoretical insights into the efficiency of bootstrapping in TD learning with eligibility traces and the bias-variance tradeoff in off-policy learning.

Federated quanvolutional neural network: a new paradigm for collaborative quantum learning

In recent years, the concept of federated machine learning has been actively driven by scientists to ease the privacy concerns of data owners. Currently, the combination of machine learning and quantum computing technologies is a hot industry topic and is positioned to be a major disruptor. It has become an effective new tool for reshaping several industries ranging from healthcare to finance. Data sharing poses a significant hurdle for large-scale machine learning in numerous industries. It is a natural goal to study the advanced quantum computing ecosystem, which will be comprised of heterogeneous federated resources. In this work, the problem of data governance and privacy is handled by developing a quantum federated learning approach, that can be efficiently executed on quantum hardware in the noisy intermediate-scale quantum era. We present the federated hybrid quantum–classical algorithm called a quanvolutional neural network with distributed training on different sites without exchanging data. The hybrid algorithm requires small quantum circuits to produce meaningful features for image classification tasks, which makes it ideal for near-term quantum computing. The primary goal of this work is to evaluate the potential benefits of hybrid quantum–classical and classical-quantum convolutional neural networks on non-independently and non-identically partitioned (Non-IID) and real-world data partitioned datasets among several healthcare institutions/clients. We investigated the performance of a collaborative quanvolutional neural network on two medical machine learning datasets, COVID-19 and MedNIST. Extensive experiments are carried out to validate the robustness and feasibility of the proposed quantum federated learning framework. Our findings demonstrate a decrease of 2%–39% times in necessary communication rounds compared to the federated stochastic gradient descent approach. The hybrid federated framework maintained a high classification testing accuracy and generalizability, even in scenarios where the medical data is unevenly distributed among clients.

Predicting the Fracture Propensity of Amorphous Silica Using Molecular Dynamics Simulations and Machine Learning

Amorphous silica (a-SiO2) is a widely used inorganic material. Interestingly, the relationship between the local atomic structures of a-SiO2 and their effects on ductility and fracture is seldom explored. Here, we combine large-scale molecular dynamics simulations and machine learning methods to examine the molecular deformations and fracture mechanisms of a-SiO2. By quenching at high pressures, we demonstrate that densifying a-SiO2 increases the ductility and toughness. Through theoretical analysis and simulation results, we find that changes in local bonding topologies greatly facilitate energy dissipation during plastic deformation, particularly if the coordination numbers decrease. The appearance of fracture can then be accurately located based on the spatial distribution of the atoms. We further observe that the static unstrained structure encodes the propensity for local atomic coordination to change during applied strain, hence a distinct connection can be made between the initial atomic configurations before loading and the final far-from-equilibrium atomic configurations upon fracture. These results are essential for understanding how atomic arrangements strongly influence the mechanical properties and structural features in amorphous solids and will be useful in atomistic design of functional materials.

A Quantum-Chemical Bonding Database for Solid-State Materials

An in-depth insight into the chemistry and nature of the individual chemical bonds is essential for understanding materials. Bonding analysis is thus expected to provide important features for large-scale data analysis and machine learning of material properties. Such chemical bonding information can be computed using the LOBSTER software package, which post-processes modern density functional theory data by projecting the plane wave-based wave functions onto an atomic orbital basis. With the help of a fully automatic workflow, the VASP and LOBSTER software packages are used to generate the data. We then perform bonding analyses on 1520 compounds (insulators and semiconductors) and provide the results as a database. The projected densities of states and bonding indicators are benchmarked on standard density-functional theory computations and available heuristics, respectively. Lastly, we illustrate the predictive power of bonding descriptors by constructing a machine learning model for phononic properties, which shows an increase in prediction accuracies by 27% (mean absolute errors) compared to a benchmark model differing only by not relying on any quantum-chemical bonding features.

An adaptive federated learning system for community building energy load forecasting and anomaly prediction

Energy load forecasting is critical for sustainable building development and management. Although the energy data could be collected through Internet of Things (IoT) systems, it is a big challenge to train a large-scale machine learning model due to data isolation. Since the building energy data could reveal confidential information such as user behaviors and building operations, the privacy regulations would not allow central service to collect distributed data from data owners directly. This paper designs a secure federated data analytics system for forecasting community buildings' energy data load. A novel adaptive weight federated learning algorithm is proposed to handle the system faults frequently happening during networking operations. Moreover, a new deep learning model is re-invented to improve energy load forecasting performance. The experiments of the system are performed on an actual university campus dataset, and the results show the new federated algorithm improves the load forecasting accuracy and achieves the best load forecasting result. The new deep learning model improves the forecasting accuracy by almost 10% on error reduction under the same federated learning settings. To evaluate the load forecasting model's practical usefulness, an anomaly prediction pipeline is designed through the combination of gaussian mixture model and load forecasting model, which reveals the system's effectiveness at building energy management that 92% F1 score with 97% accuracy is achieved by the best model.

Nonlinear reduced-order modeling for three-dimensional turbulent flow by large-scale machine learning

A large-scale machine learning-based nonlinear reduced-order modeling method was developed for a three-dimensional turbulent flow field (Re=1000) using a neural-network with unsupervised learning. First, a mode decomposition method was applied to three-dimensional flow field data using a convolutional-autoencoder-like neural network. Then, a reduced-order model (ROM) was constructed using long short-term memory neural networks (LSTMs). Consequently, it was demonstrated that the time evolution of the turbulent three-dimensional flow field can be simulated at a significantly lower cost (approximately three orders of magnitude) without a major loss in accuracy. However, neural-network-based mode decomposition for the three-dimensional flow field requires a huge computational cost in terms of calculation and memory usage. Therefore, a distributed machine-learning method was implemented using a hybrid parallelism scheme tailored to the network structure. Thus, it was possible to decompose 1.7 million cells of the three-dimensional flow field data into 64 modes and reproduce those with sufficient accuracy. In this study, a uniform flow around a circular cylinder model was used as a test case. To validate the method, the reduction performance of the proposed mode decomposition method was compared with the proper orthogonal decomposition (POD) method. Furthermore, the target flow field was reproduced using ROM, and the reconstruction accuracy was evaluated in terms of various criteria compared with the accuracy based on POD in conjunction with Galerkin projection method.

Investigating Thermal Conductivity of Ferrofluids

Heat transfer is one of the most important aspects of large-scale industries and machines, linking importantly to the efficiency associated with different mechanisms while also emphasizing the importance of sustainable, low-cost methods of heat transfer. One such method is the use of ferrofluids. Through this paper, it is clearly explained that the conductivity of ferrofluids has vast applications across industries and using its magnetic and thermal properties, it can be a cost-effective solution as well. Previously researched works in this field ferrofluids are also acknowledged and furthered as per experimental data.

A clinician's guide to large language models

The rapid advancement of artificial intelligence (AI) has led to the emergence of large language models (LLMs) as powerful tools for various applications, including healthcare. These large-scale machine learning models, such as GPT and LLaMA have demonstrated potential for improving patient outcomes and transforming medical practice. However, healthcare professionals without a background in data science may find it challenging to understand and utilize these models effectively. This paper aims to provide an accessible introduction to LLMs for healthcare professionals, discussing their core concepts, relevant applications in healthcare, ethical considerations, challenges, and future directions. With an overview of LLMs, we foster a more collaborative future between healthcare professionals and data scientists, ultimately driving better patient care and medical advancements.

Low-Rank Combinatorial Optimization and Statistical Learning by Spatial Photonic Ising Machine.

The spatial photonic Ising machine (SPIM) [13D. Pierangeli et al., Large-Scale Photonic Ising Machine by Spatial Light Modulation, Phys. Rev. Lett. 122, 213902 (2019).PRLTAO0031-900710.1103/PhysRevLett.122.213902] is a promising optical architecture utilizing spatial light modulation for solving large-scale combinatorial optimization problems efficiently. The primitive version of the SPIM, however, can accommodate Ising problems with only rank-one interaction matrices. In this Letter, we propose a new computing model for the SPIM that can accommodate any Ising problem without changing its optical implementation. The proposed model is particularly efficient for Ising problems with low-rank interaction matrices, such as knapsack problems. Moreover, it acquires the learning ability of Boltzmann machines. We demonstrate that learning, classification, and sampling of the MNIST handwritten digit images are achieved efficiently using the model with low-rank interactions. Thus, the proposed model exhibits higher practical applicability to various problems of combinatorial optimization and statistical learning, without losing the scalability inherent in the SPIM architecture.

StainGAN: Learning a structural preserving translation for white blood cell images.

Analysis of white blood cells in blood smear images plays a vital role in computer-aided diagnosis for the analysis and treatment of many diseases. However, different techniques for blood smear preparation result in images with large appearance variations, which limits the performance of large-scale machine learning algorithms. In this paper, we propose StainGAN, an image translation framework to transform the conventional Wright-stained white blood cell images into their rapidly-stained counterpart. Moreover, we designed a cluster-based learning strategy that does not require manual annotations and a multi-scale discriminator that incorporates a richer hierarchy of the spatial context to generate sharper images with better semantic consistency. Experimental results on multiple real-world datasets prove the effectiveness of our proposed framework. Moreover, we show that the transformed images from StainGAN can be used to boost the downstream segmentation performance under the label-limiting scenario.

The Story of GraphLab - From Scaling Machine Learning to Shaping Graph Systems Research (VLDB 2023 Test-of-Time Award Talk)

The GraphLab project spanned almost a decade and had profound academic and industrial impact on large-scale machine learning and graph processing systems. There were numerous papers written describing the innovations in GraphLab including the original vertex-centric [8] and edge-centric [3] programming abstractions, high-performance asynchronous execution engines [9], out-of-core graph computation [6], tabular graph-systems [4], and even new statistical inference algorithms [2] enabled by the GraphLab project. This work became the basis of multiple PhD theses [1, 5, 7]. The GraphLab open-source project had broad academic and industrial adoption and ultimately lead to the launch of Turi. In this talk, we tell the story of GraphLab, how it began and the key ideas behind it. We will focus on the approach to achieving scalable asynchronous systems in machine learning. During our talk, we will explore the impact that GraphLab has had on the development of graph processing systems, graph databases, and AI/ML; Additionally, we will share our insights and opinions into where we see the future of these fields heading. In the process, we highlight some of the lessons we learned and provide guidance for future students.

Large-Scale Exact Epidemiological Modelling with Multi-Dimensional Constrained Optimization Approaches.

Multi-dimensional prediction models of the pandemic diseases should be constructed in a way to reflect their peculiar epidemiological characters. In this paper, a graph theory-based constrained multi-dimensional (CM) mathematical and meta-heuristic algorithms (MA) are formed to learn the unknown parameters of a large-scale epidemiological model. The specified parameter signs and the coupling parameters of the sub-models constitute the constraints of the optimization problem. In addition, magnitude constraints on the unknown parameters are imposed to proportionally weight the input-output data importance. To learn these parameters, a gradient-based CM recursive least square (CM-RLS) algorithm, and three search-based MAs; namely, the CM particle swarm optimization (CM-PSO), the CM success history-based adaptive differential evolution (CM-SHADE), and the CM-SHADEWO enriched with the whale optimization (WO) algorithms are constructed. The traditional SHADE algorithm was the winner of the 2018 IEEE congress on evolutionary computation (CEC) and its versions in this paper are modified to create more certain parameter search spaces. The results obtained under the equal conditions show that the mathematical optimization algorithm CM-RLS outperforms the MA algorithms, which is expected since it uses the available gradient information. However, the search-based CM-SHADEWO algorithm is able to capture the dominant character of the CM optimization solution and produce satisfactory estimates in the presence of the hard constraints, uncertainties and lack of gradient information.

ORCCA: Optimal Randomized Canonical Correlation Analysis.

Random features approach has been widely used for kernel approximation in large-scale machine learning. A number of recent studies have explored data-dependent sampling of features, modifying the stochastic oracle from which random features are sampled. While proposed techniques in this realm improve the approximation, their suitability is often verified on a single learning task. In this article, we propose a task-specific scoring rule for selecting random features, which can be employed for different applications with some adjustments. We restrict our attention to canonical correlation analysis (CCA) and provide a novel, principled guide for finding the score function maximizing the canonical correlations. We prove that this method, called optimal randomized CCA (ORCCA), can outperform (in expectation) the corresponding kernel CCA with a default kernel. Numerical experiments verify that ORCCA is significantly superior to other approximation techniques in the CCA task.

Implementation of cross-scale plane micro-scratching process driven by hybrid piezoelectric actuation

The atomic force microscope tip-based mechanical machining technique has been used to fabricate complex micro-structures successfully. However, the machining scope and depth are limited, which holds back this technique severely. This work presents the design and evaluation of a piezoelectric manufacturing system (PMS) for implementation of cross-scale (large scale and high resolution) plane micro-scratching. The PMS has the two-dimensional (2D) micro-machining ability with depths up to 0.98 μm in the millimeter machining scope through the combination of a travel range XY positioning stage and a three-degree-of-freedom (3-DOF) sandwich piezoelectric manipulator. The tracing property of the PMS is feasible for large scale machining by controlling the multi-axis cooperative motion of the 2-DOF stage. The experiments show that the micro-triangles and micro-circles arrays with dimensions from 40 μm to 400 μm are machined successfully. The deviations of the position determination and machined depths for all the micro-structures are less than 5%. Further, it is verified that the manipulator has potential to be used in vibration-assisted cutting to improve machining quality. To sum up, the PMS shows great prospects in the applications of machining cross-scale and large depth planar micro-structures.

Memory Classifiers for Robust ECG Classification against Physiological Noise.

The development of sophisticated machine learning algorithms has made it possible to detect critical health conditions like cardiac arrhythmia, directly from electrocardiogram (ECG) recordings. Large-scale machine learning models, like deep neural networks, are well known to underperform when subjected to small perturbations which would not pose a challenge to physicians. This is a hurdle that needs to be removed to facilitate wide-scale adoption. We find this to be true even for models trained using data-augmentation schemes.In this paper, we show that using memory classifiers it is possible to attain a boost in robustness using expert-informed features. Memory classifiers combine standard deep neural network training with a domain knowledge-guided similarity metric to boost the robustness of classifiers. We evaluate the performance of the models against naturally occurring physiological perturbations, specifically electrode movement, muscle artifact, and baseline wander noise. Our approach demonstrates improved robustness across all evaluated noises for an average improvement in F1 score of 3.13% compared to models using data augmentation techniques.Clinical relevance- This approach improves the robustness of deep learning methods in safety-critical medical applications.