Large-scale Machine Learning Research Articles

Evaluating strength behavior and failure mode of ice materials based on large-scale database and machine learning algorithms

The strength characteristic and failure mode of ice materials are widely used to analyze the interaction between ice and structure to ensure the construction stability in ocean engineering and ice engineering. This study establishes a large-scale database (comprising 5542 testing samples) for the ice mechanics to study the influences of six effective features (grain size, density, porosity, salinity, temperature, and strain rate) on mechanical behaviors. The correlation degrees between effective features and strength behavior (failure mode) are investigated through correlation analysis. The Regression algorithm and Classification algorithm of four machine learning models are respectively used to evaluate and predict the strength and deformation behaviors of ice materials. Four evaluation parameters (R2, RMSE, MAE, and MBE) are adopted to further investigate the predictive ability of those machine learning models. Based on the partial dependence interpretation, the contributions of effective features to strength prediction are quantitatively described. The results indicate that temperature is the most important factor for strength behavior. In the classification prediction for failure mode, the prediction accuracy for ductile behavior is enhanced in the Random Forest algorithm to improve the overall classification accuracy, and the Random Forest algorithm exhibits well performance compared to the other three algorithms.

Lactylation in cancer: Mechanisms in tumour biology and therapeutic potentials.

Lactylation, a recently identified form of protein post-translational modification (PTM), has emerged as a key player in cancer biology. The Warburg effect, a hallmark of tumour metabolism, underscores the significance of lactylation in cancer progression. By regulating gene transcription and protein function, lactylation facilitates metabolic reprogramming, enabling tumours to adapt to nutrient limitations and sustain rapid growth. Over the past decade, extensive research has revealed the intricate regulatory network underlying lactylation in tumours. Large-scale sequencing and machine learning have confirmed the widespread occurrence of lactylation sites across the tumour proteome. Targeting lactylation enzymes or metabolic pathways has demonstrated promising anti-tumour effects, highlighting the therapeutic potential of this modification. This review comprehensively explores the mechanisms of lactylation in cancer cells and the tumour microenvironment. We expound on the application of advanced omics technologies for target identification and data modelling within the lactylation field. Additionally, we summarise existing anti-lactylation drugs and discuss their clinical implications. By providing a comprehensive overview of recent advancements, this review aims to stimulate innovative research and accelerate the translation of lactylation-based therapies into clinical practice. KEY POINTS: Lactylation significantly influences tumour metabolism and gene regulation, contributing to cancer progression. Advanced sequencing and machine learning reveal widespread lactylation sites in tumours. Targeting lactylation enzymes shows promise in enhancing anti-tumour drug efficacy and overcoming chemotherapy resistance. This review outlines the clinical implications and future research directions of lactylation in oncology.

Intraoperative Hypotension Prediction: Current Methods, Controversies, and Research Outlook.

Intraoperative hypotension prediction has been increasingly emphasized due to its potential clinical value in reducing organ injury and the broad availability of large-scale patient datasets and powerful machine learning tools. Hypotension prediction methods can mitigate low blood pressure exposure time. However, they have yet to be convincingly demonstrated to improve objective outcomes; furthermore, they have recently become controversial. This review presents the current state of intraoperative hypotension prediction and makes recommendations on future research. We begin by overviewing the current hypotension prediction methods, which generally rely on the prevailing mean arterial pressure as one of the important input variables and typically show good sensitivity and specificity but low positive predictive value in forecasting near-term acute hypotensive events. We make specific suggestions on improving the definition of acute hypotensive events and evaluating hypotension prediction methods, along with general proposals on extending the methods to predict reduced blood flow and treatment effects. We present a start of a risk-benefit analysis of hypotension prediction methods in clinical practice. We conclude by coalescing this analysis with the current evidence to offer an outlook on prediction methods for intraoperative hypotension. A shift in research toward tailoring hypotension prediction methods to individual patients and pursuing methods to predict appropriate treatment in response to hypotension appear most promising to improve outcomes.

Nanostructure and dislocation interactions in refractory complex concentrated alloy: From chemical short-range order to nanoscale B2 precipitates

Refractory complex concentrated alloys (RCCAs) have emerged as a promising class of structural materials, demonstrating exceptional mechanical performance in aggressive environments. However, the complex atomic environments, significant lattice distortion, and vast compositional space of RCCAs present challenges to understanding the mechanisms that govern structure-property relationships. In this study, we explore the dislocation mechanisms in three model quaternary RCCAs, namely Mo25Nb10Ta25W40 (at. %), Mo25Nb25Ta25W25, and Mo25Nb40Ta25W10 using large-scale atomistic simulations and machine learning based Spectral Neighbor Analysis Potential. Our atomistic simulations examine how the chemical composition and local ordering influence the mobility of both edge and screw dislocations, and how lattice distortion and diffuse anti-phase boundary energy (DAPBE) affect dislocation behaviors during nanostructural evolution. Notably, with the increase in Nb concentration in the model RCCAs, both DAPBE and lattice distortion are simultaneously enhanced as the chemical short-range order (CSRO) evolves into nanoscale B2 precipitates. This evolution results in high lattice distortion due to the lattice mismatch between B2 precipitates and the random matrix. Consequently, B2 nanoprecipitates provide a stronger pinning effect, hindering edge dislocation motion while promoting cross-slip of screw dislocations, leading to a reduced screw-to-edge ratio in slip resistance and mobility discrepancy. These findings offer valuable insights into dislocation behaviors and interactions with ordered precipitates, highlighting the importance of exploring non-equiatomic compositions and advancing beyond CSRO in RCCAs. This study has implications for optimizing alloy compositions and processing methods for superior performance in aggressive environments.

Painless Stochastic Conjugate Gradient for Large-Scale Machine Learning.

Conjugate gradient (CG), as an effective technique to speed up gradient descent algorithms, has shown great potential and has widely been used for large-scale machine-learning problems. However, CG and its variants have not been devised for the stochastic setting, which makes them extremely unstable, and even leads to divergence when using noisy gradients. This article develops a novel class of stable stochastic CG (SCG) algorithms with a faster convergence rate via the variance-reduced technique and an adaptive step size rule in the mini-batch setting. Actually, replacing the use of a line search in the CG-type approaches which is time-consuming, or even fails for SCG, this article considers using the random stabilized Barzilai-Borwein (RSBB) method to obtain an online step size. We rigorously analyze the convergence properties of the proposed algorithms and show that the proposed algorithms attain a linear convergence rate for both the strongly convex and nonconvex settings. Also, we show that the total complexity of the proposed algorithms matches that of modern stochastic optimization algorithms under different cases. Scores of numerical experiments on machine-learning problems demonstrate that the proposed algorithms outperform state-of-the-art stochastic optimization algorithms.

Federated learning: A cutting-edge survey of the latest advancements and applications

Robust machine learning (ML) models can be developed by leveraging large volumes of data and distributing the computational tasks across numerous devices or servers. Federated learning (FL) is a technique in the realm of ML that facilitates this goal by utilizing cloud infrastructure to enable collaborative model training among a network of decentralized devices. Beyond distributing the computational load, FL targets the resolution of privacy issues and the reduction of communication costs simultaneously. To protect user privacy, FL requires users to send model updates rather than transmitting large quantities of raw and potentially confidential data. Specifically, individuals train ML models locally using their own data and then upload the results in the form of weights and gradients to the cloud for aggregation into the global model. This strategy is also advantageous in environments with limited bandwidth or high communication costs, as it prevents the transmission of large data volumes. With the increasing volume of data and rising privacy concerns, alongside the emergence of large-scale ML models like Large Language Models (LLMs), FL presents itself as a timely and relevant solution. It is therefore essential to review current FL algorithms to guide future research that meets the rapidly evolving ML demands. This survey provides a comprehensive analysis and comparison of the most recent FL algorithms, evaluating them on various fronts including mathematical frameworks, privacy protection, resource allocation, and applications. Beyond summarizing existing FL methods, this survey identifies potential gaps, open areas, and future challenges based on the performance reports and algorithms used in recent studies. This survey enables researchers to readily identify existing limitations in the FL field for further exploration.

Machine learning to predict distant metastasis and prognostic analysis of moderately differentiated gastric adenocarcinoma patients: a novel focus on lymph node indicators.

Moderately differentiated gastric adenocarcinoma (MDGA) has a high risk of metastasis and individual variation, which strongly affects patient prognosis. Using large-scale datasets and machine learning algorithms for prediction can improve individualized treatment. The specific efficacy of several lymph node indicators in predicting distant metastasis (DM) and patient prognosis in MDGA remains obscure. We collected data from MDGA patients from the SEER database from 2010 to 2019. Additionally, we collected data from MDGA patients in China. We used nine machine learning algorithms to predict DM. Subsequently, we used Cox regression analysis to determine the risk factors affecting overall survival (OS) and cancer-specific survival (CSS) in DM patients and constructed nomograms. Furthermore, we used logistic regression and Cox regression analyses to assess the specific impact of six lymph node indicators on DM incidence and patient prognosis. We collected data from 5,377 MDGA patients from the SEER database and 109 MDGC patients from hospitals. T stage, N stage, tumor size, primary site, number of positive lymph nodes, and chemotherapy were identified as independent risk factors for DM. The random forest prediction model had the best overall predictive performance (AUC = 0.919). T stage, primary site, chemotherapy, and the number of regional lymph nodes were identified as prognostic factors for OS. Moreover, T stage, number of regional lymph nodes, primary site, and chemotherapy were also influential factors for CSS. The nomograms showed good predictive value and stability in predicting the 1-, 3-, and 5-year OS and CSS in DM patients. Additionally, the log odds of a metastatic lymph node and the number of negative lymph nodes may be risk factors for DM, while the regional lymph node ratio and the number of regional lymph nodes are prognostic factors for OS. The random forest prediction model accurately identified high-risk populations, and we established OS and CSS survival prediction models for MDGA patients with DM. Our hospital samples demonstrated different characteristics of lymph node indicators in terms of distant metastasis and prognosis.

Towards Fairness-Aware Federated Learning.

Recent advances in federated learning (FL) have brought large-scale collaborative machine learning opportunities for massively distributed clients with performance and data privacy guarantees. However, most current works focus on the interest of the central controller in FL and overlook the interests of the FL clients. This may result in unfair treatment of clients, which discourages them from actively participating in the learning process and damages the sustainability of the FL ecosystem. Therefore, the topic of ensuring fairness in FL is attracting a great deal of research interest. In recent years, diverse fairness-aware FL (FAFL) approaches have been proposed in an effort to achieve fairness in FL from different perspectives. However, there is no comprehensive survey that helps readers gain insight into this interdisciplinary field. This article aims to provide such a survey. By examining the fundamental and simplifying assumptions, as well as the notions of fairness adopted by the existing literature in this field, we propose a taxonomy of FAFL approaches covering major steps in FL, including client selection, optimization, contribution evaluation, and incentive distribution. In addition, we discuss the main metrics for experimentally evaluating the performance of FAFL approaches and suggest promising future research directions toward FAFL.

Single-cell and whole-transcriptome sequencing of lymph node metastasis-related gene signature: a large-scale pan-cancer cohort, machine learning and experimental study: Retraction.

Single-cell and whole-transcriptome sequencing of lymph node metastasis-related gene signature: a large-scale pan-cancer cohort, machine learning and experimental study: Retraction.

Enhancing productiveness in E-commerce through machine learning: Challenges, Future Perspectives, and a MATLAB-Based Approach

The rapid growth of e-commerce has created an urgent need for more productive and efficient systems to handle the increasing complexity of online transactions and consumer behaviour. Machine Learning (ML) offers a transformative approach to enhancing productiveness in e-commerce by optimizing processes such as personalized recommendations, dynamic pricing, customer segmentation, and supply chain management. This article explores the integration of ML-driven initiatives in e-commerce, addressing key challenges such as data privacy, algorithmic bias, and the computational demands of large-scale ML applications. A MATLAB-based approach is proposed for developing and implementing these ML models, leveraging MATLAB’s robust toolboxes for data analysis, model training, and system simulation. Through an extensive literature review, this study highlights the current state of ML in e-commerce, identifies existing challenges, and discusses future perspectives. The introduction of ML-driven strategies not only improves operational efficiency but also enhances customer satisfaction, ultimately driving productivity in the highly competitive e-commerce landscape. The findings of this article are expected to provide valuable insights for both academic researchers and industry professionals aiming to harness the full potential of ML in e-commerce.

Large-Scale Computational Screening-Aided Development of High-Performance Adsorbent for Simultaneous Capture of Aromatic Volatile Organic Compounds.

The development of an efficient adsorbent for the simultaneous capture of large amounts of benzene, toluene, ethylbenzene, and xylene isomers (BTEX) is an important and challenging issue. Here, through a stepwise screening of 10,142 metal-organic framework (MOF) structures from the computation-ready, experimental (CoRE) MOF database, 65 MOFs are proposed as promising adsorbent candidates for BTEX capture by considering the structures with accessible pore sizes for BTEX adsorption, sufficient hydrophobicity, high benzene selectivity (>0.2), and large total BTEX uptake (>3 mmol/g). Among the top-performing MOFs in terms of the BTEXmatrix (total BTEX uptake × benzene selectivity), EGUELUY01 was synthesized, and it exhibited large uptakes (≈5 mmol/g) for all BTEX components at concentrations of 1200-1500 ppm, which are superior to the BTEX uptake of the benchmark adsorbent, activated carbon. Moreover, some structure-property relationships required for BTEX adsorbents are provided through the obtained large-scale simulation data and machine learning analysis. The determined relationships will be useful for the future development of efficient BTEX adsorbents.

Uncovering the most robust predictors of problematic pornography use: A large-scale machine learning study across 16 countries.

Problematic pornography use (PPU) is the most common manifestation of the newly introduced compulsive sexual behavior disorder diagnosis in the 11th revision of the International Classification of Diseases. Research related to PPU has proliferated in the past two decades, but most prior studies were characterized by several shortcomings (e.g., using homogenous, small samples), resulting in crucial knowledge gaps and a limited understanding concerning empirically based risk factors for PPU. This study aimed to identify the most robust risk factors for PPU using a preregistered study design. Independent laboratories' 74 preexisting self-report data sets (Nparticipants = 112,397; Ncountries = 16) were combined to identify which factors can best predict PPU using an artificial intelligence-based method (i.e., machine learning). We conducted random forest models on each data set to examine how different sociodemographic, psychological, and other characteristics predict PPU, and combined the results of all data sets using random-effects meta-analysis with meta-analytic moderators (e.g., community vs. treatment-seeking samples). Predictors explained 45.84% of the variance in PPU scores. Out of the 700+ potential predictors, 17 variables emerged as significant predictors across data sets, with the top five being (a) pornography use frequency, (b) emotional avoidance pornography use motivation, (c) stress reduction pornography use motivation, (d) moral incongruence toward pornography use, and (e) sexual shame. This study is the largest and most integrative data analytic effort in the field to date. Findings contribute to a better understanding of PPU's etiology and may provide deeper insights for developing more efficient, cost-effective, empirically based directions for future research as well as prevention and intervention programs targeting PPU. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Interpretable machine learning models for displacement demand prediction in reinforced concrete buildings under pulse-like earthquakes

This work proposes a novel procedure to guide the development of machine learning models for estimating the seismic demand in existing reinforced concrete (RC) buildings. The proposed approach is organized across two scales. A large-scale (nonparametric) machine learning model is first obtained by means of Gaussian Process Regression (GPR) using all candidate building attributes and intensity measures. SHapley Additive exPlanations (SHAP) values are utilized to facilitate its interpretation and to assist the rational selection of a small subset of intensity measures, which is finally employed to develop a (symbolic) reduced-scale machine learning model by means of Genetic Programming (GP). Simplified models of archetype buildings are adopted to develop machine learning techniques at both scales, in such a way to alleviate the simulation time for preparing large datasets. Refined models representative of actual buildings are instead considered for the unbiased final assessment.The proposed approach is applied to develop predictive machine learning models for the maximum inter-storey drift in bare frames, pilotis frames and frames with infills under pulse-like seismic ground motions. Consequently, the critical examination of the SHAP values revealed the most significant intensity measures and unfolded interesting patterns depending on the occupancy rate of the infills. Moreover, the final assessment demonstrates that this approach allows the management of a non-homogeneous building stock consisting of very diverse structural systems (i.e., spanning from existing buildings designed against gravity loads only to buildings that comply with outdated seismic codes) while providing satisfactory predictions of the seismic demand with minimum computational effort.

Measuring transparency in the social sciences: political science and international relations.

The scientific method is predicated on transparency-yet the pace at which transparent research practices are being adopted by the scientific community is slow. The replication crisis in psychology showed that published findings employing statistical inference are threatened by undetected errors, data manipulation and data falsification. To mitigate these problems and bolster research credibility, open data and preregistration practices have gained traction in the natural and social sciences. However, the extent of their adoption in different disciplines is unknown. We introduce computational procedures to identify the transparency of a research field using large-scale text analysis and machine learning classifiers. Using political science and international relations as an illustrative case, we examine 93 931 articles across the top 160 political science and international relations journals between 2010 and 2021. We find that approximately 21% of all statistical inference papers have open data and 5% of all experiments are preregistered. Despite this shortfall, the example of leading journals in the field shows that change is feasible and can be effected quickly.

Data-Driven and Machine Learning to Screen Metal-Organic Frameworks for the Efficient Separation of Methane.

With the rapid growth of the economy, people are increasingly reliant on energy sources. However, in recent years, the energy crisis has gradually intensified. As a clean energy source, methane has garnered widespread attention for its development and utilization. This study employed both large-scale computational screening and machine learning to investigate the adsorption and diffusion properties of thousands of metal-organic frameworks (MOFs) in six gas binary mixtures of CH4 (H2/CH4, N2/CH4, O2/CH4, CO2/CH4, H2S/CH4, He/CH4) for methane purification. Firstly, a univariate analysis was conducted to discuss the relationships between the performance indicators of adsorbents and their characteristic descriptors. Subsequently, four machine learning methods were utilized to predict the diffusivity/selectivity of gas, with the light gradient boosting machine (LGBM) algorithm emerging as the optimal one, yielding R2 values of 0.954 for the diffusivity and 0.931 for the selectivity. Furthermore, the LGBM algorithm was combined with the SHapley Additive exPlanation (SHAP) technique to quantitatively analyze the relative importance of each MOF descriptor, revealing that the pore limiting diameter (PLD) was the most critical structural descriptor affecting molecular diffusivity. Finally, for each system of CH4 mixture, three high-performance MOFs were identified, and the commonalities among high-performance MOFs were analyzed, leading to the proposals of three design principles involving changes only to the metal centers, organic linkers, or topological structures. Thus, this work reveals microscopic insights into the separation mechanisms of CH4 from different binary mixtures in MOFs.

A family of spectral conjugate gradient methods with strong convergence and its applications in image restoration and machine learning

In this paper, we propose a family of spectral conjugate gradient methods for solving unconstrained optimization problems. Specifically, we provide two classes of bounded spectral parameters to be chosen, design a new truncation scheme of the non-negative conjugate parameter and set a restart procedure in our search direction. Independently of the specific spectral parameter, conjugate parameter and line search criterion, we prove that our proposed family satisfies the sufficient descent condition. We also prove its strong convergence under mild assumptions and the weak Wolfe line search. Numerical comparisons with other methods demonstrate the outstanding performances of our algorithm for solving medium–large-scale unconstrained optimization, image restoration and machine learning problems.

Partial melting nature of phase-change memory Ge-Sb-Te superlattice uncovered by large-scale machine learning interatomic potential molecular dynamics

[GeTe]m-[Sb2Te3]n superlattice (GST-SL) is a promising material candidate to solve the critical problem of high-power consumption for phase-change memory technology. However, the switching mechanism is still under strong debate during the last decade. A key controversial question is that whether melting in GST-SL is possible. In this work, a large-scale machine learning interatomic potential (MLIP) molecular dynamics (MD) simulation with a one-order-of-magnitude larger atom number than that of conventional density functional theory (DFT) MD captures a unique partial melting behavior in GST-SL, where a sparse-nucleation-and-growth governed melting behavior is triggered by the flipping of atoms into van der Waals (vdW) gaps and forming cation-in-vdW-gap (CiV) defect as starting point, and then is slowly developed via propagating liquid-crystalline interfaces. In great contrast, the traditional melting of rock salt (RS)-GST occurs very fast due to instant homogeneous nucleation from high-density intrinsic vacancies distributed randomly in its cubic lattice. Therefore, the melting regions in GST-SL can be spatially localized in form of partial melting and are readily controlled. Moreover, the atomic distribution after melting in GST-SL is more chemical ordering than that in RS-GST. By the large-scale MLIP-MD, the present study provides a critical atomic insight into GST-SL phase-change behaviors that conventional DFT-MDs hardly achieve. This partial-melting nature in GST-SL helps explain the long-term-confused working principle of GST-SL-based phase-change memory, which will accelerate its application in the big-data era.

Stagewise Training With Exponentially Growing Training Sets.

In the world of big data, training large-scale machine learning problems has gained considerable attention. Numerous innovative optimization strategies have been presented in recent years to accelerate the large-scale training process. However, the possibility of further accelerating the training process of various optimization algorithms remains an unresolved subject. To begin addressing this difficult problem, we exploit the researched findings that when training data are independent and identically distributed, the learning problem on a smaller dataset is not significantly different from the original one. Upon that, we propose a stagewise training technique that grows the size of the training set exponentially while solving nonsmooth subproblem. We demonstrate that our stagewise training via exponentially growing the size of the training sets (STEGSs) are compatible with a large number of proximal gradient descent and gradient hard thresholding (GHT) techniques. Interestingly, we demonstrate that STEGS can greatly reduce overall complexity while maintaining statistical accuracy or even surpassing the intrinsic error introduced by GHT approaches. In addition, we analyze the effect of the training data growth rate on the overall complexity. The practical results of applying l2,1 -and l0 -norms to a variety of large-scale real-world datasets not only corroborate our theories but also demonstrate the benefits of our STEGS framework.

Multi-Endpoint Acute Toxicity Assessment of Organic Compounds Using Large-Scale Machine Learning Modeling.

In recent years, alternative animal testing methods such as computational and machine learning approaches have become increasingly crucial for toxicity testing. However, the complexity and scarcity of available biomedical data challenge the development of predictive models. Combining nonlinear machine learning together with multicondition descriptors offers a solution for using data from various assays to create a robust model. This work applies multicondition descriptors (MCDs) to develop a QSTR (Quantitative Structure-Toxicity Relationship) model based on a large toxicity data set comprising more than 80,000 compounds and 59 different end points (122,572 data points). The prediction capabilities of developed single-task multi-end point machine learning models as well as a novel data analysis approach with the use of Convolutional Neural Networks (CNN) are discussed. The results show that using MCDs significantly improves the model and using them with CNN-1D yields the best result (R2train = 0.93, R2ext = 0.70). Several structural features showed a high level of contribution to the toxicity, including van der Waals surface area (VSA), number of nitrogen-containing fragments (nN+), presence of S-P fragments, ionization potential, and presence of C-N fragments. The developed models can be very useful tools to predict the toxicity of various compounds under different conditions, enabling quick toxicity assessment of new compounds.

Usability of novel beat-by-beat ECG annotation software portal for generating large-scale machine learning training datasets

Abstract Introduction Despite the availability of numerous large publicly available ECG datasets, there is a distinct absence of specialized software to enable the beat-by-beat annotation and labeling, limiting the ability to generate high-quality machine learning training datasets. This study analyzed the usability of such a novel software interface. Through a survey of several cardiologists and clinic staff, several metrics were gauged, including user satisfaction, system and app simplicity, user interface, and effectiveness. The survey results gleaned insight into the users’ experiences, providing an understanding of the software’s user-friendliness and potential in developing a large-scale machine learning annotated ECG training dataset to advance cardiac health research. Methods 3 cardiology physicians, and 9 cardiac telemetry unit staff were surveyed following use of the SafeBeat Annotation Software Platform to annotate a large publicly available dataset ECGs (n=10,646) beat-by-beat. The validated Post-Study System Usability Questionnaire (PSSUQ) and mobile Health App Usability Questionnaire (MAUQ) were used to assess the overall usability of the software interface. Results are reported as mean ± standard deviation. Results Users completed 10,646 ECG beat-by-beat annotations in 62 days, and averaged 164.0 ECGs per day. Physicians rated the software interface highly, with an overall mean PSSUQ of 6.4 ± 0.8 and MAUQ of 6.4 ± 0.7. Staff also rated the software interface highly, with an overall mean PSSUQ of 6.7 ± 0.6 and MAUQ of 6.8 ± 0.5. Overall 92% of the users reported that they strongly agreed they would use the application again and 83% strongly agreed that it was easy to learn how to use the system. Conclusion The usability analysis of this novel beat-by-beat ECG annotation software revealed that it was a highly effective tool for both cardiologists and clinic staff. Both the system and app were reported to be simple to learn, comfortable to use, and easily navigable. With most users responding that they could become productive quickly using this system, there is clear potential for it to be used to develop a large-scale machine learning dataset.Figure 1