Real-world Application Domains Research Articles

Cost-sensitive ordinal classification methods to predict SARS-CoV-2 pneumonia severity.

To study the suitability of costsensitive ordinal artificial intelligence-machine learning (AIML) strategies in the prognosis of SARS-CoV-2 pneumonia severity. Observational, retrospective, longitudinal, cohort study in 4 hospitals in Spain. Information regarding demographic and clinical status was supplemented by socioeconomic data and air pollution exposures. We proposed AI-ML algorithms for ordinal classification via ordinal decomposition and for cost-sensitive learning via resampling techniques. For performance-based model selection, we defined a custom score including per-class sensitivities and asymmetric misprognosis costs. 260 distinct AI-ML models were evaluated via 10 repetitions of 5×5 nested cross-validation with hyperparameter tuning. Model selection was followed by the calibration of predicted probabilities. Final overall performance was compared against five well-established clinical severity scores and against a 'standard' (non-cost sensitive, non-ordinal) AI-ML baseline. In our best model, we also evaluated its explainability with respect to each of the input variables. The study enrolled n = 1548 patients: 712 experienced low, 238 medium, and 598 high clinical severity. d = 131 variables were collected, becoming d ' = 148 features after categorical encoding. Model selection resulted in our best-performing AI-ML pipeline having: a) no imputation of missing data, b) no feature selection (i.e. using the full set of d ' features), c) 'Ordered Partitions' ordinal decomposition, d) cost-based reimbalance, and e) a Histogram-based Gradient Boosting classifier. This best model (calibrated) obtained a median accuracy of 68.1% [67.3%, 68.8%] (95% confidence interval), a balanced accuracy of 57.0% [55.6%, 57.9%], and an overall area under the curve (AUC) 0.802 [0.795, 0.808]. In our dataset, it outperformed all five clinical severity scores and the 'standard' AI-ML baseline. We conducted an exhaustive exploration of AI-ML methods designed for both ordinal and cost-sensitive classification, motivated by a real-world application domain (clinical severity prognosis) in which these topics arise naturally. Our model with the best classification performance exploited successfully the ordering information of ground truth classes, coping with imbalance and asymmetric costs. However, these ordinal and cost-sensitive aspects are seldom explored in the literature.

Assisted design of data science pipelines

AbstractWhen designing data science (DS) pipelines, end-users can get overwhelmed by the large and growing set of available data preprocessing and modeling techniques. Intelligent discovery assistants (IDAs) and automated machine learning (AutoML) solutions aim to facilitate end-users by (semi-)automating the process. However, they are expensive to compute and yield limited applicability for a wide range of real-world use cases and application domains. This is due to (a) their need to execute thousands of pipelines to get the optimal one, (b) their limited support of DS tasks, e.g., supervised classification or regression only, and a small, static set of available data preprocessing and ML algorithms; and (c) their restriction to quantifiable evaluation processes and metrics, e.g., tenfold cross-validation using the ROC AUC score for classification. To overcome these limitations, we propose a human-in-the-loop approach for the assisteddesignofdatasciencepipelines using previously executed pipelines. Based on a user query, i.e., data and a DS task, our framework outputs a ranked list of pipeline candidates from which the user can choose to execute or modify in real time. To recommend pipelines, it first identifies relevant datasets and pipelines utilizing efficient similarity search. It then ranks the candidate pipelines using multi-objective sorting and takes user interactions into account to improve suggestions over time. In our experimental evaluation, the proposed framework significantly outperforms the state-of-the-art IDA tool and achieves similar predictive performance with state-of-the-art long-running AutoML solutions while being real-time, generic to any evaluation processes and DS tasks, and extensible to new operators.

Finding Subgraphs with Maximum Total Density and Limited Overlap in Weighted Hypergraphs

Finding dense subgraphs in large (hyper)graphs is a key primitive in a variety of real-world application domains, encompassing social network analytics, event detection, biology, and finance. In most such applications, one typically aims at finding several (possibly overlapping) dense subgraphs, which might correspond to communities in social networks or interesting events. While a large amount of work is devoted to finding a single densest subgraph, perhaps surprisingly, the problem of finding several dense subgraphs in weighted hypergraphs with limited overlap has not been studied in a principled way, to the best of our knowledge. In this work, we define and study a natural generalization of the densest subgraph problem in weighted hypergraphs, where the main goal is to find at most k subgraphs with maximum total aggregate density, while satisfying an upper bound on the pairwise weighted Jaccard coefficient, i.e., the ratio of weights of intersection divided by weights of union on two nodes sets of the subgraphs. After showing that such a problem is NP-Hard, we devise an efficient algorithm that comes with provable guarantees in some cases of interest, as well as, an efficient practical heuristic. Our extensive evaluation on large real-world hypergraphs confirms the efficiency and effectiveness of our algorithms.

Feature extraction and representation learning of 3D point cloud data

Three-dimensional point cloud data serves as a critical source of information in various real-world application domains, such as computer vision, robotics, geographic information systems, and medical image processing. Due to the discrete and unordered nature of point clouds, applying 2D image feature extractors directly to the extraction of 3D point cloud features is challenging. Therefore, we propose a novel variational feature component extraction method called PointFEA. This paper aims to research and propose a series of methods to enhance the feature extraction and representation learning of 3D point cloud data. Firstly, in terms of feature extraction, local neighborhood encoding is combined with the local latent representation of point clouds to obtain more correlated point cloud features. Secondly, in the domain of point cloud representation learning, the multi-scale representation learning method maps point cloud data into a high-dimensional space to better capture critical features and adapt to different granularities of point cloud data. Lastly, features of different dimensions are input into a cross-fusion transformer to obtain local attention coefficients. We validate our methods on commonly used point cloud datasets, and the experiments demonstrate the effectiveness of our approach, achieving accuracies of 94.8% on ModelNet40 and 89.1% on ScanObjectNN.

Coupled Attention Networks for Multivariate Time Series Anomaly Detection

Multivariate time series anomaly detection (MTAD) plays a vital role in a wide variety of real-world application domains. Over the past few years, MTAD has attracted rapidly increasing attention from both academia and industry. Many deep learning and graph learning models have been developed for effective anomaly detection in multivariate time series data, which enable advanced applications such as smart surveillance and risk management with unprecedented capabilities. Nevertheless, MTAD is facing critical challenges deriving from the dependencies among sensors and variables, which often change over time. To address this issue, we propose a coupled attention-based neural network framework (CAN) for anomaly detection in multivariate time series data featuring dynamic variable relationships. We combine adaptive graph learning methods with graph attention to generate a global-local graph that can represent both global correlations and dynamic local correlations among sensors. To capture inter-sensor relationships and temporal dependencies, a convolutional neural network based on the global-local graph is integrated with a temporal self-attention module to construct a coupled attention module. In addition, we develop a multilevel encoder-decoder architecture that accommodates reconstruction and prediction tasks to better characterize multivariate time series data. Extensive experiments on real-world datasets have been conducted to evaluate the performance of the proposed CAN approach, and the results show that CAN significantly outperforms state-of-the-art baselines.

The dynamical study and analysis of diverse bright-dark and breathers wave solutions of nonlinear evolution equations and their applications

In this work, we establish novel solitary wave solutions of three nonlinear evolution dynamical models, namely, Boussinesq model, [Formula: see text]-dimensional generalized shallow water wave (SWW) and [Formula: see text]-dimensional Fokas dynamical models by applying two-variable [Formula: see text]- expansion technique. The SWW equations are usually appropriate when the fluid is in a state where the vertical length scale is significantly smaller than the length of horizontal scale. In computer simulations for ocean engineering, Boussinesq-type models are frequently used to represent water waves in harbors and shallow seas. In order to understand the physical processes of waves inside and on the surface of water, the Fokas dynamical model plays a crucial part in wave theory. Analytical solutions in different forms such as solitons, solitary waves, trigonometric, hyperbolic, rational function solutions, breathers-type waves and more wave solutions are devised through using the proposed method. The exact solutions are also presented in graphical form having applications in engineering and other areas of applied sciences. The obtained results show that the given technique is universal and efficient. In addition, this technique can also be applied on lots of other nonlinear dynamical wave models occurring in many scientific real-world application domains, including engineering sciences, mathematics physics, and many more.

Anthropogenic Object Localization: Evaluation of Broad-Area High-Resolution Imagery Scans Using Deep Learning in Overhead Imagery.

Too often, the testing and evaluation of object detection, as well as the classification techniques for high-resolution remote sensing imagery, are confined to clean, discretely partitioned datasets, i.e., the closed-world model. In recent years, the performance on a number of benchmark datasets has exceeded 99% when evaluated using cross-validation techniques. However, real-world remote sensing data are truly big data, which often exceed billions of pixels. Therefore, one of the greatest challenges regarding the evaluation of machine learning models taken out of the clean laboratory setting and into the real world is the difficulty of measuring performance. It is necessary to evaluate these models on a grander scale, namely, tens of thousands of square kilometers, where it is intractable to the ground truth and the ever-changing anthropogenic surface of Earth. The ultimate goal of computer vision model development for automated analysis and broad area search and discovery is to augment and assist humans, specifically human-machine teaming for real-world tasks. In this research, various models have been trained using object classes from benchmark datasets such as UC Merced, PatternNet, RESISC-45, and MDSv2. We detail techniques to scan broad swaths of the Earth with deep convolutional neural networks. We present algorithms for localizing object detection results, as well as a methodology for the evaluation of the results of broad-area scans. Our research explores the challenges of transitioning these models out of the training-validation laboratory setting and into the real-world application domain. We show a scalable approach to leverage state-of-the-art deep convolutional neural networks for the search, detection, and annotation of objects within large swaths of imagery, with the ultimate goal of providing a methodology for evaluating object detection machine learning models in real-world scenarios.

Clustered Task-Aware Meta-Learning by Learning From Learning Paths.

To enable effective learning of new tasks with only a few examples, meta-learning acquires common knowledge from the existing tasks with a globally shared meta-learner. To further address the problem of task heterogeneity, recent developments balance between customization and generalization by incorporating task clustering to generate task-aware modulation to be applied to the global meta-learner. However, these methods learn task representation mostly from the features ofinput data, while the task-specific optimization process with respect to the base-learner is often neglected. In this work, we propose a Clustered Task-Aware Meta-Learning (CTML) framework with task representation learned from both features and learning paths. We first conduct rehearsed task learning from the common initialization, and collect a set of geometric quantities that adequately describes this learning path. By inputting this set of values into a meta path learner, we automatically abstract path representation optimized for downstream clustering and modulation. Aggregating the path and feature representations results in an improved task representation. To further improve inference efficiency, we devise a shortcut tunnel to bypass the rehearsed learning process at a meta-testing time. Extensive experiments on two real-world application domains: few-shot image classification and cold-start recommendation demonstrate the superiority of CTML compared to state-of-the-art methods. We provide our code at https://github.com/didiya0825.

Synthesizing credit data using autoencoders and generative adversarial networks

Data quality is an essential element necessary for the development of a successful machine-learning project. One of the biggest challenges in various real-world application domains is class imbalance. This paper proposes a new framework for oversampling credit data by combining two deep learning techniques: autoencoders and generative adversarial networks. A trivial autoencoder (TAE) is used to change data representation, and modified generative adversarial networks (GAN) are used to create new instances from random noise. The experiment on three different datasets demonstrates that the same classifier achieves a better area under the receiver operating characteristic curve (AUC) on datasets augmented by the proposed framework compared to datasets oversampled by other techniques. Additionally, the results show that datasets balanced by the new framework influence the classifier to change the prediction error types, significantly reducing false negatives; more expensive misclassification case in the imbalance learning. The improvements are significant, and considering the change in error distribution, the proposed technique is an excellent complement to existing oversampling techniques.

CEEMD-MultiRocket: Integrating CEEMD with Improved MultiRocket for Time Series Classification

Time series classification (TSC) is always a very important research topic in many real-world application domains. MultiRocket has been shown to be an efficient approach for TSC, by adding multiple pooling operators and a first-order difference transformation. To classify time series with higher accuracy, this study proposes a hybrid ensemble learning algorithm combining Complementary Ensemble Empirical Mode Decomposition (CEEMD) with improved MultiRocket, namely CEEMD-MultiRocket. Firstly, we utilize the decomposition method CEEMD to decompose raw time series into three sub-series: two Intrinsic Mode Functions (IMFs) and one residue. Then, the selection of these decomposed sub-series is executed on the known training set by comparing the classification accuracy of each IMF with that of raw time series using a given threshold. Finally, we optimize convolution kernels and pooling operators, and apply our improved MultiRocket to the raw time series, the selected decomposed sub-series and the first-order difference of the raw time series to generate the final classification results. Experiments were conducted on 109 datasets from the UCR time series repository to assess the classification performance of our CEEMD-MultiRocket. The extensive experimental results demonstrate that our CEEMD-MultiRocket has the second-best average rank on classification accuracy against a spread of the state-of-the-art (SOTA) TSC models. Specifically, CEEMD-MultiRocket is significantly more accurate than MultiRocket even though it requires a relatively long time, and is competitive with the currently most accurate model, HIVE-COTE 2.0, only with 1.4% of the computing load of the latter.

A memetic algorithm for finding multiple subgraphs that optimally cover an input network.

Finding dense subgraphs is a central problem in graph mining, with a variety of real-world application domains including biological analysis, financial market evaluation, and sociological surveys. While a series of studies have been devoted to finding subgraphs with maximum density, the problem of finding multiple subgraphs that best cover an input network has not been systematically explored. The present study discusses a variant of the densest subgraph problem and presents a mathematical model for optimizing the total coverage of an input network by extracting multiple subgraphs. A memetic algorithm that maximizes coverage is proposed and shown to be both effective and efficient. The method is applied to real-world networks. The empirical meaning of the optimal sampling method is discussed.

Group-preserving label-specific feature selection for multi-label learning

In many real-world application domains, e.g., text categorization and image annotation, objects naturally belong to more than one class label, giving rise to the multi-label learning problem. The performance of multi-label learning greatly relies on the quality of available features, whereas the data generally involve a lot of irrelevant, redundant, even noisy features. This fact has led to that a surge of research on feature selection methods that select significant features for multi-label learning. Nevertheless, most of the previous approaches suffer from the deficiency that label-specific features are not taken into account, and they are also inefficient in exploiting labeling information such as local label correlations. Moreover, these methods lack interpretability, which can only find a feature subset for all labels, however, cannot show how features are related to different labels. Based on this, we present a new group-preserving label-specific feature selection (GLFS) framework for multi-label learning, which simultaneously considers the features special to the labels in the same group and specific features owned by each label to execute feature selection. In addition, we further consider to learn label-group and instance-group correlations for the exploitation of labeling information, and make a collaborative use of them to improve the model generalization. Extensive experiments validate the advantages of the proposed GLFS method.

DORIAN in action

Existing automated machine learning solutions and intelligent discovery assistants are popular tools that facilitate the end-user with the design of data science (DS) pipelines. However, they yield limited applicability for a wide range of real-world use cases and application domains due to (a) the limited support of DS tasks; (b) a small, static set of available operators; and (c) restriction to evaluation processes with quantifiable loss functions. We demonstrate DORIAN, a human-in-the-loop approach for the assisted design of data science pipelines that supports a large and growing set of DS tasks, operators, and arbitrary user-defined evaluation processes. Based on the user query, i.e., a dataset and a DS task, DORIAN computes a ranked list of candidate pipelines that the end-user can choose from, alter, execute and evaluate. It stores executed pipelines in an experiment database and utilizes similarity-based search to identify relevant previously-run pipelines from the experiment database. DORIAN also takes user interaction into account to improve suggestions over time. We show how users can interact with DORIAN to create and compare DS pipelines on various real-world DS tasks without the need for writing any code.

Efficient Coalition Structure Generation via Approximately Equivalent Induced Subgraph Games.

We show that any characteristic function game (CFG) G can be always turned into an approximately equivalent game represented using the induced subgraph game (ISG) representation. Such a transformation incurs obvious benefits in terms of tractability of computing solution concepts for G . Our transformation approach, namely, AE-ISG, is based on the solution of a norm approximation problem. We then propose a novel coalition structure generation (CSG) approach for ISGs that is based on graph clustering, which outperforms existing CSG approaches for ISGs by using off-the-shelf optimization solvers. Finally, we provide theoretical guarantees on the value of the optimal CSG solution of G with respect to the optimal CSG solution of the approximately equivalent ISG. As a consequence, our approach allows one to compute approximate CSG solutions with quality guarantees for any CFG. Results on a real-world application domain show that our approach outperforms a domain-specific CSG algorithm, both in terms of quality of the solutions and theoretical quality guarantees.

Data Science and Analytics: An Overview from Data-Driven Smart Computing, Decision-Making and Applications Perspective.

The digital world has a wealth of data, such as internet of things (IoT) data, business data, health data, mobile data, urban data, security data, and many more, in the current age of the Fourth Industrial Revolution (Industry 4.0 or 4IR). Extracting knowledge or useful insights from these data can be used for smart decision-making in various applications domains. In the area of data science, advanced analytics methods including machine learning modeling can provide actionable insights or deeper knowledge about data, which makes the computing process automatic and smart. In this paper, we present a comprehensive view on “Data Science” including various types of advanced analytics methods that can be applied to enhance the intelligence and capabilities of an application through smart decision-making in different scenarios. We also discuss and summarize ten potential real-world application domains including business, healthcare, cybersecurity, urban and rural data science, and so on by taking into account data-driven smart computing and decision making. Based on this, we finally highlight the challenges and potential research directions within the scope of our study. Overall, this paper aims to serve as a reference point on data science and advanced analytics to the researchers and decision-makers as well as application developers, particularly from the data-driven solution point of view for real-world problems.

Machine Learning: Algorithms, Real-World Applications and Research Directions.

In the current age of the Fourth Industrial Revolution (4IR or Industry 4.0), the digital world has a wealth of data, such as Internet of Things (IoT) data, cybersecurity data, mobile data, business data, social media data, health data, etc. To intelligently analyze these data and develop the corresponding smart and automated applications, the knowledge of artificial intelligence (AI), particularly, machine learning (ML) is the key. Various types of machine learning algorithms such as supervised, unsupervised, semi-supervised, and reinforcement learning exist in the area. Besides, the deep learning, which is part of a broader family of machine learning methods, can intelligently analyze the data on a large scale. In this paper, we present a comprehensive view on these machine learning algorithms that can be applied to enhance the intelligence and the capabilities of an application. Thus, this study’s key contribution is explaining the principles of different machine learning techniques and their applicability in various real-world application domains, such as cybersecurity systems, smart cities, healthcare, e-commerce, agriculture, and many more. We also highlight the challenges and potential research directions based on our study. Overall, this paper aims to serve as a reference point for both academia and industry professionals as well as for decision-makers in various real-world situations and application areas, particularly from the technical point of view.

Dynamic simulation of gas turbines via feature similarity-based transfer learning

Since gas turbine plays a key role in electricity power generating, the requirements on the safety and reliability of this classical thermal system are becoming gradually strict. With a large amount of renewable energy being integrated into the power grid, the request of deep peak load regulation for satisfying the varying demand of users and maintaining the stability of the whole power grid leads to more unstable working conditions of gas turbines. The startup, shutdown, and load fluctuation are dominating the operating condition of gas turbines. Hence simulating and analyzing the dynamic behavior of the engines under such instable working conditions are important in improving their design, operation, and maintenance. However, conventional dynamic simulation methods based on the physic differential equations is unable to tackle the uncertainty and noise when faced with variant real-world operations. Although data-driven simulating methods, to some extent, can mitigate the problem, it is impossible to perform simulations with insufficient data. To tackle the issue, a novel transfer learning framework is proposed to transfer the knowledge from the physics equation domain to the real-world application domain to compensate for the lack of data. A strong dynamic operating data set with steep slope signals is created based on physics equations and then a feature similarity-based learning model with an encoder and a decoder is built and trained to achieve feature adaptive knowledge transferring. The simulation accuracy is significantly increased by 24.6% and the predicting error reduced by 63.6% compared with the baseline model. Moreover, compared with the other classical transfer learning modes, the method proposed has the best simulating performance on field testing data set. Furthermore, the effect study on the hyper parameters indicates that the method proposed is able to adaptively balance the weight of learning knowledge from the physical theory domain or from the real-world operation domain.

Visual interpretation of regression error

AbstractSeveral sophisticated machine learning tools (e.g., ensembles or deep networks) have shown outstanding performance in different regression forecasting tasks. In many real world application domains the numeric predictions of the models drive important and costly decisions. Nevertheless, decision makers frequently require more than a black box model to be able to “trust” the predictions up to the point that they base their decisions on them. In this context, understanding these black boxes has become one of the hot topics in Machine Learning research. This paper proposes a series of visualization tools that explain the relationship between the expected predictive performance of black box regression models and the values of the input variables of any given test case. This type of information thus allows end‐users to correctly assess the risks associated with the use of a model, by showing how concrete values of the predictors may affect the performance of the model. Our illustrations with different real world data sets and learning algorithms provide insights on the type of usage and information these tools bring to both the data analyst and the end‐user. Furthermore, a thorough evaluation of the proposed tools is performed to showcase the reliability of this approach.

Mobile Apps as Personal Assistant Agents: the JaCa-Android Framework for programming Agents-based applications on mobile devices

A relevant application domain for agent-based software is given by mobile and wearable applications. In this context, the impressive progress of technologies in the last decade makes it possible to explore the use of agent-oriented programming languages and frameworks based on cognitive architectures, such as the Belief–Desire–Intention (BDI) one. Accordingly, in this paper we provide a comprehensive description of the JaCa-Android approach, a framework based on the JaCaMo platform that allows for designing and programming smart mobile apps using cognitive agents based on the BDI architecture and the Agents & Artifacts environment conceptual model. In these years, the framework has been applied in real-world projects and application domains, and extended and evolved accordingly. The aim of the paper is to report our experience about designing and programming mobile apps as personal assistant agents, as well as to discuss in detail the architecture of the framework.

A novel validation framework to enhance deep learning models in time-series forecasting

Time-series analysis and forecasting is generally considered as one of the most challenging problems in data mining. During the last decade, powerful deep learning methodologies have been efficiently applied for time-series forecasting; however, they cannot guarantee the development of reliable prediction models. In this work, we introduce a novel framework for supporting deep learning in enhancing accurate, efficient and reliable time-series models. The major novelty of our proposed methodology is that it ensures a time-series to be “suitable” for fitting a deep learning model by performing a series of transformations in order to satisfy the stationarity property. The enforcement of stationarity is performed by the application of Augmented Dickey–Fuller test and transformations based on first differences or returns, without the loss of any embedded information. The reliability of the deep learning model’s predictions is guaranteed by rejecting the hypothesis of autocorrelation in the model’s errors, which is demonstrated by autocorrelation function plots and Ljung–Box Q test. Our numerical experiments were performed utilizing time-series from three real-world application domains (financial market, energy sector, cryptocurrency area), which incorporate most of global research interest. The performance of all forecasting models was compared on both problems of forecasting time-series price (regression) and time-series directional movements (classification). Additionally, the reliability of the models’ forecasts was evaluated by examining the existence of autocorrelation in the errors. Our numerical experiments indicate that our proposed methodology considerably improves the forecasting performance of a deep learning model, in terms of efficiency, accuracy and reliability.