Real-world Tasks Research Articles

Chaotic behavior learning via information tracking

Chaotic time series are often encountered in real-world applications, where modeling and understanding such time series present a significant challenge. Existing simulation-based methods for characterizing chaotic behavior may be sensitive to the respective model settings, while data-driven methods cannot adapt well to irregularities and aperiodicity in chaotic time series during the prediction stage, leading to nonlinearly aggregated forecasting errors after a finite number of time steps. In our preliminary work (Ren et al., 2023), we introduced a data-driven method that incorporates the idea of information tracking to capture and adapt to the chaotic changes over time, and tested its effectiveness on a real-world task—decadal temperature prediction. While the practical usefulness of the method has been initially validated on one domain-specific application scenario, a set of open questions is yet to be answered: Is the concept of information tracking generalizable to model and solve a broad spectrum of learning tasks that involve chaotic behavior? If yes, what are the underlying principles and learning behaviors that make the general method work well? Moreover, how can we assess the efficacy of the method in different scenarios? This study constitutes a significant step forward by systematically investigating and addressing the questions mentioned above. Specifically, we first provide a rigorous definition of the formalism and computational mechanism of the general method, with all necessary variables, symbols, and formulations. More importantly, we theoretically characterize and understand the generalizability of the method by mathematically uncovering the fundamental principles and behavior of information tracking-based chaotic behavioral learning, as prescribed by two chaotic behavioral indexes, namely, the Lyapunov and the Hurst exponents. Further, we comprehensively demonstrate the generalizability and robustness of the method by empirically examining a variety of chaotic behavioral learning problems generated by three representative chaotic systems: The logistic map, the double pendulum, and the Lorenz system.

Anthropogenic Object Localization: Evaluation of Broad-Area High-Resolution Imagery Scans Using Deep Learning in Overhead Imagery.

Too often, the testing and evaluation of object detection, as well as the classification techniques for high-resolution remote sensing imagery, are confined to clean, discretely partitioned datasets, i.e., the closed-world model. In recent years, the performance on a number of benchmark datasets has exceeded 99% when evaluated using cross-validation techniques. However, real-world remote sensing data are truly big data, which often exceed billions of pixels. Therefore, one of the greatest challenges regarding the evaluation of machine learning models taken out of the clean laboratory setting and into the real world is the difficulty of measuring performance. It is necessary to evaluate these models on a grander scale, namely, tens of thousands of square kilometers, where it is intractable to the ground truth and the ever-changing anthropogenic surface of Earth. The ultimate goal of computer vision model development for automated analysis and broad area search and discovery is to augment and assist humans, specifically human-machine teaming for real-world tasks. In this research, various models have been trained using object classes from benchmark datasets such as UC Merced, PatternNet, RESISC-45, and MDSv2. We detail techniques to scan broad swaths of the Earth with deep convolutional neural networks. We present algorithms for localizing object detection results, as well as a methodology for the evaluation of the results of broad-area scans. Our research explores the challenges of transitioning these models out of the training-validation laboratory setting and into the real-world application domain. We show a scalable approach to leverage state-of-the-art deep convolutional neural networks for the search, detection, and annotation of objects within large swaths of imagery, with the ultimate goal of providing a methodology for evaluating object detection machine learning models in real-world scenarios.

Corporate Mole: A Grouping Strategy for Minimizing the Free-Rider Problem in a Managerial Finance Course

Group projects in upper level finance courses are becoming common, both for the reflection of real-world group tasks in business firms and for their appropriateness in dealing with certain types of assignments in the classroom and beyond. Professors often lament over effective formation of groups and the task of managing group dynamics, including the problem of free-riders: those students who, knowing that the group gets the grade, will shirk their respective responsibilities in accomplishing group tasks. The following article represents one evolution of grouping methods over two decades and the observations surrounding each. The most successful, in this author’s experience, is a recent marriage of a so-called “reality TV” recipe with structured groupings

A novel phishing website classification method based on hybrid sampling

ABSTRACT In real-world anomaly detection tasks such as Credit Card Fraud Detection, Cancer Patients Detection, Phishing Website Detection, etc., the training datasets often suffer from skewed class distribution. But the traditional Machine Learning (ML) classification algorithms assume balanced class distribution and equal misclassification costs. As a result, when class-imbalanced data are presented to the traditional ML algorithms they tend to produce biased and inaccurate predictive ML models. In this study, we propose four novel Phishing Website Classification models namely, SMOTEENN-XGB, SMOTEENN-RF, SMOTEENN-LR, and SMOTEENN-SVM by combining SMOTEENN (SMOTE + ENN) hybrid sampling technique with eXtreme Gradient Boosting (XGB), Random Forest (RF), Logistic Regression (LR), and Support Vector Machine (SVM) classifiers respectively. We propose the use of SMOTEENN hybrid sampling as the novel approach to address the problem of class imbalance in Phishing Website datasets prior to building classification models. To the best of our knowledge and belief, our novel proposed four models SMOTEENN-XGB, SMOTEENN-RF, SMOTEEEN-LR, and SMOTEENN-SVM for Phishing Website Detection based on SMOTEENN hybrid sampling approach have not been published in the existing studies as of now.

A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification

Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications. Recent advances further enable adapting foundation models in downstream tasks efficiently using only a few training samples, e.g., in-context learning. Yet, the application of such learning paradigms in medical image analysis remains scarce due to the shortage of publicly accessible data and benchmarks. In this paper, we aim at approaches adapting the foundation models for medical image classification and present a novel dataset and benchmark for the evaluation, i.e., examining the overall performance of accommodating the large-scale foundation models downstream on a set of diverse real-world clinical tasks. We collect five sets of medical imaging data from multiple institutes targeting a variety of real-world clinical tasks (22,349 images in total), i.e., thoracic diseases screening in X-rays, pathological lesion tissue screening, lesion detection in endoscopy images, neonatal jaundice evaluation, and diabetic retinopathy grading. Results of multiple baseline methods are demonstrated using the proposed dataset from both accuracy and cost-effective perspectives.

Concept-Level Model Interpretation From the Causal Aspect

With the increasing growth of data and the ability of learning with them, machine learning models are adopted in various domains. However, few of machine learning models are able to reason their prediction, which limits their further applications in real-world tasks. With the potential to address this dilemma, model interpretation has become an important research topic because of the ability to provide the underlying reasons for model predictions at the feature level or concept level. Model interpretation at the concept level focuses on exploring the roles of concepts in model prediction, which enables more compact and understandable interpretations. Concept-level model interpretation requires the identification of the concepts that contribute to model prediction and the exploration of the rules underneath these concepts. To achieve the two objectives, we propose a Concept-level Model Interpretation framework (CMIC) from the perspective of causality. CMIC can automatically detect concepts in data and discover the causal relation between the detected concepts and the model's predicted labels. Furthermore, CMIC ranks the contributions of concepts by their causal effect on the model prediction, reflecting the detected concepts' importance. We evaluate the proposed CMIC framework on both synthetic and real-world datasets to demonstrate the quality of the provided interpretation.

Multifingered Robot Hand Compliant Manipulation Based on Vision-Based Demonstration and Adaptive Force Control.

Multifingered hand dexterous manipulation is quite challenging in the domain of robotics. One remaining issue is how to achieve compliant behaviors. In this work, we propose a human-in-the-loop learning-control approach for acquiring compliant grasping and manipulation skills of a multifinger robot hand. This approach takes the depth image of the human hand as input and generates the desired force commands for the robot. The markerless vision-based teleoperation system is used for the task demonstration, and an end-to-end neural network model (i.e., TeachNet) is trained to map the pose of the human hand to the joint angles of the robot hand in real-time. To endow the robot hand with compliant human-like behaviors, an adaptive force control strategy is designed to predict the desired force control commands based on the pose difference between the robot hand and the human hand during the demonstration. The force controller is derived from a computational model of the biomimetic control strategy in human motor learning, which allows adapting the control variables (impedance and feedforward force) online during the execution of the reference joint angles. The simultaneous adaptation of the impedance and feedforward profiles enables the robot to interact with the environment compliantly. Our approach has been verified in both simulation and real-world task scenarios based on a multifingered robot hand, that is, the Shadow Hand, and has shown more reliable performances than the current widely used position control mode for obtaining compliant grasping and manipulation behaviors.

Versatile multicontact planning and control for legged loco-manipulation.

Loco-manipulation planning skills are pivotal for expanding the utility of robots in everyday environments. These skills can be assessed on the basis of a system's ability to coordinate complex holistic movements and multiple contact interactions when solving different tasks. However, existing approaches have been merely able to shape such behaviors with hand-crafted state machines, densely engineered rewards, or prerecorded expert demonstrations. Here, we propose a minimally guided framework that automatically discovers whole-body trajectories jointly with contact schedules for solving general loco-manipulation tasks in premodeled environments. The key insight is that multimodal problems of this nature can be formulated and treated within the context of integrated task and motion planning (TAMP). An effective bilevel search strategy was achieved by incorporating domain-specific rules and adequately combining the strengths of different planning techniques: trajectory optimization and informed graph search coupled with sampling-based planning. We showcase emergent behaviors for a quadrupedal mobile manipulator exploiting both prehensile and nonprehensile interactions to perform real-world tasks such as opening/closing heavy dishwashers and traversing spring-loaded doors. These behaviors were also deployed on the real system using a two-layer whole-body tracking controller.

Improving the matching of deformable objects by learning to detect keypoints

We propose a novel learned keypoint detection method to increase the number of correct matches for the task of non-rigid image correspondence. By leveraging true correspondences acquired by matching annotated image pairs with a specified descriptor extractor, we train an end-to-end convolutional neural network (CNN) to find keypoint locations that are more appropriate to the considered descriptor. Experiments demonstrate that our method enhances the Mean Matching Accuracy of numerous descriptors when used in conjunction with our detection method, while outperforming the state-of-the-art keypoint detectors on real images of non-rigid objects by 20 p.p. We also apply our method on the complex real-world task of object retrieval where our detector performs on par with the finest keypoint detectors currently available for this task. The source code and trained models are publicly available at https://github.com/verlab/LearningToDetect_PRL_2023.

Astrometric calibration and source characterisation of the latest generation neuromorphic event-based cameras for space imaging

As an emerging approach to space situational awareness and space imaging, the practical use of an event-based camera (EBC) in space imaging for precise source analysis is still in its infancy. The nature of event-based space imaging and data collection needs to be further explored to develop more effective event-based space imaging systems and advance the capabilities of event-based tracking systems with improved target measurement models. Moreover, for event measurements to be meaningful, a framework must be investigated for EBC calibration to project events from pixel array coordinates in the image plane to coordinates in a target resident space object’s reference frame. In this paper, the traditional techniques of conventional astronomy are reconsidered to properly utilise the EBC for space imaging and space situational awareness. This paper presents the techniques and systems used for calibrating an EBC for reliable and accurate measurement acquisition. These techniques are vital in building event-based space imaging systems capable of real-world space situational awareness tasks. By calibrating sources detected using the EBC, the spatiotemporal characteristics of detected sources or “event sources” can be related to the photometric characteristics of the underlying astrophysical objects. Finally, these characteristics are analysed to establish a foundation for principled processing and observing techniques which appropriately exploit the capabilities of the EBC.

Release of cognitive and multimodal MRI data including real-world tasks and hippocampal subfield segmentations

We share data from N = 217 healthy adults (mean age 29 years, range 20–41; 109 females, 108 males) who underwent extensive cognitive assessment and neuroimaging to examine the neural basis of individual differences, with a particular focus on a brain structure called the hippocampus. Cognitive data were collected using a wide array of questionnaires, naturalistic tests that examined imagination, autobiographical memory recall and spatial navigation, traditional laboratory-based tests such as recalling word pairs, and comprehensive characterisation of the strategies used to perform the cognitive tests. 3 Tesla MRI data were also acquired and include multi-parameter mapping to examine tissue microstructure, diffusion-weighted MRI, T2-weighted high-resolution partial volume structural MRI scans (with the masks of hippocampal subfields manually segmented from these scans), whole brain resting state functional MRI scans and partial volume high resolution resting state functional MRI scans. This rich dataset will be of value to cognitive and clinical neuroscientists researching individual differences, real-world cognition, brain-behaviour associations, hippocampal subfields and more. All data are freely available on Dryad.

Time Series Forest Fire Prediction Based on Improved Transformer

Forest fires, severe natural disasters causing substantial damage, necessitate accurate predictive modeling to guide preventative measures effectively. This study introduces an enhanced window-based Transformer time series forecasting model aimed at improving the precision of forest fire predictions. Leveraging time series data from 2020 to 2021 in Chongli, a myriad of forest fire influencing factors were ascertained using remote sensing satellite and GIS technologies, with their interrelationships estimated through a multicollinearity test. Given the intricate nature of real-world forest fire prediction tasks, we propose a novel window-based Transformer architecture complemented by a dual time series input strategy premised on 13 influential factors. Subsequently, time series data were incorporated into the model to generate a forest fire risk prediction map in Chongli District. The model’s effectiveness was then evaluated using various metrics, including accuracy (ACC), root mean square error (RMSE), and mean absolute error (MAE), and compared with traditional deep learning methods. Our model demonstrated superior predictive performance (ACC = 91.56%, RMSE = 0.37, MAE = 0.05), harnessing spatial background information efficiently and effectively utilizing the periodicity of forest fire factors. Consequently, the study proves this method to be a novel and potent approach for time series fire prediction.

Goal-Conditioned Reinforcement Learning With Disentanglement-Based Reachability Planning

Goal-Conditioned Reinforcement Learning (GCRL) can enable agents to spontaneously set diverse goals to learn a set of skills. Despite the excellent works proposed in various fields, reaching distant goals in temporally extended tasks remains a challenge for GCRL. Current works tackled this problem by leveraging planning algorithms to plan intermediate subgoals to augment GCRL. Their methods need two crucial requirements: (i) a state representation space to search valid subgoals, and (ii) a distance function to measure the reachability of subgoals. However, they struggle to scale to high-dimensional state space due to their non-compact representations. Moreover, they cannot collect high-quality training data through standard GC policies, which results in an inaccurate distance function. Both affect the efficiency and performance of planning and policy learning. In the paper, we propose a goal-conditioned RL algorithm combined with Disentanglement-based Reachability Planning (REPlan) to solve temporally extended tasks. In REPlan, a Disentangled Representation Module (DRM) is proposed to learn compact representations which disentangle robot poses and object positions from high-dimensional observations in a self-supervised manner. A simple REachability discrimination Module (REM) is also designed to determine the temporal distance of subgoals. Moreover, REM computes intrinsic bonuses to encourage the collection of novel states for training. We evaluate our REPlan in three vision-based simulation tasks and one real-world task. The experiments demonstrate that our REPlan significantly outperforms the prior state-of-the-art methods in solving temporally extended tasks.

Oculomotor “laziness” constrains fixation selection in real-world tasks

Oculomotor “laziness” constrains fixation selection in real-world tasks

Source-Aware Embedding Training on Heterogeneous Information Networks

ABSTRACT Heterogeneous information networks (HINs) have been extensively applied to real-world tasks, such as recommendation systems, social networks, and citation networks. While existing HIN representation learning methods can effectively learn the semantic and structural features in the network, little awareness was given to the distribution discrepancy of subgraphs within a single HIN. However, we find that ignoring such distribution discrepancy among subgraphs from multiple sources would hinder the effectiveness of graph embedding learning algorithms. This motivates us to propose SUMSHINE (Scalable Unsupervised Multi-Source Heterogeneous Information Network Embedding)—a scalable unsupervised framework to align the embedding distributions among multiple sources of an HIN. Experimental results on real-world datasets in a variety of downstream tasks validate the performance of our method over the state-of-the-art heterogeneous information network embedding algorithms.

Prototyping and implementing Robotic Process Automation in accounting firms: Benefits, challenges and opportunities to audit automation

Despite the advancements in technology, auditors still spend a significant amount of time performing repetitive and rule-based tasks. Our paper examines various audit scenarios within four accounting firms and discusses the potential for robotic process automation (RPA) to improve the efficiency and accuracy of these tasks. We propose and implement RPA-enabled solutions to make various procedures more efficient and effective. These anecdotes were based on our consultancy experiences with auditors from a group of Big 4 and mid-sized accounting firms. We present four practical business process scenarios and the prototyped RPA solutions. In each scenario, we explain our automation strategies and the duration of development. We also describe the implementation benefits and challenges, as well as analyze how we have helped the accounting firms automate their real-world audit tasks by providing the as-is and to-be Business Process Model and Notation that one can extend for future RPA development. Finally, we discuss the future challenges and opportunities regarding the broader implementation of RPA in accounting and audit tasks.

Computational approaches to Explainable Artificial Intelligence: Advances in theory, applications and trends

Deep Learning (DL), a groundbreaking branch of Machine Learning (ML), has emerged as a driving force in both theoretical and applied Artificial Intelligence (AI). DL algorithms, rooted in complex and non-linear artificial neural systems, excel at extracting high-level features from data. DL has demonstrated human-level performance in real-world tasks, including clinical diagnostics, and has unlocked solutions to previously intractable problems in virtual agent design, robotics, genomics, neuroimaging, computer vision, and industrial automation. In this paper, the most relevant advances from the last few years in Artificial Intelligence (AI) and several applications to neuroscience, neuroimaging, computer vision, and robotics are presented, reviewed and discussed. In this way, we summarize the state-of-the-art in AI methods, models and applications within a collection of works presented at the 9th International Conference on the Interplay between Natural and Artificial Computation (IWINAC). The works presented in this paper are excellent examples of new scientific discoveries made in laboratories that have successfully transitioned to real-life applications.

Facilitating Authentic Teaching with 360-Degree Videos

Visiting construction sites by civil or construction engineering students is an engaging way to advance students' understanding of construction and structural engineering design practices. Such visits offer a valuable interactive learning opportunity for students to understand real-world projects that scaffolds their in-class theoretical learning. This teaching practice intends to prepare graduate-ready students for the workforce. Facilitating immersive and interactive 360-degree videos are deemed to overcome challenges associated with implementing site visits, including logistics complexity and safety concerns. This paper demonstrated the development and implementation of authentic learning activities incorporating 360-degree videos taken at construction sites for teaching two engineering subjects delivered at La Trobe University-Australia. The construction-site types are selected to serve the intended learning outcomes of civil engineering subjects that teach the structural design methods for steel, timber and reinforced concrete members. The videos are established pedagogically alongside the projects' authentic construction documents, including structural and architectural drawings. These materials are collectively used to establish industry-driven activities for students to learn and practise real-world engineering tasks with onsite exposure enabled through 360-degree videos. Findings from quantitative and qualitative analyses examining students' perceptions of learning from 360-degree videos are reported in this paper. A 10-item survey was created to measure students' reflections on using the 360-degree videos to understand the intended skills, engagement in learning, and the development of their employability skills. From receiving 23 responses to the survey, on average, 94.8% of the participating students reflected their agreement with finding 360-degree videos supporting their understanding, engagement and employability measures, with 80% being in strong to complete agreement.

An updating-based working memory load alters the dynamics of eye movements but not their spatial extent during free viewing of natural scenes

The relationship between spatial deployments of attention and working memory load is an important topic of study, with clear implications for real-world tasks such as driving. Previous research has generally shown that attentional breadth broadens under higher load, while exploratory eye-movement behaviour also appears to change with increasing load. However, relatively little research has compared the effects of working memory load on different kinds of spatial deployment, especially in conditions that require updating of the contents of working memory rather than simple retrieval. The present study undertook such a comparison by measuring participants’ attentional breadth (via an undirected Navon task) and their exploratory eye-movement behaviour (a free-viewing recall task) under low and high updating working memory loads. While spatial aspects of task performance (attentional breadth, and peripheral extent of image exploration in the free-viewing task) were unaffected by the load manipulation, the exploratory dynamics of the free-viewing task (including fixation durations and scan-path lengths) changed under increasing load. These findings suggest that temporal dynamics, rather than the spatial extent of exploration, are the primary mechanism affected by working memory load during the spatial deployment of attention. Further, individual differences in exploratory behaviour were observed on the free-viewing task: all metrics were highly correlated across working memory load blocks. These findings suggest a need for further investigation of individual differences in eye-movement behaviour; potential factors associated with these individual differences, including working memory capacity and persistence versus flexibility orientations, are discussed.

The DeCAMFounder: nonlinear causal discovery in the presence of hidden variables

Abstract Many real-world decision-making tasks require learning causal relationships between a set of variables. Traditional causal discovery methods, however, require that all variables are observed, which is often not feasible in practical scenarios. Without additional assumptions about the unobserved variables, it is not possible to recover any causal relationships from observational data. Fortunately, in many applied settings, additional structure among the confounders can be expected. In particular, pervasive confounding is commonly encountered and has been utilised for consistent causal estimation in linear causal models. In this article, we present a provably consistent method to estimate causal relationships in the nonlinear, pervasive confounding setting. The core of our procedure relies on the ability to estimate the confounding variation through a simple spectral decomposition of the observed data matrix. We derive a DAG score function based on this insight, prove its consistency in recovering a correct ordering of the DAG, and empirically compare it to previous approaches. We demonstrate improved performance on both simulated and real datasets by explicitly accounting for both confounders and nonlinear effects.