Real-world Tasks Research Articles

Measuring the building blocks of everyday cognition: executive functions and relational reasoning.

Measurement of the building blocks of everyday thought must capture the range of different ways that humans may train, develop, and use their cognitive resources in real world tasks. Executive function as a construct has been enthusiastically adopted by cognitive and education sciences due to its theorized role as an underpinning of, and constraint on, humans' accomplishment of complex cognitively demanding tasks in the world, such as identifying problems, reasoning about and executing multi-step solutions while inhibiting prepotent responses or competing desires. As EF measures have been continually refined for increased precision; however, they have also become increasingly dissociated from those everyday accomplishments. We posit three implications of this insight: (1) extant measures of EFs that reduce context actually add an implicit requirement that children reason using abstract rules that are not accomplishing a function in the world, meaning that EF scores may in part reflect experience with formal schooling and Western, Educated, Industrialized, Rich, Democratic (WEIRD) socialization norms, limiting their ability to predict success in everyday life across contexts, (2) measurement of relational attention and relational reasoning have not received adequate consideration in this context but are highly aligned with the key aims for measuring EFs, and may be more aligned with humans' everyday cognitive practices, but (3) relational attention and reasoning should be considered alongside rather than as an additional EF as has been suggested, for measurement clarity.

Model-based variable impedance learning control for robotic manipulation

The capability to adapt compliance by varying muscle stiffness is crucial for dexterous manipulation skills in humans. Incorporating compliance in robot motor control is crucial for enabling real-world force interaction tasks with human-like dexterity. In this study, we introduce a novel approach, we call “deep Model Predictive Variable Impedance Controller (MPVIC)” for compliant robotic manipulation, which combines Variable Impedance Control with Model Predictive Control (MPC). The method involves learning a generalized Cartesian impedance model of a robot manipulator through an exploration strategy to maximize information gain. Within the MPC framework, this learned model is utilized to adapt the impedance parameters of a low-level variable impedance controller, thereby achieving the desired compliance behavior for various manipulation tasks without requiring retraining or finetuning. We assess the efficacy of the proposed deep MPVIC approach using a Franka Emika Panda robotic manipulator in simulations and real-world experiments involving diverse manipulation tasks. Comparative evaluations against model-free and model-based reinforcement learning approaches in variable impedance control are conducted, considering aspects such as transferability between tasks and performance. The results demonstrate the effectiveness and potential of the presented approach for advancing robotic manipulation capabilities.

Study of Robot Manipulator Control via Remote Method

INTRODUCTION: The study introduces a novel approach to the design and management of industrial robots using virtual reality technology, enabling humans to observe a wide range of robot behaviors across various environments.OBJECTIVES: Through a simulation program, the robot's movements can be reviewed, and a program for real-world task execution can be generated. Furthermore, the research delves into the algorithm governing the interaction between the industrial robot and humans.METHODS: The robot utilized in this research project has been meticulously refurbished and enhanced from the previously old version robotic manipulator, which lacked an electrical cabinet derived.RESULTS: Following the mechanical and electrical upgrades, a virtual setup, incorporating a headset and two hand controllers, has been integrated into the robot's control system, enabling control via this device.CONCLUSION: This control algorithm leverages a shared control approach and artificial potential field methods to facilitate obstacle avoidance through repulsive and attractive forces. Ultimately, the study presents experimental results using the real robot model.

CLERA: A Unified Model for Joint Cognitive Load and Eye Region Analysis in the Wild

Non-intrusive, real-time analysis of the dynamics of the eye region allows us to monitor humans’ visual attention allocation and estimate their mental state during the performance of real-world tasks, which can potentially benefit a wide range of human-computer interaction (HCI) applications. While commercial eye-tracking devices have been frequently employed, the difficulty of customizing these devices places unnecessary constraints on the exploration of more efficient, end-to-end models of eye dynamics. In this work, we propose CLERA, a unified model for Cognitive Load and Eye Region Analysis, which achieves precise keypoint detection and spatiotemporal tracking in a joint-learning framework. Our method demonstrates significant efficiency and outperforms prior work on tasks including cognitive load estimation, eye landmark detection, and blink estimation. We also introduce a large-scale dataset of 30 k human faces with joint pupil, eye-openness, and landmark annotation, which aims at supporting future HCI research on human factors and eye-related analysis.

NeuralBO: A black-box optimization algorithm using deep neural networks

Bayesian Optimization (BO) is an effective approach for the global optimization of black-box functions when function evaluations are expensive. Most prior works use Gaussian processes to model the black-box function, however, the use of kernels in Gaussian processes leads to two problems: first, the kernel-based methods scale poorly with the number of data points and second, kernel methods are usually not effective on complex structured high dimensional data due to curse of dimensionality. Therefore, we propose a novel black-box optimization algorithm where the black-box function is modeled using a neural network. Our algorithm does not need a Bayesian neural network to estimate predictive uncertainty and is therefore computationally favorable. We analyze the theoretical behavior of our algorithm in terms of regret bound using advances in NTK theory showing its efficient convergence. We perform experiments with both synthetic and real-world optimization tasks and show that our algorithm is more sample efficient compared to existing methods.

Chaotic behavior learning via information tracking

Chaotic time series are often encountered in real-world applications, where modeling and understanding such time series present a significant challenge. Existing simulation-based methods for characterizing chaotic behavior may be sensitive to the respective model settings, while data-driven methods cannot adapt well to irregularities and aperiodicity in chaotic time series during the prediction stage, leading to nonlinearly aggregated forecasting errors after a finite number of time steps. In our preliminary work (Ren et al., 2023), we introduced a data-driven method that incorporates the idea of information tracking to capture and adapt to the chaotic changes over time, and tested its effectiveness on a real-world task—decadal temperature prediction. While the practical usefulness of the method has been initially validated on one domain-specific application scenario, a set of open questions is yet to be answered: Is the concept of information tracking generalizable to model and solve a broad spectrum of learning tasks that involve chaotic behavior? If yes, what are the underlying principles and learning behaviors that make the general method work well? Moreover, how can we assess the efficacy of the method in different scenarios? This study constitutes a significant step forward by systematically investigating and addressing the questions mentioned above. Specifically, we first provide a rigorous definition of the formalism and computational mechanism of the general method, with all necessary variables, symbols, and formulations. More importantly, we theoretically characterize and understand the generalizability of the method by mathematically uncovering the fundamental principles and behavior of information tracking-based chaotic behavioral learning, as prescribed by two chaotic behavioral indexes, namely, the Lyapunov and the Hurst exponents. Further, we comprehensively demonstrate the generalizability and robustness of the method by empirically examining a variety of chaotic behavioral learning problems generated by three representative chaotic systems: The logistic map, the double pendulum, and the Lorenz system.

Anthropogenic Object Localization: Evaluation of Broad-Area High-Resolution Imagery Scans Using Deep Learning in Overhead Imagery.

Too often, the testing and evaluation of object detection, as well as the classification techniques for high-resolution remote sensing imagery, are confined to clean, discretely partitioned datasets, i.e., the closed-world model. In recent years, the performance on a number of benchmark datasets has exceeded 99% when evaluated using cross-validation techniques. However, real-world remote sensing data are truly big data, which often exceed billions of pixels. Therefore, one of the greatest challenges regarding the evaluation of machine learning models taken out of the clean laboratory setting and into the real world is the difficulty of measuring performance. It is necessary to evaluate these models on a grander scale, namely, tens of thousands of square kilometers, where it is intractable to the ground truth and the ever-changing anthropogenic surface of Earth. The ultimate goal of computer vision model development for automated analysis and broad area search and discovery is to augment and assist humans, specifically human-machine teaming for real-world tasks. In this research, various models have been trained using object classes from benchmark datasets such as UC Merced, PatternNet, RESISC-45, and MDSv2. We detail techniques to scan broad swaths of the Earth with deep convolutional neural networks. We present algorithms for localizing object detection results, as well as a methodology for the evaluation of the results of broad-area scans. Our research explores the challenges of transitioning these models out of the training-validation laboratory setting and into the real-world application domain. We show a scalable approach to leverage state-of-the-art deep convolutional neural networks for the search, detection, and annotation of objects within large swaths of imagery, with the ultimate goal of providing a methodology for evaluating object detection machine learning models in real-world scenarios.

Corporate Mole: A Grouping Strategy for Minimizing the Free-Rider Problem in a Managerial Finance Course

Group projects in upper level finance courses are becoming common, both for the reflection of real-world group tasks in business firms and for their appropriateness in dealing with certain types of assignments in the classroom and beyond. Professors often lament over effective formation of groups and the task of managing group dynamics, including the problem of free-riders: those students who, knowing that the group gets the grade, will shirk their respective responsibilities in accomplishing group tasks. The following article represents one evolution of grouping methods over two decades and the observations surrounding each. The most successful, in this author’s experience, is a recent marriage of a so-called “reality TV” recipe with structured groupings

A novel phishing website classification method based on hybrid sampling

ABSTRACT In real-world anomaly detection tasks such as Credit Card Fraud Detection, Cancer Patients Detection, Phishing Website Detection, etc., the training datasets often suffer from skewed class distribution. But the traditional Machine Learning (ML) classification algorithms assume balanced class distribution and equal misclassification costs. As a result, when class-imbalanced data are presented to the traditional ML algorithms they tend to produce biased and inaccurate predictive ML models. In this study, we propose four novel Phishing Website Classification models namely, SMOTEENN-XGB, SMOTEENN-RF, SMOTEENN-LR, and SMOTEENN-SVM by combining SMOTEENN (SMOTE + ENN) hybrid sampling technique with eXtreme Gradient Boosting (XGB), Random Forest (RF), Logistic Regression (LR), and Support Vector Machine (SVM) classifiers respectively. We propose the use of SMOTEENN hybrid sampling as the novel approach to address the problem of class imbalance in Phishing Website datasets prior to building classification models. To the best of our knowledge and belief, our novel proposed four models SMOTEENN-XGB, SMOTEENN-RF, SMOTEEEN-LR, and SMOTEENN-SVM for Phishing Website Detection based on SMOTEENN hybrid sampling approach have not been published in the existing studies as of now.

A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification

Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications. Recent advances further enable adapting foundation models in downstream tasks efficiently using only a few training samples, e.g., in-context learning. Yet, the application of such learning paradigms in medical image analysis remains scarce due to the shortage of publicly accessible data and benchmarks. In this paper, we aim at approaches adapting the foundation models for medical image classification and present a novel dataset and benchmark for the evaluation, i.e., examining the overall performance of accommodating the large-scale foundation models downstream on a set of diverse real-world clinical tasks. We collect five sets of medical imaging data from multiple institutes targeting a variety of real-world clinical tasks (22,349 images in total), i.e., thoracic diseases screening in X-rays, pathological lesion tissue screening, lesion detection in endoscopy images, neonatal jaundice evaluation, and diabetic retinopathy grading. Results of multiple baseline methods are demonstrated using the proposed dataset from both accuracy and cost-effective perspectives.

Concept-Level Model Interpretation From the Causal Aspect

With the increasing growth of data and the ability of learning with them, machine learning models are adopted in various domains. However, few of machine learning models are able to reason their prediction, which limits their further applications in real-world tasks. With the potential to address this dilemma, model interpretation has become an important research topic because of the ability to provide the underlying reasons for model predictions at the feature level or concept level. Model interpretation at the concept level focuses on exploring the roles of concepts in model prediction, which enables more compact and understandable interpretations. Concept-level model interpretation requires the identification of the concepts that contribute to model prediction and the exploration of the rules underneath these concepts. To achieve the two objectives, we propose a Concept-level Model Interpretation framework (CMIC) from the perspective of causality. CMIC can automatically detect concepts in data and discover the causal relation between the detected concepts and the model's predicted labels. Furthermore, CMIC ranks the contributions of concepts by their causal effect on the model prediction, reflecting the detected concepts' importance. We evaluate the proposed CMIC framework on both synthetic and real-world datasets to demonstrate the quality of the provided interpretation.

Multifingered Robot Hand Compliant Manipulation Based on Vision-Based Demonstration and Adaptive Force Control.

Multifingered hand dexterous manipulation is quite challenging in the domain of robotics. One remaining issue is how to achieve compliant behaviors. In this work, we propose a human-in-the-loop learning-control approach for acquiring compliant grasping and manipulation skills of a multifinger robot hand. This approach takes the depth image of the human hand as input and generates the desired force commands for the robot. The markerless vision-based teleoperation system is used for the task demonstration, and an end-to-end neural network model (i.e., TeachNet) is trained to map the pose of the human hand to the joint angles of the robot hand in real-time. To endow the robot hand with compliant human-like behaviors, an adaptive force control strategy is designed to predict the desired force control commands based on the pose difference between the robot hand and the human hand during the demonstration. The force controller is derived from a computational model of the biomimetic control strategy in human motor learning, which allows adapting the control variables (impedance and feedforward force) online during the execution of the reference joint angles. The simultaneous adaptation of the impedance and feedforward profiles enables the robot to interact with the environment compliantly. Our approach has been verified in both simulation and real-world task scenarios based on a multifingered robot hand, that is, the Shadow Hand, and has shown more reliable performances than the current widely used position control mode for obtaining compliant grasping and manipulation behaviors.

Versatile multicontact planning and control for legged loco-manipulation.

Loco-manipulation planning skills are pivotal for expanding the utility of robots in everyday environments. These skills can be assessed on the basis of a system's ability to coordinate complex holistic movements and multiple contact interactions when solving different tasks. However, existing approaches have been merely able to shape such behaviors with hand-crafted state machines, densely engineered rewards, or prerecorded expert demonstrations. Here, we propose a minimally guided framework that automatically discovers whole-body trajectories jointly with contact schedules for solving general loco-manipulation tasks in premodeled environments. The key insight is that multimodal problems of this nature can be formulated and treated within the context of integrated task and motion planning (TAMP). An effective bilevel search strategy was achieved by incorporating domain-specific rules and adequately combining the strengths of different planning techniques: trajectory optimization and informed graph search coupled with sampling-based planning. We showcase emergent behaviors for a quadrupedal mobile manipulator exploiting both prehensile and nonprehensile interactions to perform real-world tasks such as opening/closing heavy dishwashers and traversing spring-loaded doors. These behaviors were also deployed on the real system using a two-layer whole-body tracking controller.

Improving the matching of deformable objects by learning to detect keypoints

We propose a novel learned keypoint detection method to increase the number of correct matches for the task of non-rigid image correspondence. By leveraging true correspondences acquired by matching annotated image pairs with a specified descriptor extractor, we train an end-to-end convolutional neural network (CNN) to find keypoint locations that are more appropriate to the considered descriptor. Experiments demonstrate that our method enhances the Mean Matching Accuracy of numerous descriptors when used in conjunction with our detection method, while outperforming the state-of-the-art keypoint detectors on real images of non-rigid objects by 20 p.p. We also apply our method on the complex real-world task of object retrieval where our detector performs on par with the finest keypoint detectors currently available for this task. The source code and trained models are publicly available at https://github.com/verlab/LearningToDetect_PRL_2023.

Astrometric calibration and source characterisation of the latest generation neuromorphic event-based cameras for space imaging

As an emerging approach to space situational awareness and space imaging, the practical use of an event-based camera (EBC) in space imaging for precise source analysis is still in its infancy. The nature of event-based space imaging and data collection needs to be further explored to develop more effective event-based space imaging systems and advance the capabilities of event-based tracking systems with improved target measurement models. Moreover, for event measurements to be meaningful, a framework must be investigated for EBC calibration to project events from pixel array coordinates in the image plane to coordinates in a target resident space object’s reference frame. In this paper, the traditional techniques of conventional astronomy are reconsidered to properly utilise the EBC for space imaging and space situational awareness. This paper presents the techniques and systems used for calibrating an EBC for reliable and accurate measurement acquisition. These techniques are vital in building event-based space imaging systems capable of real-world space situational awareness tasks. By calibrating sources detected using the EBC, the spatiotemporal characteristics of detected sources or “event sources” can be related to the photometric characteristics of the underlying astrophysical objects. Finally, these characteristics are analysed to establish a foundation for principled processing and observing techniques which appropriately exploit the capabilities of the EBC.

Release of cognitive and multimodal MRI data including real-world tasks and hippocampal subfield segmentations

We share data from N = 217 healthy adults (mean age 29 years, range 20–41; 109 females, 108 males) who underwent extensive cognitive assessment and neuroimaging to examine the neural basis of individual differences, with a particular focus on a brain structure called the hippocampus. Cognitive data were collected using a wide array of questionnaires, naturalistic tests that examined imagination, autobiographical memory recall and spatial navigation, traditional laboratory-based tests such as recalling word pairs, and comprehensive characterisation of the strategies used to perform the cognitive tests. 3 Tesla MRI data were also acquired and include multi-parameter mapping to examine tissue microstructure, diffusion-weighted MRI, T2-weighted high-resolution partial volume structural MRI scans (with the masks of hippocampal subfields manually segmented from these scans), whole brain resting state functional MRI scans and partial volume high resolution resting state functional MRI scans. This rich dataset will be of value to cognitive and clinical neuroscientists researching individual differences, real-world cognition, brain-behaviour associations, hippocampal subfields and more. All data are freely available on Dryad.

Time Series Forest Fire Prediction Based on Improved Transformer

Forest fires, severe natural disasters causing substantial damage, necessitate accurate predictive modeling to guide preventative measures effectively. This study introduces an enhanced window-based Transformer time series forecasting model aimed at improving the precision of forest fire predictions. Leveraging time series data from 2020 to 2021 in Chongli, a myriad of forest fire influencing factors were ascertained using remote sensing satellite and GIS technologies, with their interrelationships estimated through a multicollinearity test. Given the intricate nature of real-world forest fire prediction tasks, we propose a novel window-based Transformer architecture complemented by a dual time series input strategy premised on 13 influential factors. Subsequently, time series data were incorporated into the model to generate a forest fire risk prediction map in Chongli District. The model’s effectiveness was then evaluated using various metrics, including accuracy (ACC), root mean square error (RMSE), and mean absolute error (MAE), and compared with traditional deep learning methods. Our model demonstrated superior predictive performance (ACC = 91.56%, RMSE = 0.37, MAE = 0.05), harnessing spatial background information efficiently and effectively utilizing the periodicity of forest fire factors. Consequently, the study proves this method to be a novel and potent approach for time series fire prediction.

Goal-Conditioned Reinforcement Learning With Disentanglement-Based Reachability Planning

Goal-Conditioned Reinforcement Learning (GCRL) can enable agents to spontaneously set diverse goals to learn a set of skills. Despite the excellent works proposed in various fields, reaching distant goals in temporally extended tasks remains a challenge for GCRL. Current works tackled this problem by leveraging planning algorithms to plan intermediate subgoals to augment GCRL. Their methods need two crucial requirements: (i) a state representation space to search valid subgoals, and (ii) a distance function to measure the reachability of subgoals. However, they struggle to scale to high-dimensional state space due to their non-compact representations. Moreover, they cannot collect high-quality training data through standard GC policies, which results in an inaccurate distance function. Both affect the efficiency and performance of planning and policy learning. In the paper, we propose a goal-conditioned RL algorithm combined with Disentanglement-based Reachability Planning (REPlan) to solve temporally extended tasks. In REPlan, a Disentangled Representation Module (DRM) is proposed to learn compact representations which disentangle robot poses and object positions from high-dimensional observations in a self-supervised manner. A simple REachability discrimination Module (REM) is also designed to determine the temporal distance of subgoals. Moreover, REM computes intrinsic bonuses to encourage the collection of novel states for training. We evaluate our REPlan in three vision-based simulation tasks and one real-world task. The experiments demonstrate that our REPlan significantly outperforms the prior state-of-the-art methods in solving temporally extended tasks.

Oculomotor “laziness” constrains fixation selection in real-world tasks

Oculomotor “laziness” constrains fixation selection in real-world tasks

Source-Aware Embedding Training on Heterogeneous Information Networks

ABSTRACT Heterogeneous information networks (HINs) have been extensively applied to real-world tasks, such as recommendation systems, social networks, and citation networks. While existing HIN representation learning methods can effectively learn the semantic and structural features in the network, little awareness was given to the distribution discrepancy of subgraphs within a single HIN. However, we find that ignoring such distribution discrepancy among subgraphs from multiple sources would hinder the effectiveness of graph embedding learning algorithms. This motivates us to propose SUMSHINE (Scalable Unsupervised Multi-Source Heterogeneous Information Network Embedding)—a scalable unsupervised framework to align the embedding distributions among multiple sources of an HIN. Experimental results on real-world datasets in a variety of downstream tasks validate the performance of our method over the state-of-the-art heterogeneous information network embedding algorithms.