Human Behavioral Data Research Articles

Malaria burden and residual transmission: two thirds of mosquito bites may not be preventable with current vector control tools on Bioko Island, Equatorial Guinea

ObjectivesThis study assesses exposure to malaria vector mosquitos that is nonpreventable through use of nets, the contribution of outdoor and indoor biting towards residual vector exposure, and the risk factors for being bitten and for being infected with malaria parasites on Bioko Island, Equatorial Guinea. MethodsHuman behavior and malaria infection data were collected from 13,735 randomly selected residents during cross-sectional surveys, concomitantly with entomological human landing catches, indoors and outdoors, in 20 locations on the Island. Self-reported time of going indoors, going to bed and whether using a net were analyzed to impute for each respondent the number of bites received outdoors and indoors during the night before the survey. ResultsOn average, each person received 2.7 (95% CI: 2.6-2.8) bites per night outdoors, 8.5 (8.3 to 8.7) bites indoors if not using a net, and 4.7 (4.5 to 4.8) bites indoors if using a net. Malaria infection was associated with more bites, regardless of whether received indoors or outdoors. Older age, male gender, not using a net, rural location, and going indoors later increased the risk of being bitten. The proportion of bites not averted by using a net was estimated as 66% (61 to 71). ConclusionsA large proportion of biting, mostly indoors, may not be preventable by bednets. Tools targeting indoor biting should be prioritized in Bioko. Novel vector control tools are urgently needed to reduce overall exposure to mosquito bites.

Exploration–Exploitation Mechanisms in Recurrent Neural Networks and Human Learners in Restless Bandit Problems

A key feature of animal and human decision-making is to balance the exploration of unknown options for information gain (directed exploration) versus selecting known options for immediate reward (exploitation), which is often examined using restless bandit tasks. Recurrent neural network models (RNNs) have recently gained traction in both human and systems neuroscience work on reinforcement learning, due to their ability to show meta-learning of task domains. Here we comprehensively compared the performance of a range of RNN architectures as well as human learners on restless four-armed bandit problems. The best-performing architecture (LSTM network with computation noise) exhibited human-level performance. Computational modeling of behavior first revealed that both human and RNN behavioral data contain signatures of higher-order perseveration, i.e., perseveration beyond the last trial, but this effect was more pronounced in RNNs. In contrast, human learners, but not RNNs, exhibited a positive effect of uncertainty on choice probability (directed exploration). RNN hidden unit dynamics revealed that exploratory choices were associated with a disruption of choice predictive signals during states of low state value, resembling a win-stay-loose-shift strategy, and resonating with previous single unit recording findings in monkey prefrontal cortex. Our results highlight both similarities and differences between exploration behavior as it emerges in meta-learning RNNs, and computational mechanisms identified in cognitive and systems neuroscience work.

Tutorial on Matching-based Causal Analysis of Human Behaviors Using Smartphone Sensor Data

Smartphones can unobtrusively capture human behavior and contextual data such as user interaction and mobility. Thus far, smartphone sensor data have primarily been used to gain behavioral insights through correlation analysis. This article provides a tutorial on the causal analysis of human behavior using smartphone sensor data by reviewing well-known matching methods. The key steps of the causal inference pipeline employing matching methods are illustrated using a concrete scenario involving the identification of a causal relationship between phone usage and physical activity. Several practical considerations for conducting causal inferences about human behaviors using smartphone sensor data are also discussed.

A Deep Learning Approach to Merge Rule-Based and Human-Operated Camera Control for Teleoperated Robotic Systems

Controlling a laparoscopic camera during robotic surgery represents a multifaceted challenge, demanding considerable physical and cognitive exertion from operators. While manual control presents the advantage of enabling optimal viewing angles, it is offset by its taxing nature. In contrast, current autonomous camera systems offer predictability in tool tracking but are often rigid, lacking the adaptability of human operators. This research investigates the potential of two distinct network architectures: a dense neural network (DNN) and a recurrent network (RNN), both trained using a diverse dataset comprising autonomous and human-driven camera movements. A comparative assessment of network-controlled, autonomous, and human-operated camera systems is conducted to gauge network efficacies. While the dense neural network exhibits proficiency in basic tool tracking, it grapples with inherent architectural limitations that hinder its ability to master the camera’s zoom functionality. In stark contrast, the recurrent network excels, demonstrating a capacity to sufficiently replicate the behaviors exhibited by a mixture of both autonomous and human-operated methods. In total, 96.8% of the dense network predictions had up to a one-centimeter error when compared to the test datasets, while the recurrent network achieved a 100% sub-millimeter testing error. This paper trains and evaluates neural networks on autonomous and human behavior data for camera control.

A Big Data and FRAM-Based Model for Epidemic Risk Analysis of Infectious Diseases.

The use of multi-source precursor data to predict the epidemic risk level would aid in the early and timely identification of the epidemic risk of infectious diseases. To achieve this, a new comprehensive big data fusion assessment method must be developed. With the help of the Functional Resonance Analysis Method (FRAM) model, this paper proposes a risk portrait for the whole process of a pandemic spreading. Using medical, human behaviour, internet and geo-meteorological data, a hierarchical multi-source dataset was developed with three function module tags, ie, Basic Risk Factors (BRF), the Spread of Epidemic Threats (SET) and Risk Influencing Factors (RIF). Using the dynamic functional network diagram of the risk assessment functional module, the FRAM portrait was applied to pandemic case analysis in Wuhan in 2020. This new-format FRAM portrait model offers a potential early and rapid risk assessment method that could be applied in future acute public health events.

Computational models can distinguish the contribution from different mechanisms to familiarity recognition.

Familiarity is the strange feeling of knowing that something has already been seen in our past. Over the past decades, several attempts have been made to model familiarity using artificial neural networks. Recently, two learning algorithms successfully reproduced the functioning of the perirhinal cortex, a key structure involved during familiarity: Hebbian and anti-Hebbian learning. However, performance of these learning rules is very different from one to another thus raising the question of their complementarity. In this work, we designed two distinct computational models that combined Deep Learning and a Hebbian learning rule to reproduce familiarity on natural images, the Hebbian model and the anti-Hebbian model, respectively. We compared the performance of both models during different simulations to highlight the inner functioning of both learning rules. We showed that the anti-Hebbian model fits human behavioral data whereas the Hebbian model fails to fit the data under large training set sizes. Besides, we observed that only our Hebbian model is highly sensitive to homogeneity between images. Taken together, we interpreted these results considering the distinction between absolute and relative familiarity. With our framework, we proposed a novel way to distinguish the contribution of these familiarity mechanisms to the overall feeling of familiarity. By viewing them as complementary, our two models allow us to make new testable predictions that could be of interest to shed light on the familiarity phenomenon.

Human behavior recognition based on sparse transformer with channel attention mechanism.

Human activity recognition (HAR) has recently become a popular research field in the wearable sensor technology scene. By analyzing the human behavior data, some disease risks or potential health issues can be detected, and patients' rehabilitation progress can be evaluated. With the excellent performance of Transformer in natural language processing and visual tasks, researchers have begun to focus on its application in time series. The Transformer model models long-term dependencies between sequences through self-attention mechanisms, capturing contextual information over extended periods. In this paper, we propose a hybrid model based on the channel attention mechanism and Transformer model to improve the feature representation ability of sensor-based HAR tasks. Extensive experiments were conducted on three public HAR datasets, and the results show that our network achieved accuracies of 98.10%, 97.21%, and 98.82% on the HARTH, PAMAP2, and UCI-HAR datasets, respectively, The overall performance is at the level of the most advanced methods.

VeLP: Vehicle Loading Plan Learning from Human Behavior in Nationwide Logistics System

For a nationwide logistics transportation system, it is critical to make the vehicle loading plans (i.e., given many packages, deciding vehicle types and numbers) at each sorting and distribution center. This task is currently completed by dispatchers at each center in many logistics companies and consumes a lot of workloads for dispatchers. Existing works formulate such an issue as a cargo loading problem and solve it by combinatorial optimization methods. However, it cannot work in some real-world nationwide applications due to the lack of accurate cargo volume information and effective model design under complicated impact factors as well as temporal correlation. In this paper, we explore a new opportunity to utilize large-scale route and human behavior data (i.e., dispatchers' decision process on planning vehicles) to generate vehicle loading plans (i.e., plans). Specifically, we collect a five-month nationwide operational dataset from JD Logistics in China and comprehensively analyze human behaviors. Based on the data-driven analytics insights, we design a <u>Ve</u>hicle <u>L</u>oading <u>P</u>lan learning model, named VeLP, which consists of a pattern mining module and a deep temporal cross neural network, to learn the human behaviors on regular and irregular routes, respectively. Extensive experiments demonstrate the superiority of VeLP, which achieves performance improvement by 35.8% and 50% for trunk and branch routes compared with baselines, respectively. Besides, we deployed VeLP in JDL and applied it in about 400 routes, reducing the time by approximately 20% in creating plans. It saves significant human workload and improves operational efficiency for the logistics company.

Computer-Aided Autism Spectrum Disorder Diagnosis With Behavior Signal Processing

Behavioral observation plays an essential role in the diagnosis of Autism Spectrum Disorder (ASD) by analyzing children’s atypical patterns in social activities (e.g., impaired social interaction, restricted interests, and repetitive behavior). To date, this process still heavily relies on the questionnaire survey, clinical observation, or retrospective video analysis, leading to high demand for professionals with massive labor costs. This paper proposes a standardized platform for stimulating, gathering, analyzing, modeling, and interpreting human behavioral data in the application of computer-aided ASD diagnosis. By a structured assessment process, the proposed system can automatically evaluate children’s multiple social interaction skills using the captured audio-visual data and provide the final diagnostic suggestions. We collect a multimodal behavioral database of 95 participants (71 children with ASD and 24 age-matched typical controls) in a real clinic environment, the Third Affiliated Hospital of Sun Yat-sen University, China. On the clinical database, our proposed computer-aided ASD diagnosis system obtains an accuracy of 88.42% for identifying ASD children with an average age of 24 months, representing a performance comparable to top-level human experts. As a unified and replicable solution, it has good potential to be promoted to less developed areas with limited high-quality medical resources.

A whole-task brain model of associative recognition that accounts for human behavior and neuroimaging data.

Brain models typically focus either on low-level biological detail or on qualitative behavioral effects. In contrast, we present a biologically-plausible spiking-neuron model of associative learning and recognition that accounts for both human behavior and low-level brain activity across the whole task. Based on cognitive theories and insights from machine-learning analyses of M/EEG data, the model proceeds through five processing stages: stimulus encoding, familiarity judgement, associative retrieval, decision making, and motor response. The results matched human response times and source-localized MEG data in occipital, temporal, prefrontal, and precentral brain regions; as well as a classic fMRI effect in prefrontal cortex. This required two main conceptual advances: a basal-ganglia-thalamus action-selection system that relies on brief thalamic pulses to change the functional connectivity of the cortex, and a new unsupervised learning rule that causes very strong pattern separation in the hippocampus. The resulting model shows how low-level brain activity can result in goal-directed cognitive behavior in humans.

PL-TARMI: A deep learning framework for pixel-level traffic crash risk map inference

A citywide traffic crash risk map is of great significance for preventing future traffic crashes. However, the fine-grained geographic traffic crash risk inference is still a challenging task, mainly due to the complex road network structure, human behavior and high data requirements. In this work, we propose a deep-learning framework PL-TARMI, which leverages easily accessible data to achieve accurate fine-grained traffic crash risk map inference. Specifically, we integrate the satellite image and road network image, combine with other accessible data (e.g., point of interest distribution, human mobility data, traffic data, etc.) as input, and finally obtain the pixel-level traffic crash risk map, which could provide more reasonable traffic crash prevention guidance with a lower cost. Extensive experiments on real-world datasets demonstrate the effectiveness of PL-TARMI.

Optimal policies for free recall.

There is rich structure in the order in which studied material is recalled in a free recall task (Howard & Kahana, 2002a). Extensive effort has been directed at understanding the processes and representations that give rise to this structure; however, it remains unclear why certain types of recall organization might be favored in the first place. We provide a rational analysis of the free recall task, deriving the optimal policy for recalling items under the internal representations and processes described by the context maintenance and retrieval (CMR) model of memory search (Polyn et al., 2009a). Our model, which we call rational-CMR, shows that the optimal policy for free recall is to start from the beginning of the list and then sequentially recall forwards, providing a rational account of the primacy and forward asymmetry effects typically observed in free recall. In addition, when recall is not initiated from the beginning of list, it is optimal during recall transitions to minimize the amount of forward asymmetry. Predictions from the rational model are confirmed in human behavioral data: Top-performing human participants demonstrate a stronger tendency to initiate recall from the beginning of the list and carry forward recalls, and the amount of forward asymmetry in participants depends on whether they start recall from the beginning or end of the list. We discuss the resemblance of optimal behavior in free recall to participants' behavior when applying mnemonic techniques such as the method of loci. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

ViSpa (Vision Spaces): A computer-vision-based representation system for individual images and concept prototypes, with large-scale evaluation.

Quantitative, data-driven models for mental representations have long enjoyed popularity and success in psychology (e.g., distributional semantic models in the language domain), but have largely been missing for the visual domain. To overcome this, we present ViSpa (Vision Spaces), high-dimensional vector spaces that include vision-based representation for naturalistic images as well as concept prototypes. These vectors are derived directly from visual stimuli through a deep convolutional neural network trained to classify images and allow us to compute vision-based similarity scores between any pair of images and/or concept prototypes. We successfully evaluate these similarities against human behavioral data in a series of large-scale studies, including off-line judgments-visual similarity judgments for the referents of word pairs (Study 1) and for image pairs (Study 2), and typicality judgments for images given a label (Study 3)-as well as online processing times and error rates in a discrimination (Study 4) and priming task (Study 5) with naturalistic image material. ViSpa similarities predict behavioral data across all tasks, which renders ViSpa a theoretically appealing model for vision-based representations and a valuable research tool for data analysis and the construction of experimental material: ViSpa allows for precise control over experimental material consisting of images and/or words denoting imageable concepts and introduces a specifically vision-based similarity for word pairs. To make ViSpa available to a wide audience, this article (a) includes (video) tutorials on how to use ViSpa in R and (b) presents a user-friendly web interface at http://vispa.fritzguenther.de. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Internet of Behaviors: A Survey

The Internet of Behavior is a research theme that aims to analyze human behavior data on the Internet from the perspective of behavioral psychology, obtain insights about human behavior, and better understand the intention behind the behavior. In this way, the Internet of Behavior can predict human behavioral trends in the future and even change human behavior, which can provide more convenience for human life. With the increasing prosperity of the Internet of Things, more and more behavior-related data is collected on the Internet by connected devices such as sensors. People and behavior are connected through the extension of the Internet of Things – the Internet of Behavior. At present, the Internet of Behavior has gradually been applied to our lives, but it is still in its early stages, and many opportunities and challenges are emerging. This article provides an in-depth overview of the fundamental aspects of the Internet of Behavior: (1) We introduce the development process and research status of the Internet of Behavior from the perspective of the Internet of Things. (2) We propose the characteristics of the Internet of Behavior and define its development direction in terms of three aspects: real-time, autonomy, and reliability. (3) We provide a comprehensive summary of the current applications of the Internet of Behavior, including specific discussions in five scenarios that give an overview of the application status of the Internet of Behavior. (4) We discuss the challenges of the Internet of Behavior’s development and its future directions, which hopefully will bring some progress to the Internet of Behavior. To the best of our knowledge, this is the first survey paper on the Internet of Behavior. We hope that this in-depth review can provide some useful directions for more productive research in related fields.

A deep learning factor analysis model based on importance‐weighted variational inference and normalizing flow priors: Evaluation within a set of multidimensional performance assessments in youth elite soccer players

AbstractExploratory factor analysis is a widely used framework in the social and behavioral sciences. Since measurement errors are always present in human behavior data, latent factors, generating the observed data, are important to identify. While most factor analysis methods rely on linear relationships in the data‐generating process, deep learning models can provide more flexible modeling approaches. However, two problems need to be addressed. First, for interpretation, scaling assumptions are required, which can be (at least) cumbersome for deep generative models. Second, deep generative models are typically not identifiable, which is required in order to identify the underlying latent constructs. We developed a model that uses a variational autoencoder as an estimator for a complex factor analysis model based on importance‐weighted variational inference. In order to receive interpretable results and an identified model, we use a linear factor model with identification constraints in the measurement model. To maintain the flexibility of the model, we use normalizing flow latent priors. Within the evaluation of performance measures in a talent development program in soccer, we found more clarity in the separation of the identified underlying latent dimensions with our models compared to traditional PCA analyses.

Visual working memory in immersive visualization: a change detection experiment and an image-computable model

Visual working memory (VWM) is a cognitive mechanism essential for interacting with the environment and accomplishing ongoing tasks, as it allows fast processing of visual inputs at the expense of the amount of information that can be stored. A better understanding of its functioning would be beneficial to research fields such as simulation and training in immersive Virtual Reality or information visualization and computer graphics. The current work focuses on the design and implementation of a paradigm for evaluating VWM in immersive visualization and of a novel image-based computational model for mimicking the human behavioral data of VWM. We evaluated the VWM at the variation of four conditions: set size, spatial layout, visual angle (VA) subtending stimuli presentation space, and observation time. We adopted a full factorial design and analysed participants’ performances in the change detection experiment. The analysis of hit rates and false alarm rates confirms the existence of a limit of VWM capacity of around 7 ± 2 items, as found in the literature based on the use of 2D videos and images. Only VA and observation time influence performances (p<0.0001). Indeed, with VA enlargement, participants need more time to have a complete overview of the presented stimuli. Moreover, we show that our model has a high level of agreement with the human data, r>0.88 (p<0.05).

Sparse Maximum Margin Learning from Multimodal Human Behavioral Patterns

We propose a multimodal data fusion framework to systematically analyze human behavioral data from specialized domains that are inherently dynamic, sparse, and heterogeneous. We develop a two-tier architecture of probabilistic mixtures, where the lower tier leverages parametric distributions from the exponential family to extract significant behavioral patterns from each data modality. These patterns are then organized into a dynamic latent state space at the higher tier to fuse patterns from different modalities. In addition, our framework jointly performs pattern discovery and maximum-margin learning for downstream classification tasks by using a group-wise sparse prior that regularizes the coefficients of the maximum-margin classifier. Therefore, the discovered patterns are highly interpretable and discriminative to support downstream classification tasks. Experiments on real-world behavioral data from medical and psychological domains demonstrate that our framework discovers meaningful multimodal behavioral patterns with improved interpretability and prediction performance.

Modeling Human Trust and Reliance in AI-Assisted Decision Making: A Markovian Approach

The increased integration of artificial intelligence (AI) technologies in human workflows has resulted in a new paradigm of AI-assisted decision making, in which an AI model provides decision recommendations while humans make the final decisions. To best support humans in decision making, it is critical to obtain a quantitative understanding of how humans interact with and rely on AI. Previous studies often model humans' reliance on AI as an analytical process, i.e., reliance decisions are made based on cost-benefit analysis. However, theoretical models in psychology suggest that the reliance decisions can often be driven by emotions like humans' trust in AI models. In this paper, we propose a hidden Markov model to capture the affective process underlying the human-AI interaction in AI-assisted decision making, by characterizing how decision makers adjust their trust in AI over time and make reliance decisions based on their trust. Evaluations on real human behavior data collected from human-subject experiments show that the proposed model outperforms various baselines in accurately predicting humans' reliance behavior in AI-assisted decision making. Based on the proposed model, we further provide insights into how humans' trust and reliance dynamics in AI-assisted decision making is influenced by contextual factors like decision stakes and their interaction experiences.

Behavioural Models of Risk-Taking in Human-Robot Tactile Interactions.

Touch can have a strong effect on interactions between people, and as such, it is expected to be important to the interactions people have with robots. In an earlier work, we showed that the intensity of tactile interaction with a robot can change how much people are willing to take risks. This study further develops our understanding of the relationship between human risk-taking behaviour, the physiological responses by the user, and the intensity of the tactile interaction with a social robot. We used data collected with physiological sensors during the playing of a risk-taking game (the Balloon Analogue Risk Task, or BART). The results of a mixed-effects model were used as a baseline to predict risk-taking propensity from physiological measures, and these results were further improved through the use of two machine learning techniques-support vector regression (SVR) and multi-input convolutional multihead attention (MCMA)-to achieve low-latency risk-taking behaviour prediction during human-robot tactile interaction. The performance of the models was evaluated based on mean absolute error (MAE), root mean squared error (RMSE), and R squared score (R2), which obtained the optimal result with MCMA yielding an MAE of 3.17, an RMSE of 4.38, and an R2 of 0.93 compared with the baseline of 10.97 MAE, 14.73 RMSE, and 0.30 R2. The results of this study offer new insights into the interplay between physiological data and the intensity of risk-taking behaviour in predicting human risk-taking behaviour during human-robot tactile interactions. This work illustrates that physiological activation and the intensity of tactile interaction play a prominent role in risk processing during human-robot tactile interaction and demonstrates that it is feasible to use human physiological data and behavioural data to predict risk-taking behaviour in human-robot tactile interaction.

Student close contact behavior and COVID-19 transmission in China's classrooms.

Classrooms are high-risk indoor environments, so analysis of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) transmission in classrooms is important for determining optimal interventions. Due to the absence of human behavior data, it is challenging to accurately determine virus exposure in classrooms. A wearable device for close contact behavior detection was developed, and we recorded >250,000 data points of close contact behaviors of students from grades 1 to 12. Combined with a survey on students' behaviors, we analyzed virus transmission in classrooms. Close contact rates for students were 37 ± 11% during classes and 48 ± 13% during breaks. Students in lower grades had higher close contact rates and virus transmission potential. The long-range airborne transmission route is dominant, accounting for 90 ± 3.6% and 75 ± 7.7% with and without mask wearing, respectively. During breaks, the short-range airborne route became more important, contributing 48 ± 3.1% in grades 1 to 9 (without wearing masks). Ventilation alone cannot always meet the demands of COVID-19 control; 30 m3/h/person is suggested as the threshold outdoor air ventilation rate in a classroom. This study provides scientific support for COVID-19 prevention and control in classrooms, and our proposed human behavior detection and analysis methods offer a powerful tool to understand virus transmission characteristics and can be employed in various indoor environments.