Human Performance Measures Research Articles

Human-Integrated System Simulation Analysis “in the Wild”

Systems engineers often use model-based simulations to evaluate design concepts before building physical prototypes and running live experiments. However, their simulations typically lack infrastructure representing human behavior (e.g., cognitive tasks) and performance measures (e.g., workload). While human factors practitioners have developed simulation approaches that include such infrastructure, they typically employ languages, tools, and techniques that engineering teams cannot easily adopt given real-world budget and schedule constraints. To address this challenge, we propose a novel, lightweight approach to human-integrated system simulation analysis. Our approach enables the analyst to take a preexisting system model, add minimal human-model infrastructure, and run whole-system simulations that produce operator-workload and task-duration estimates, all using a common engineering language and tool, SysML and Cameo Systems Modeler, and any preexisting behavioral modeling technique. We demonstrate the approach using a simple, preexisting model of a fictitious space-telescope system.

Human Resource Assessment and Sustainability : Investigating the Measurability of Human Activity

Sustainability, as a complex set of goals, has many possible interpretations and approaches the 17 Sustainable Development Goals (SDGs) set out in the document have 169 sub-goals, of which Goal 10 is "reduced inequalities". There are 10 sub-goals linked to this goal, the first of which is aimed at reducing income inequality. Employers have direct responsibility and considerable leeway to develop fair wage systems. For HR professionals, the practical implementation of this is a major task, and their knowledge can therefore be a key factor in achieving SDG 10. In our research, we sought to answer the question of what future HR professionals know about the accounting measurability of human performance and how they assess its monetary valuation. In our view, HR professionals who are aware of the issues of human performance measurement and recognize the fundamental problems can play a significant role in meeting SDG 10. In our primary research, we surveyed 448 undergraduate human resources students from early December 2022 to the end of January 2023 using an online questionnaire. The questionnaire contains closed and open questions in addition to demographic data. Our research shows that the value of human activity is not well expressed in monetary terms, which implies that wages and the value of work are not closely related.

Propagating cortical waves coordinate sensory encoding and memory retrieval in the human brain.

Complex behavior entails a balance between taking in sensory information from the environment and utilizing previously learned internal information. Experiments in behaving mice have demonstrated that the brain continually alternates between outward and inward modes of cognition, switching its mode of operation every few seconds. Further, each state transition is marked by a stereotyped cascade of neuronal spiking that pervades most forebrain structures. Here we analyzed large fMRI datasets to demonstrate that a similar switching mechanism governs the operation of the human brain. We found that human brain activity was punctuated every several seconds by coherent, propagating waves emerging in the exteroceptive sensorimotor regions and terminating in the interoceptive default mode network. As in the mouse, the issuance of such events coincided with fluctuations in pupil size, indicating a tight relationship with arousal fluctuations, and this phenomenon occurred across behavioral states. Strikingly, concurrent measurement of human performance in a visual memory task indicated that each cycle of propagating fMRI waves sequentially promoted the encoding of semantic information and self-directed retrieval of memories. Together, these findings indicate that human cognitive performance is governed by autonomous switching between exteroceptive and interoceptive states. This apparently conserved feature of mammalian brain physiology bears directly on the integration of sensory and mnemonic information during everyday behavior.

Human Factors at Play: Understanding the Impact of Conditioning on Presence and Reaction Time in Mixed Reality.

A prerequisite to improving the presence of a user in mixed reality (MR) is the ability to measure and quantify presence. Traditionally, subjective questionnaires have been used to assess the level of presence. However, recent studies have shown that presence is correlated with objective and systemic human performance measures such as reaction time. These studies analyze the correlation between presence and reaction time when technical factors such as object realism and plausibility of the object's behavior change. However, additional psychological and physiological human factors can also impact presence. It is unclear if presence can be mapped to and correlated with reaction time when human factors such as conditioning are involved. To answer this question, we conducted an exploratory study ($N=60$) where the relationship between presence and reaction time was assessed under three different conditioning scenarios: control, positive, and negative. We demonstrated that human factors impact presence. We found that presence scores and reaction times are significantly correlated (correlation coefficient of -0.64), suggesting that the impact of human factors on reaction time correlates with its effect on presence. In demonstrating that, our study takes another important step toward using objective and systemic measures like reaction time as a presence measure.

Human performance measures for cyber security operations centers

Human performance metrics and measures are important to Security Operations Centers (SOCs). Metrics are used by SOC managers to assess analysts’ performance. However, existing metrics for analysts tend to focus solely on task performance. Researchers and practitioners may value ideas on other dimensions by which the performance of an analyst can be measured. As studies on performance metrics for analysts continue to evolve, this paper provides timely insights and recommendations for researchers and practitioners on different dimensions by which the performance of an analyst can be measured. This paper identifies and presents various dimensions by which analysts’ performance could be measured. Beyond the identification of the different dimensions, this paper provides examples of measurable constructs under each suggested dimension in the context of a SOC.

Repurposing a Digital Twin-Based Decision Support System as a Training Tool by Incorporating Learning Science Principles

The U.S. Navy has a sophisticated manpower planning process for recruiting, selecting, training, and placing thousands of enlisted Sailors into hundreds of specific jobs every year. Given the large number of “moving parts” in this manpower planning process, some inefficiencies are unavoidable. For example, there may be a non-trivial time gap between when some Sailors complete their job-specific training course and when they arrive at their duty station. During this time, their knowledge and skills will invariably decay due to lack of use. Electronic job aids, such as Interactive Electronic Technical Manuals (IETMs) and Decision Support Systems (DSS), are important to mission success because they help to reduce the need for Sailors to memorize rarely used factual details. However, IETMs and DSSs are job aids, not training tools. In this paper, we describe how we repurposed an existing digital twin-based DSS to create a ubiquitous, online training tool. We did so by leveraging the learning science principles of human performance measurement, adaptive learning with scaffolding, performance feedback, and training data analytics. We conclude with a series of best practices and lessons learned that other practitioners could follow when attempting to repurpose electronic job aids that were not originally designed for training.

Developing AI enabled sensors and decision support for military operators in the field.

Wearable sensors enable down range data collection of physiological and cognitive performance of the warfighter. However, autonomous teams may find the sensor data impractical to interpret and hence influence real-time decisions without the support of subject matter experts. Decision support tools can reduce the burden of interpreting physiological data in the field and incorporate a systems perspective where noisy field data can contain useful additional signals. We present a methodology of how artificial intelligence can be used for modeling human performance with decision-making to achieve actionable decision support. We provide a framework for systems design and advancing from the laboratory to real world environments. The result is a validated measure of down-range human performance with a low burden of operation.

Towards objective human performance measurement for maritime safety: A new psychophysiological data-driven machine learning method

Towards objective human performance measurement for maritime safety: A new psychophysiological data-driven machine learning method

Design and experimentation of remote driving system for robotic speed sprayer operating in orchard environment

AbstractThe automation of agricultural machines is an irreversible trend considering the demand for improved productivity and lack of labor in handling agricultural tasks. Unstructured working environments and weather often inhibit a seemingly simple task from being fully autonomously performed. In this context, we propose a remote driving system (RDS) to aid agricultural machines designed to operate autonomously. Particularly, we modify a commercial speed sprayer for orchard environments into a robotic speed sprayer to evaluate the proposed RDS's usability and test three sensor configurations in terms of human performance. Furthermore, we propose a confidence error ellipse‐based task performance measure to evaluate human performance. In addition, we present field experimental results describing how the sensor configurations affect human performance. We find that a combination of a semiautonomous line tracking device and a wide‐angle camera is the most effective for spraying. Finally, we discuss how to improve the proposed RDS in terms of usability and obtain a more accurate measure of human performance.

原子炉運転員のパフォーマンス測定

人間信頼性分析において，ヒューマンエラー率を確実に予測することは難しく，運転訓練における運転員パフォーマンス評価を基にヒューマンエラー率を推定する取り組みが検討されている。ここでは，運転訓練における運転員パフォーマンスに係る行動データを自動で取集し，運転員の行動をパターン化した自動評価ロジックを適用することで，運転員のパフォーマンス，ヒューマンエラー率を自動推定可能な評価システムについて紹介する。

Pelatihan: Evaluasi dan Rekonstruksi Kinerja ASN Pemerintah Kota Kupang Berbasis Sumber Daya Manusia

Abstract: Purpose: The provision of community service activities aims to provide knowledge, understanding, and skills to state civil servants related to the ability to evaluate and reconstruct the performance of ASN based on human resources by organizational needs and existing conditions. Methodology/approach: Implementing activities using focus group discussions and workshops with the following stages: (1) surveys to find out partner problems, (2) needs analysis and initial questionnaire distribution using purposive sampling and quota sampling methods; (3) training and design consist of sharing information and materials, training on evaluation techniques, and training on reconstruction techniques; (4) the distribution of the final questionnaire to determine the increase in knowledge of participants' skills related to evaluation and reconstruction methods; and (5) prepare reports. Analysis of human resource-based ASN performance reconstruction using factorial analysis of SPSS vs. 25 Results/findings: The results of evaluating the performance of ASN based on human resources identified 33 dimensions of employee performance based on existing human resources. After being reconstructed into 9 dimensions of ASN performance based on human resources, namely professional, target, credible, teamwork, focus, capability, innovative, infrastructure, and perfectionist. Conclusion: The results of this training show that there is an increase in the knowledge and skills of ASN in the Kupang City Government to evaluate and reconstruct the elements of human resource-based performance appraisal and measurement that are in accordance with existing conditions and organizational needs. Keywords: 1. performance 2. human resource 3. evaluation 4. reconstruction

Analysis of human performance differences between students and operators when using the Rancor Microworld simulator

From within the umbrella of the Simplified Human Error Experimental Program (SHEEP) framework, this paper analyzes human performance differences between professional and student operators when using a simplified simulator (i.e., Rancor Microworld). This paper represents a crucial step in understanding the fidelity of the simplified simulators and student operators within the SHEEP study. This paper explores a randomized factorial experimental design that features two independent variables: participant type and event class. Six human performance measurements are considered in the experiment. The experiment is conducted using 20 professional reactor operators employed at actual nuclear power plants (NPPs), along with 20 trained students. The experimental data are analyzed via statistical analysis methods. Lastly, this paper examines the differences in human performance between actual operators and students when using Rancor Microworld.

Longitudinal Impacts of Simulated Long-Duration Spaceflight Missions on Operationally Relevant Measures of Human Performance Using a Portable Simulation Platform.

This project quantifies operationally relevant measures of flight performance and workload in a high-fidelity long-duration spaceflight analog, longitudinally across mission duration, using a portable simulation platform. Real-time performance measures allow for the objective assessment of task performance and the timely identification of performance degradations. Measures of flight performance on a piloted lunar lander task were collected on 32 total crewmembers across 8 simulated space missions of 45days each (623 total sessions). Mission duration demonstrated a significant effect on measures of flight performance across all campaigns. Flight measures showed a general pattern of peaking in accuracy during the middle-late quartiles of overall mission time, then degrading again towards baseline. On the workload measure, however, a general linear decrease in workload consistent with progressive task learning was observed in both campaigns. This investigation demonstrated the disruptive effect of time in mission on some, but not all, aspects of task performance. While mission interval differentially impacted measures of flight accuracy, workload, by contrast, seemed to steadily decrease with in-mission time. While more work is needed, the observed discrepancy between progression of flight performance and workload assessment highlights the importance of sensitive and specific measurement tools for the tracking of distinct performance metrics.

Aristotle’s Conception of Arete and the Meaning of Records in Sport

The modern sports world is currently obsessed with records, data, statistics, and/or the objective measurement of human performance. A primary message originating from this trend—manifesting in youth, club, interscholastic, intercollegiate, and professional sport—is that breaking records and improving one’s statistical output is the main objective of sport. In this article, we argue for why a predominant focus on records, stats, and winning is self-limiting and thereby misses the mark of what sport and sporting performance are. It is human beings who play sports rather than mere physical mechanical objects. Furthermore, we propose an arete-based philosophical perspective—taken directly from the ancient Greeks and particularly Aristotle—for how we ought to conceptualize and pursue sport. An arete-based philosophy captures the true essence of what sport is about by rooting it in what is “good and beautiful” (kalokagathia as the Greeks called it). Arete or “virtue” is, for Aristotle, about the cultivation of human excellence. Excellence, however, is not myopically reduced to “being the best,” “achieving fame or honor,” or “winning.” Instead, arete aims at cultivating the skills, both kinesthetic and moral, that lead to a good life. Elite performance, no matter how impressive, is never more than one small aspect of such a life. Character matters too, which means human excellence is never reducible to the measurable. After articulating this Aristotelian philosophy of sport, we then conclude the article by offering five recommendations for how teachers, coaches, and leaders of sport organizations can improve the culture of sport. Physical educators and coaches would be wise to take this Aristotelian conception of arete to heart.

Analisis Pengukuran Kinerja Sumber Daya Manusia Berbasis Balance Scorecard dengan pendekatan Human Resources Scorecard

Human resources are employees or individuals who play an important role in the success of a company. Human resources scorecard is part of the Balanced Scorecard that can be used to measure HR performance. This study aims to analyze the measurement of human resource performance with the Balanced Scorecard method with the human resources scorecard approach at PT. ISM in Pontianak, West Kalimantan Province. The method used in this research is descriptive and cross sectional. The type of data used is primary data, which is obtained from questionnaires containing questions and written statements. The results of the study found that the measurement of human resource performance seen from indicators of work quality, responsibility for work, cooperation with colleagues, orientation to customers, employee initiative and motivation was classified as good. Measurement of HR performance using indicators from the balance scorecard with the human resources scorecard approach, namely learning and growth perspective, internal business process perspective, customer perspective and financial perspective were found in good category. The company's employees are not only qualified in their work, have initiative, are motivated, cooperate, and are customer-oriented in their work, they have also implemented the principles of measuring the balanced scorecard performance with the human resources scorecard approach properly.

An Automatic Foot and Shank IMU Synchronization Algorithm: Proof-of-concept.

When using wearable sensors for measurement and analysis of human performance, it is often necessary to integrate and synchronise data from separate sensor systems. This paper describes a synchronization technique between IMUs attached to the shanks and insoles attached at the feet and aims to solve the need to compute the ankle joint angle, which relies on synchronized sensor data. This will additionally enable concurrent analysis using gait kinematic and kinetic features. A proof-of-concept of the algorithm, which relies on cross-correlation of gyroscope sensor data from the shank and foot, to align the sensor systems is demonstrated. The algorithm output is validated against those signals synchronized using manually annotated heel-strike and toe-off ground-truth signal landmarks, identified in both the shank and feet signals using previously published definitions. Results demonstrate that the developed algorithm is capable of synchronizing both sensor systems, based on IMU data from both healthy participants and participants suffering from knee osteoarthritis, with a mean lag time bias of 25.56ms when compared to the ground truth. A proof-of-concept of technique to synchronise IMUs attached to the shanks and insoles attached at the feet is demonstrated and offers an alternative approach to sensor system synchronisation.

The Bar-On Model and Multifactor Measure of Human Performance: Validation and Application.

In this article, we describe the ongoing validation and application of the Bar-On model of human performance that is assessed with the Multifactor Measure of Performance (MMP). (The Bar-On Multifactor Measure of Performance (MMP) is the intellectual property of Into Performance ULC.) The MMP is a psychometric instrument designed to study, evaluate and enhance performance. We discuss the meaning and importance of performance, and explain the need for creating and applying a comprehensive model and measure of this construct. To address this need, the MMP is structurally organized to assess and strengthen 18 Core Factors that contribute to performance. Five Ring Factors were added to facilitate a deeper understanding of leadership, industriousness, productiveness, risk for burnout, and coachability. Together, they represent a multifactor approach that focuses on current behavior of the “whole person” by evaluating physical, cognitive, personal, social, and inspirational factors combined. We discuss the properties of the MMP’s normative population, as the baseline for accurate reporting, tailored to different workplace activities and needs. Possible limitations of the research are indicated, together with the need for additional studies to address them. We reflect on the MMP within the Unified Validity Framework and conclude with recommendations for researchers and practitioners to apply this model and measure.

Developing a Human Performance Model Based Systems Engineering System Architecture (MBSE‐SA) for Defense Applications

AbstractThe military has limited means to objectively evaluate soldier and squad performance, leading to sub‐optimal operational, training, and acquisition decisions. Part of this limitation stems from the fact that the ability to monitor, predict, and enhance soldier and squad performance is a complex System of Systems (SoS) undertaking. To address this integration challenge, we developed a comprehensive conceptual framework that maps mission‐level success to measures of task effectiveness and human performance that quantify the impact of emerging technologies and interventions. In addition, by leveraging lessons learned from a previous effort that developed a Wearables MBSE system architecture (MBSE‐SA) and methodology for military health monitoring systems, we developed a Human Performance MBSE‐SA to capture and apply the conceptual framework, which provides the foundation for several system‐level and program‐level analyses, including gap identification. This paper presents the Human Performance MBSE‐SA and an illustration of its utility in showing the connections among ongoing military research efforts, strategic source requirements, and operational use cases.

Designing a sustainable digital control room for public transport: a comprehensive human performance measurement model

ABSTRACT The control rooms of organizations such as bus rapid transports and automated railroad systems have changed from conventional systems with analogue equipment to modern systems with digital equipment. These changes have necessitated adaptations in the responses and habits of operators carrying out tasks. Although enhancing the system from conventional to modern is intended to improve the system performance, it often exerts the opposite effect. Therefore, the risk of decreased operator performance needs to be evaluated. This research develops a design framework incorporating comprehensive dimensions for evaluating operator performances in digital control rooms. The evaluation is based on the theory of ergonomics and is implemented by the analytical hierarchical process and fuzzy set theory. The framework has been empirically tested in digital control room for bus rapid transport and thus, it can be customized and implemented to design the work and workplace of digital control rooms for public transport design.

Reconceptualizing faculty workload as a measure of human resource motivation and performance in higher education

This paper explores faculty workload as a measuring indicator that doubles as a motivator input and an output gauge of performance in higher education management. A desk-based methodology was applied. It was found that a number of studies support the use of faculty workload as a motivator and a measure of performance in higher education. It also noted that the use of faculty workload as a motivator; and measure of performance is faced with controversies typically emanating from the inability to determine the role of the variable in studies involving the use of it.