Human Operator Research Articles

ARTIFICIAL INTELLIGENCE APPLICATIONS IN CLINICAL MICROBIOLOGY LABORATORY

Artificial intelligence is becoming an increasingly important component of clinical microbiology informatics. Researchers, microbiologists, laboratorians, and diagnosticians are interested in AI-based testing because these applications have the potential to improve the turnaround time, quality, and cost of a test. Artificial intelligence which has gained importance in the laboratory, is used to support decision-making, identification and antimicrobial susceptibility testing with various technologies, image analyses, and MALDI-TOF-MS in medical microbiology and in infectious disease testing. Treatment of infections requires rapid and accurate identification and antimicrobial susceptibility testing. Modern artificial intelligence (AI) and machine-learning (ML) methods can now complete tasks with performance characteristic comparable to those of expert human operators. As a result, many healthcare fields combine these technologies, including in vitro diagnostics and, more broadly laboratory medicine, incorporate these technologies. These technologies are rapidly being developed and disclosed, but by comparison, their application so far has been limited. We need to further establish best practices and improve our information system and communications infrastructure to promote the implementation of reliable and advanced machine learning-based technologies. İnvolvement of the clinical microbiology laboratory community is essential to ensure that laboratory data is adequately accessible and thoughtfully incorporated into robust, safe and clinically effective ML-supported clinical diagnoses. This has been made possible by advances in modern computing and the widespread digitalization of health information. In this review, it is aimed to give information about to AI and its subfield of ML application areas in clinical microbiology laboratory.

ARTIFICIAL INTELLIGENCE IN FINANCIAL FORECASTING: TECHNIQUES AND APPLICATIONS

The capacity of artificial intelligence to swiftly evaluate massive amounts of information and implement high-frequency trades (HFT) has made AI a valuable tool for human operators. The most important studies that use innovative techniques to predict patterns of financial assets are analysed in this article, along with an assessment of their usefulness and potential applications in investing in intricate financial sectors. These structures investigate connections and factors that impact trading performance via the application of machine learning and deep learning algorithms. Forecasts are calculated from either linear or nonlinear methods and often include sentiment evaluation or trend identification from internet-based participants. The majority of papers that have been examined have shown that their artificial intelligence can be used to trade financial markets successfully.

Performance-Aware Trust Modeling within a Human–Multi-Robot Collaboration Setting

In this study, a novel time-driven mathematical model for trust is developed considering human–multi-robot performance for a Human–Robot Collaboration (HRC) framework. For this purpose, a model is developed to quantify human performance considering the effects of physical and cognitive constraints and factors such as muscle fatigue and recovery, muscle isometric force, human (cognitive and physical) workload, workloads due to the robots’ mistakes, and task complexity. The performance of multi-robot in the HRC setting is modeled based upon the rate of task assignment and completion as well as the mistake probabilities of the individual robots. The human trust in HRC setting with single and multiple robots is modeled over different operation regions, namely unpredictable region, predictable region, dependable region, and faithful region. The relative performance difference between the human operator and the robot is used to analyze the effect on the human operator’s trust in robots’ operation. The developed model is simulated for a manufacturing workspace scenario considering different task complexities and involving multiple robots to complete shared tasks. The simulation results indicate that for a constant multi-robot performance in operation, the human operator’s trust in robots’ operation improves whenever the comparative performance of the robots improves with respect to the human operator performance. The impact of robot hypothetical learning capabilities on human trust in the same HRC setting is also analyzed. The results confirm that a hypothetical learning capability allows robots to reduce human workloads, which improves human performance. The simulation result analysis confirms that the human operator’s trust in the multi-robot operation increases faster with the improvement of the multi-robot performance when the robots have a hypothetical learning capability. An empirical study was conducted involving a human operator and two collaborator robots with two different performance levels in a software-based HRC setting. The experimental results closely followed the pattern of the developed mathematical models when capturing human trust and performance in terms of human–multi-robot collaboration.

Artificial intelligence, inattention and liability rules

We characterize the socially optimal liability sharing rule in a situation where a manufacturer develops an artificial intelligence (AI) system that is then used by a human operator (or user). First, the manufacturer invests to increase the autonomy of the AI (i.e, the set of situations that the AI can handle without human intervention) and sets a selling price. The user then decides whether or not to buy the AI. Since the autonomy of the AI remains limited, the human operator must sometimes intervene even when the AI is in use. Our main assumptions relate to behavioral inattention. Behavioral inattention reduces the effectiveness of user intervention and increases the expected harm. Only some users are aware of their own attentional limits. Under the assumption that AI outperforms users, we show that policymakers may face a trade-off when choosing how to allocate liability between the manufacturer and the user. Indeed, the manufacturer may underinvest in the autonomy of the AI. If this is the case, the policymaker can incentivize the latter to invest more by increasing his share of liability. On the other hand, increasing the liability of the manufacturer may come at the cost of slowing down the diffusion of AI technology.

Application of Belief Theories for Railway Track Defect Detection

Faced with increasing traffic, railway infrastructures are encountering growing demands, particularly in high-traffic areas. In this context, rail and sleepers emerge as the components most susceptible to failure. To assist infrastructure managers (IM) in optimizing network maintenance, we have explored a novel method for detecting critical defects on the track. The objective is to develop a process for real-time analysis of railway infrastructure that is both frugal and efficient and can be installed on board commercial trains. This new infrastructure monitoring system integrates deep learning networks with a data fusion model based on belief theory. By modeling the decision-making process of a human operator, this processing chain has achieved detection rates exceeding 90% for the five primary defects: defective fasteners, broken fishplates and rails, surface defects, and missing nuts.

Development of a method for recognizing emotions from a speech signal

The purpose of research is automatic recognition of the speaker's emotions, based on the processing of sound recordings intended for use in alarm systems when working with operators of locomotive crews and dispatch services.Methods. Human emotion recognition has been a rapidly developing area of research in recent years. Features of the vocal tract, such as sound power, formant frequencies, are used to detect certain emotions with good accuracy. A method was used to determine the signal energy by highlighting the dominant frequency. The work has developed a program code, on the basis of which an analysis of four emotions is given - anger, joy, fear and calm. The most important and difficult step is to determine the features most suitable for distinguishing emotions and the availability of databases. Collecting databases is a complex task requiring the manifestation of sincerity of emotions. Often, the collection of a database takes place in an artificial environment and the speech may sound staged; to eliminate such problems, it is necessary to use call center recordings.Results. Recordings of basic emotional states, such as anger, joy, sadness, fear and surprise, which are the most common case of the study, were obtained and processed. The developed software code allows us to get closer to automatically determining emotions from a speech signal. To analyze speech recordings in samples, indicators of signal energy and identification of the dominant frequency were used.Conclusion. The implemented method of monitoring the emotional state of a human operator using a speech signal is widely used in the prevention and improvement of indicators of the psychophysiological professional suitability of locomotive crew workers and the preservation of their professional health. Distinct differences are observed in the characteristics of all types of emotions.

A hybrid task-constrained motion planning for collaborative robots in intelligent remanufacturing

Industrial manipulators have extensively collaborated with human operators to execute tasks, e.g., disassembly of end-of-use products, in intelligent remanufacturing. A safety task execution requires real-time path planning for the manipulator’s end-effector to autonomously avoid human operators. This is even more challenging when the end-effector needs to follow a planned path while avoiding the collision between the manipulator body and human operators, which is usually computationally expensive and limits real-time application. This paper proposes an efficient hybrid motion planning algorithm that consists of A∗ algorithm and an online manipulator reconfiguration mechanism (OMRM) to tackle such challenges in task and configuration spaces, respectively. A∗ algorithm is first leveraged to plan the shortest collision-free path of the end-effector in task space. When the manipulator body is risky to the human operator, our OMRM then selects an alternative joint configuration with minimum reconfiguration effort from a database to assist the manipulator to follow the planned path and avoid the human operator simultaneously. The database of manipulator reconfiguration establishes the relationship between the task and configuration space offline using forward kinematics, and is able to provide multiple reconfiguration candidates for a desired end-effector’s position. The proposed new hybrid algorithm plans safe manipulator motion during the whole task execution. Extensive numerical and experimental studies, as well as comparison studies between the proposed one and the state-of-the-art ones, have been conducted to validate the proposed motion planning algorithm.

Urban micro-scale street thermal comfort prediction using a ‘graph attention network’ model

Outdoor thermal comfort (OTC) directly affects human behavior and building operations. It is also a key factor in the achievement of smart living. When modeling OTC, existing studies tend to reduce the computational burden by conceptualizing urban spatial elements (e.g., buildings, streets) as static entities without considering the intricate dynamics of their synergistic influence. This research presents an innovative network-based framework for urban micro-scale street OTC prediction. The proposed graph attention network (GAT), in conjunction with building energy modeling (BEM), treating building clusters as graph nodes and streets as edges, capturing the interrelations between urban spatial elements, and modeling the street's universal thermal climate index (UTCI) at different time periods. The GAT model is trained and tested using simulated data from representative high-density residential areas in Hong Kong. Its performance is evaluated against an artificial neural network (ANN) model that disregards interrelations among urban spatial elements. Results demonstrate that, compared to the ANN model, the GAT model achieves an improvement in overall mean absolute error (MAE) of 37.5 %, root mean square error (RMSE) of 36.07 %, and correlation coefficient (r) of 10.58 %. Furthermore, the GAT model can better predict situations with significant amplitude changes and rapid frequency variations.

BRON: A blockchained framework for privacy information retrieval in human resource management

The correctness and the true validated data in Human Resource Management (HRM) are important for organizations as the data plays an impactful role in recruiting, developing, and retaining a skilled workforce. On one hand, the validated data in an organization helps in recruiting legitimate skillful employees; on the other hand, keeping the employee's data safe and maintaining privacy laws such as compliance with the General Data Protection Regulation (GDPR) is also an organization's responsibility. Besides, transparency in human resource management operations is crucial because it promotes trust and fairness within an organization. The present HRM systems are centralized in nature and their verifiable credential system is ineffective; this leads to the intentions of internal data sabotage or internal threats. Besides, the organizations' biases also become more prominent.In this paper, we address the above-mentioned problems with a blockchain framework for HRM to utilize the privacy of data access through a Privacy Information Retrieval (PIR) process. To be specific, our proposed framework called Blockchained piR of resOurces as humaN (BRON), is the first blockchain framework to show an effective mechanism to access data from organizations globally without hampering privacy. BRON uses a generalized user registration process to use the services of data access and in the background, it uses Zero-Knowledge Proofs (ZKPs) for global verification and PIR for privacy-based data retrieval. More specifically, credential verification and ZKP-based PIR are the highlights of our proposed BRON. Another interesting aspect of BRON is the use of Proof-of-Authority (PoA) to validate the anonymity and unlinkability of any HR operation. Finally, BRON has also contributed with a smart contract to incentivize the employees. BRON is very generic and easily be customizable as per the HR requirements. We run a set of experiments on BRON and observe that it is successful in providing privacy-assured data access and decentralized human resource data management. Overall, BRON provides 30% reduced latency and 35% better throughput as compared to the existing blockchain solutions in the direction of HRM.

Trust in automation and the accuracy of human–algorithm teams performing one-to-one face matching tasks

The human face is commonly used for identity verification. While this task was once exclusively performed by humans, technological advancements have seen automated facial recognition systems (AFRS) integrated into many identification scenarios. Although many state-of-the-art AFRS are exceptionally accurate, they often require human oversight or involvement, such that a human operator actions the final decision. Previously, we have shown that on average, humans assisted by a simulated AFRS (sAFRS) failed to reach the level of accuracy achieved by the same sAFRS alone, due to overturning the system’s correct decisions and/or failing to correct sAFRS errors. The aim of the current study was to investigate whether participants’ trust in automation was related to their performance on a one-to-one face matching task when assisted by a sAFRS. Participants (n = 160) completed a standard face matching task in two phases: an unassisted baseline phase, and an assisted phase where they were shown the identification decision (95% accurate) made by a sAFRS prior to submitting their own decision. While most participants improved with sAFRS assistance, those with greater relative trust in automation achieved larger gains in performance. However, the average aided performance of participants still failed to reach that of the sAFRS alone, regardless of trust status. Nonetheless, further analysis revealed a small sample of participants who achieved 100% accuracy when aided by the sAFRS. Our results speak to the importance of considering individual differences when selecting employees for roles requiring human–algorithm interaction, including identity verification tasks that incorporate facial recognition technologies.

Aerodynamic Feedforward-Feedback Architecture for Tailsitter Control in Hybrid Flight Regimes

This paper presents a guidance-control design methodology for the autonomous maneuvering of tailsitter transitioning unmanned aerial systems (t-UASs) in hybrid flight regimes (i.e., the dynamics between VTOL and fixed-wing regimes). The tailsitter guidance-control architecture consists of a trajectory planner, an outer-loop position controller, an inner-loop attitude controller, and a control allocator. The trajectory planner uses a simplified tailsitter model, with aerodynamic and wake effect considerations, to generate a set of transition trajectories with associated aerodynamic force estimates based on an optimization metric specified by a human operator (minimum time transition). The outer-loop controller then uses the aerodynamic force estimate computed by the trajectory planner as a feedforward signal alongside feedback linearization of the outer-loop dynamics for 6DOF position control. The inner-loop attitude controller is a standard nonlinear dynamic inversion control law that generates the desired pitch, roll, and yaw moments, which are then converted to the appropriate rotor speeds by the control allocator. Analytical conditions for robust stability are derived for the outer-loop position controller to guarantee performance in the presence of uncertainty in the feedforward aerodynamic force compensation. Finally, both tracking performance and stability of the control architecture are evaluated on a high-fidelity flight dynamics simulation of a quadrotor biplane tailsitter for flight missions that demand high maneuverability in transition between flight modes.

Multi-Robot Patrol with Continuous Connectivity and Assessment of Base Station Situation Awareness

Patrolling represents a potential application area for multi-robot systems, as it can enable efficient surveillance. A key aspect in facilitating the real-world applications of such missions is the enhancement of situation awareness of the base station (BS), in addition to ensuring well-coordinated patrol behavior. This paper addresses this requirement by proposing a layered patrol algorithm designed to maintain network connectivity with the BS. The novelty of this research lies in the distributed nature of the algorithm, despite the presence of the BS. Each robot independently determines its behavior based on local information while concurrently preserving connectivity to the BS. Additionally, this study introduces a novel performance metric to assess the situation awareness of the BS, focusing on the algorithm’s ability to provide prompt information about mission progress. Simulated missions revealed that the proposed algorithm outperformed existing algorithms, visited locations of interest more frequently and comprehensively, and provided the BS with improved situation awareness. Enhancing situation awareness may enable human operators to quickly gain insights into the system’s behavior based on mission progress, allowing for timely interventions if necessary. This capability contributes to improving human trust in autonomous systems.

The Influence of the Quality of Human Resources, Operator Training and Work Facilities on the Effectiveness of the Village Financial System in Banjarnegara

This research aims to assess the influence of the quality of human resources, operator training, and work facilities on the effectiveness of the use of Siskeudes in village government. The Siskeudes application is run by operators, namely village officials/village officials who have participated in training activities. There are obstacles in using the Siskeudes application, such as the quality of human resources which are not yet optimally qualified in relation to the use of technology, facilities which are not updated and up to date, and a lack of training to run and understand the Siskeudes application. The research method used was quantitative. Data collected through questionnaires that were distributed to Siskeudes operators in villages in Banjarnegara regency with a sample of 100 villages. The results of the research show that the quality of human resources has a positive and significant effect on the effectiveness of using Siskeudes. Training does not have a positive and significant effect on the effectiveness of using Siskeudes. Work facilities have a positive and significant effect on the effectiveness of using Siskeudes. The government needs to improve the quality of human resources for village officials by holding intensive training on the use of Siskeudes, and adding or updating existing work facilities to maximize the operation of the Siskeudes.

Incorporating uncertain human behavior in production scheduling for enhanced productivity in Industry 5.0 context

Human-centered production systems are of increasing interest to researchers, especially with the advent of the Industry 5.0 paradigm. Most research into production scheduling has long neglected human workers’ specific roles and unpredictable behavior in a production system, treating them as machines with deterministic behavior. This work studies the impact of human operational behavior on the performance of a production system and proposes an optimization model to allocate workers’ profiles to workstations. We modeled the punctuality profile as a Markov chain representing a worker’s productive and non-productive states. We developed a simulation process based on the multi-agent system (MAS) paradigm to test the effectiveness of the proposed model and to measure the impact of workers’ behaviors and their assignments to different workstations on the productivity of the workshop. A non-linear programming model is also proposed to provide the optimal assignment of workers to workstations while maximizing the throughput of a dual-resource-constrained flow-shop production system for a given mix of production. The results obtained highlight the significant impact of human operator behavior on the performance of a production system. The findings demonstrate the importance of incorporating human behavior models into the decision-making process for assigning workers to workstations based on their operational profiles.

HUMAN FOLLOWING CART

The development of a human-following robot utilizing Raspberry Pi and camera technology represents a significant advancement in autonomous robotics, offering practical solutions for various industries. This project integrates a Raspberry Pi microcomputer, a camera, and ultrasonic sensors to create a robot capable of tracking and following a human operator. The system employs computer vision algorithms, facilitated by a Haar Cascade Classifier, to detect faces in real-time video feeds. The video feed is processed to determine the position of the human within the frame, enabling the robot to navigate accordingly. Ultrasonic sensors are used to measure the distance between the robot and the human, ensuring that a safe and optimal following distance is maintained. The motors and motor drivers are controlled based on the visual and distance data, allowing the robot to move forward, backward, or turn as needed. This combination of technologies enhances worker safety by reducing physical strain and improves operational efficiency by automating the transportation of goods, tools, and supplies. The implementation of this system demonstrates the practical application of affordable and accessible technologies like the Raspberry Pi in creating intelligent, autonomous robots. The project highlights the potential for such robots to revolutionize workflows in warehouses, manufacturing plants, and hospitals, making advanced automation more attainable for a wide range of businesses. The integration of computer vision and sensor-based navigation in human-following robots marks a significant step forward in the evolution of intelligent workspaces and autonomous systems.

Robust Adaptive-Sliding-Mode Control for Teleoperation Systems with Time-Varying Delays and Uncertainties

Master–slave teleoperation systems with haptic feedback enable human operators to interact with objects or perform tasks in remote environments. This paper presents a sliding-mode control scheme tailored for bilateral teleoperation systems operating in the presence of unknown uncertainties and time-varying delays. To address unknown but bounded uncertainties, adaptive laws are derived alongside controller design. Additionally, a linear matrix inequality is solved to determine the allowable bound of delays. Stability of the closed-loop system is ensured through Lyapunov–Krasovskii functional analysis. Two-degree-of-freedom mechanisms are self-built as haptic devices. Free-motion and force-perception scenarios are examined, with experimental results validating and comparing performances. The proposed adaptive-sliding-control method increases the position performance from 58.48% to 82.55% and the force performance from 83.48% to 99.77%. The proposed control scheme demonstrates enhanced position tracking and force perception in bilateral teleoperation systems.

The Best Decisions Are Not the Best Advice: Making Adherence-Aware Recommendations

Many high-stake decisions follow an expert-in-loop structure in that a human operator receives recommendations from an algorithm but is the ultimate decision maker. Hence, the algorithm’s recommendation may differ from the actual decision implemented in practice. However, most algorithmic recommendations are obtained by solving an optimization problem that assumes recommendations will be perfectly implemented. We propose an adherence-aware optimization framework to capture the dichotomy between the recommended and the implemented policy and analyze the impact of partial adherence on the optimal recommendation. Our framework provides useful tools to analyze the structure and to compute optimal recommendation policies that are naturally immune against such human deviations and are guaranteed to improve upon the baseline policy. This paper was accepted by Nicolas Stier-Moses, special issue on the human-algorithm connection. Funding: J. Grand-Clément was supported by the Agence Nationale de la Recherche [Grant 11-LABX-0047] and Hi! Paris. Supplemental Material: The data files are available at https://doi.org/10.1287/mnsc.2023.01851 .

Deception abilities emerged in large language models

Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Thus, aligning them with human values is of great importance. However, given the steady increase in reasoning abilities, future LLMs are under suspicion of becoming able to deceive human operators and utilizing this ability to bypass monitoring efforts. As a prerequisite to this, LLMs need to possess a conceptual understanding of deception strategies. This study reveals that such strategies emerged in state-of-the-art LLMs, but were nonexistent in earlier LLMs. We conduct a series of experiments showing that state-of-the-art LLMs are able to understand and induce false beliefs in other agents, that their performance in complex deception scenarios can be amplified utilizing chain-of-thought reasoning, and that eliciting Machiavellianism in LLMs can trigger misaligned deceptive behavior. GPT-4, for instance, exhibits deceptive behavior in simple test scenarios 99.16% of the time (P < 0.001). In complex second-order deception test scenarios where the aim is to mislead someone who expects to be deceived, GPT-4 resorts to deceptive behavior 71.46% of the time (P < 0.001) when augmented with chain-of-thought reasoning. In sum, revealing hitherto unknown machine behavior in LLMs, our study contributes to the nascent field of machine psychology.

Comparison Between an Expert Operator an Inexperienced Operator, and Artificial Intelligence Software: A Brief Clinical Study of Cephalometric Diagnostic.

Artificial intelligence (AI) is constantly developing in several medical areas and has become useful to assist with treatment planning. Orthodontics and maxillofacial surgery use AI-based technology to identify and select cephalometric points for diagnostics. Although some studies have shown promising results from the use of AI, the evidence is still limited. Hence, additional investigation is justified. In this retrospective study, 2 human operators (1 expert and 1 inexperienced) and 1 software analyzed 30 lateral cephalograms of individuals with orthodontic treatment indications. They measured 10 cephalometric variables and then 2 weeks later, repeated measurements on 30% of the sample. We evaluated the reliability of the measurements between the 2-time points and the differences in the means between the expert operator and the AI software and between the expert and inexperienced operators. There was high reliability for the expert operator and AI measurements, and moderate reliability for the inexperienced operator measurements. There were some significant differences in the means produced by the AI software and the inexperienced operator compared with the expert operator. Although AI is useful for cephalometric analysis, it should be used with caution because there are differences compared with analysis by humans.

Interaction between clinicians and artificial intelligence to detect fetal atrioventricular septal defects on ultrasound: how can we optimize collaborative performance?

Artificial intelligence (AI) has shown promise in improving the performance of fetal ultrasound screening in detecting congenital heart disease (CHD). The effect of giving AI advice to human operators has not been studied in this context. Giving additional information about AI model workings, such as confidence scores for AI predictions, may be a way of further improving performance. Our aims were to investigate whether AI advice improved overall diagnostic accuracy (using a single CHD lesion as an exemplar), and to determine what, if any, additional information given to clinicians optimized the overall performance of the clinician-AI team. An AI model was trained to classify a single fetal CHD lesion (atrioventricular septal defect (AVSD)), using a retrospective cohort of 121 130 cardiac four-chamber images extracted from 173 ultrasound scan videos (98 with normal hearts, 75 with AVSD); a ResNet50 model architecture was used. Temperature scaling of model prediction probability was performed on a validation set, and gradient-weighted class activation maps (grad-CAMs) produced. Ten clinicians (two consultant fetal cardiologists, three trainees in pediatric cardiology and five fetal cardiac sonographers) were recruited from a center of fetal cardiology to participate. Each participant was shown 2000 fetal four-chamber images in a random order (1000 normal and 1000 AVSD). The dataset comprised 500 images, each shown in four conditions: (1) image alone without AI output; (2) image with binary AI classification; (3) image with AI model confidence; and (4) image with grad-CAM image overlays. The clinicians were asked to classify each image as normal or AVSD. A total of 20 000 image classifications were recorded from 10 clinicians. The AI model alone achieved an accuracy of 0.798 (95% CI, 0.760-0.832), a sensitivity of 0.868 (95% CI, 0.834-0.902) and a specificity of 0.728 (95% CI, 0.702-0.754), and the clinicians without AI achieved an accuracy of 0.844 (95% CI, 0.834-0.854), a sensitivity of 0.827 (95% CI, 0.795-0.858) and a specificity of 0.861 (95% CI, 0.828-0.895). Showing a binary (normal or AVSD) AI model output resulted in significant improvement in accuracy to 0.865 (P < 0.001). This effect was seen in both experienced and less-experienced participants. Giving incorrect AI advice resulted in a significant deterioration in overall accuracy, from 0.761 to 0.693 (P < 0.001), which was driven by an increase in both Type-I and Type-II errors by the clinicians. This effect was worsened by showing model confidence (accuracy, 0.649; P < 0.001) or grad-CAM (accuracy, 0.644; P < 0.001). AI has the potential to improve performance when used in collaboration with clinicians, even if the model performance does not reach expert level. Giving additional information about model workings such as model confidence and class activation map image overlays did not improve overall performance, and actually worsened performance for images for which the AI model was incorrect. © 2024 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.