Semi-autonomous System Research Articles

Towards full autonomy in mobile robot navigation and manipulation

AbstractRobot autonomy is a key for automatizing industrial production plants, facilitating responsive search and rescue (SAR), and for interacting in urban settings. This work presents mobile systems capable of carrying out these tasks and summarises experience from practical deployments in three different domains for navigation. For industry, a mobile manipulator is presented, which is used for dynamic precise docking and object handling. For SAR, a semi-autonomous system is presented which is used for sampling hazardous materials, manipulating tools, and for localizing nuclear radiation sources. To improve urban logistics, a mobile robot autonomously navigates complex outdoor environments using semantic mapping techniques. Future work will explore further advancements that strive toward increased flexibility, reasoning and universal applicability of mobile robots. The goal is to reduce training, onboarding and tool changing times to reduce manual interfacing times.

Implementing social and affective touch to enhance user experience in human-robot interaction.

In this paper, we discuss the potential contribution of affective touch to the user experience and robot performance in human-robot interaction, with an in-depth look into upper-limb prosthesis use as a well-suited example. Research on providing haptic feedback in human-robot interaction has worked to relay discriminative information during functional activities of daily living, like grasping a cup of tea. However, this approach neglects to recognize the affective information our bodies give and receive during social activities of daily living, like shaking hands. The discussion covers the emotional dimensions of affective touch and its role in conveying distinct emotions. In this work, we provide a human needs-centered approach to human-robot interaction design and argue for an equal emphasis to be placed on providing affective haptic feedback channels to meet the social tactile needs and interactions of human agents. We suggest incorporating affective touch to enhance user experience when interacting with and through semi-autonomous systems such as prosthetic limbs, particularly in fostering trust. Real-time analysis of trust as a dynamic phenomenon can pave the way towards adaptive shared autonomy strategies and consequently enhance the acceptance of prosthetic limbs. Here we highlight certain feasibility considerations, emphasizing practical designs and multi-sensory approaches for the effective implementation of affective touch interfaces.

Semi-autonomous sensor fusion-based strategy for unmanned aerial atmospheric surveillance system in open field environments

<p>This paper explores the convergence of semi-autonomous systems and sensor fusion for monitoring hazardous atmospheric substances in open and semi-confined environments such as open-pit mines. As the heart of this system, a universal and flexible device leveraging wireless technologies to collect and analyze large volumes of data, designed by the authors, is proposed. The article outlines the key components of the platform, emphasizing its potential for increasing personnel safety levels as stipulated by the international public exposure guidelines. In modern mines, it becomes crucial to monitor elevated concentrations of nitrogen and carbon oxides, as well as other pollutants after blasting and the toxic gas emissions produced by the heavy transportation equipment such as mining trucks and excavators. In this paper, industrial-grade electrochemical sensors are compared with price-affordable but less precise microelectrochemical ones. The performed experiments for lowering computational requirements show promising results. The integration of an ultrasonic anemometer enhances the system’s capabilities, contributing to a comprehensive understanding of atmospheric conditions. In parallel, the research discusses the role of computer vision in autonomous control systems, with a focus on the architecture and processing pipeline of the state-of-the-art system-on-module Kria by AMD. Advocating the potential of adaptive computing for enhanced efficiency in dynamic environments, the paper underlines the importance of the integrated approach in developing a semi-autonomous atmospheric surveillance system and highlights the impact of adaptive computing in dynamic scenarios. The main part of the measuring equipment and methods has been experimented with in real conditions in open-pit mines in Bulgaria. This is an initial phase of ongoing research aimed at serving as a foundation for future improvements and the elaboration of a fully autonomous prototype for hazardous substances evaluation in the atmosphere of open-pit mines.<strong></strong></p>

A Situated Inspection of Autonomous Vehicle Acceptance – A Population Study in Virtual Reality

The successful integration of Autonomous Vehicles (AVs) into daily life is influenced by various individual and contextual factors. This study examines key factors affecting acceptance and trust in AVs by assessing how interactive experiences influence user perceptions and adoption. Utilizing a Virtual Reality (VR) environment to simulate real-world driving conditions, the research investigates the impact of different levels of vehicle interaction—manual control, semi-autonomous, fully autonomous, and taxi-riding modes—on user and passenger trust, anxiety, perceived ease of use, and overall acceptance. A dual-method approach was employed, combining objective data collection through eye-tracking, which measures engagement and vigilance, with subjective user feedback gathered via the Autonomous Vehicle Acceptance Model (AVAM) questionnaire. The findings reveal a complex relationship between user control and trust in AV systems. While semi-autonomous conditions that incorporate user participation in decision-making enhance trust and intention to use, fully autonomous conditions influence trust but not intention to use significantly. However, semi- autonomous conditions also elicited higher anxiety levels compared to fully autonomous modes, suggesting a trade-off between interaction and comfort. Engagement levels, as indicated by visual metrics such as gaze patterns, were found to be critical for understanding user acceptance. These results highlight the importance of considering both subjective perceptions and objective behaviors in developing AV interfaces. The study contributes significant insights into the cognitive underpinnings of AV acceptance and underscores the need for strategies that both enhance user trust and minimize anxiety by finding an optimal balance between automation and user control. Further research into this balance is recommended to better accommodate the evolving nature of user trust and the cognitive complexities associated with semi-autonomous systems.

Suboptimal Trajectory Planning Technique in Real UAV Scenarios with Partial Knowledge of the Environment

In recent years, the issue of trajectory planning for autonomous unmanned aerial vehicles (UAVs) has received significant attention due to the rising demand for these vehicles across various applications. Despite advancements, real-time trajectory planning remains computationally demanding, particularly with the inclusion of 3D localization using computer vision or advanced sensors. Consequently, much of the existing research focuses on semi-autonomous systems, which rely on ground assistance through the use of external sensors (motion capture systems) and remote computing power. This study addresses the challenge by proposing a fully autonomous trajectory planning solution. By introducing a real-time path planning algorithm based on the minimization of the snap, the optimal trajectory is dynamically recalculated as needed. Evaluation of the algorithm’s performance is conducted in an unknown real-world scenario, utilizing both simulations and experimental data. The algorithm was implemented in MATLAB and subsequently translated to C++ for onboard execution on the drone.

The Importance of System Transition in The Transfer of State Assets Management to Public Institutions: Analysis at The State Islamic University - Public Service Agency in Indonesia

State Higher Education Institutions in Indonesia have experienced a paradigm shift in their management, one of which is the State Islamic University. More than half of State Islamic Universities have Public Service Agency status. They focus on teaching, research, and community service as their primary business and must manage assets professionally and with economic value as potential income. This study seeks to uncover and assess asset management and management transformation of UINs with BLU status. This study uses a qualitative research paradigm with a multi-site approach. This research used in-depth interviews conducted naturally with key informants at three state Islamic universities in Indonesia that have had BLU status for more than ten years. These three universities are considered to have experience in managing assets and carrying out transformation. Data analysis approaches are Domain and Content Analysis. The research data is presented in four domains: cash management, tangible asset management, intangible asset management, and endowment fund management. Research data found that UINs with BLU status still manage assets in a normative and conservative manner. This finding is because UIN, which has BLU status, has yet to transform its university management fully. This study shows that there are demands for UIN with BLU status to transform from a government institution into an "agencification" and semi-autonomous system in line with the delegation of management of state assets from the central government.

Hybrid decision-making in atmospheric plasma spraying enables human–machine teaming

With the development of human-cyber-physical-production systems in intelligent manufacturing, cyber-supported production based on artificial intelligence is becoming an increasingly powerful means of controlling machines and collaborating with human users. Semi-autonomous systems with a medium degree of automation enable human-centered, flexible, and sustainable production, for instance, in hybrid decision-making. Especially in applications that do not meet the requirements for full automation and when humans are to be involved in their role as qualified decision-makers, teaming-capable systems are desirable and offer considerable advantages. This paper outlines the transdisciplinary concept of human–machine teaming and the role of human cognition in engineering tasks with multi-criteria decision-making. An illustrative real-life example from thermal spray technology is used to show how explainable artificial intelligence models offer targeted, hybrid cyber decision support. This new approach based on fuzzy pattern classifiers combines expert knowledge- and data-based modeling and enables a transparent interpretation of the results by the human user, as shown here using the example of test data from atmospheric plasma spraying. The method outlined can potentially be used to provide hybrid decision support for a variety of manufacturing processes and form the basis for advanced automation or teaming of humans and cyber-physical-production systems.

Preserving paradata for accountability of semi-autonomous AI agents in dynamic environments: An archival perspective

This paper proposes the category of real-time artificial intelligence (AI) systems as applications of computerized control systems in dynamic, time-constrained contexts normally managed by human intelligence. Noting the accountability challenges which these systems introduce, the paper posits the need for robust documentation and records capacities within these systems. The paper surveys four real-time AI systems with significant records needs: autonomous vehicles, online content targeting systems, mixed-reality tools for surgical contexts, and digital twin systems in airport facilities management. The paper identifies paradata, or the data leading up to an output in a system's operation, as a key data category necessitating preservation for full transparency in the records generated by these systems. Paradata is defined as “information about the procedure(s) and tools used to create and process information resources, along with information about the persons carrying out those procedures.” Paradata uncovers opaque technological processes underlying the production of other datasets and at a granular level must be identified and preserved to delineate the boundaries between human and system agency in semi-autonomous systems. With a basis in control theory, the paper finally offers a framework for assessing the functions of real-time AI systems' operations and their documentation and records needs.

Semiautonomous recovery system from a stuck state of an articulated mobile robot

ABSTRACT A semiautonomous system is presented in this paper that recovers an articulated mobile robot from a stuck state in which the robot cannot move forward due to contact with the environment. Assuming the specific robot, the stuck states are classified into four types, depending on the position of contact with the environment. In the recovery system, the contact area causing the movement failure is identified using a contact position and magnitude of contact force obtained by the pressure sensor mounted on the robot. Motion candidates for recovery from each stuck state are autonomously calculated and presented to the operator via the user interface. The operator then selects an appropriate motion from the candidates, and the robot performs the recovery motion. An articulated mobile robot with sensors was developed, and experiments were conducted to demonstrate the effectiveness of the proposed system. As a result, the recovery of the robot from each stuck state using the proposed system was verified.

Robotic technology in total knee arthroplasty.

Key factors for successful total knee arthroplasty include accurate implant positioning with precise tibial and femoral resection, combined with appropriate soft tissue balancing to achieve the desired alignment. Robotic-assisted total knee arthroplasty allows surgeons to execute pre-planned strategies with precision, with growing evidence suggesting that robotic-assisted-total knee arthroplasty reduces radiological outliers. This has yet to be proven to translate into long-term improvements in patient-reported outcomes and implant survivorship. Robotic-assisted-total knee arthroplasty systems can be divided into fully autonomous and semi-autonomous systems. While fully autonomous systems showed initial promise, semi-autonomous systems are gaining popularity with encouraging early outcomes suggesting improved radiological and clinical outcomes, although concerns remain regarding a significant learning curve, installation costs, radiation exposure and cost associated with preoperative imaging. The future of total knee arthroplasty seems certain to involve robotic technology, although to what degree and in what capacity will depend on further high-quality studies assessing long-term outcomes, complications, survivorship and cost-benefit analyses.

REALMS: Resilient exploration and lunar mapping system.

Space resource utilisation is opening a new space era. The scientific proof of the presence of water ice on the south pole of the Moon, the recent advances in oxygen extraction from lunar regolith, and its use as a material to build shelters are positioning the Moon, again, at the centre of important space programs. These worldwide programs, led by ARTEMIS, expect robotics to be the disrupting technology enabling humankind's next giant leap. However, Moon robots require a high level of autonomy to perform lunar exploration tasks more efficiently without being constantly controlled from Earth. Furthermore, having more than one robotic system will increase the resilience and robustness of the global system, improving its success rate, as well as providing additional redundancy. This paper introduces the Resilient Exploration and Lunar Mapping System, developed with a scalable architecture for semi-autonomous lunar mapping. It leverages Visual Simultaneous Localization and Mapping techniques on multiple rovers to map large lunar environments. Several resilience mechanisms are implemented, such as two-agent redundancy, delay invariant communications, a multi-master architecture different control modes. This study presents the experimental results of REALMS with two robots and its potential to be scaled to a larger number of robots, increasing the map coverage and system redundancy. The system's performance is verified and validated in a lunar analogue facility, and a larger lunar environment during the European Space Agency (ESA)-European Space Resources Innovation Centre Space Resources Challenge. The results of the different experiments show the efficiency of REALMS and the benefits of using semi-autonomous systems.

ASAP: A Semi-Autonomous Precise System for Telesurgery During Communication Delays

In remote, rural, and disadvantaged areas, telesurgery can be severely hindered by limitations of communication infrastructure. In conventional telesurgery, delays as small as 300ms can produce fatal surgical errors. To mitigate the effect of communication delays during telesurgery, we introduce a semi-autonomous system that decouples the user interaction from the robot execution. This system uses a physics-based simulator where a surgeon can demonstrate individual surgical subtasks, with immediate graphical feedback. Each subtask is performed asynchronously, unaffected by communication latency, jitter, and packet loss. A surgical step recognition module extracts the intended actions from the observed surgeon-simulation interaction. The remote robot can perform each one of these actions autonomously. The action recognition system leveraged a transfer learning approach that minimized the data needed during training, and most of the learning is obtained from simulated data. We tested this system in two tasks: fluid-submerged peg transfer (resembling bleeding events) and surgical debridement. The system showed robustness to delays of up to 5 seconds, maintaining a performance rate of 87% for peg transfer and 88% for debridement. Also, the framework reduced the completion time under delays by 45% and 11% during peg transfer and debridement, respectively.

Competence-aware systems

Building autonomous systems for deployment in the open world has been a longstanding objective in both artificial intelligence and robotics. The open world, however, presents challenges that question some of the assumptions often made in contemporary AI models. Autonomous systems that operate in the open world face complex, non-stationary environments wherein enumerating all situations the system may face over the course of its deployment is intractable. Nevertheless, these systems are expected to operate safely and reliably for extended durations. Consequently, AI systems often rely on some degree of human assistance to mitigate risks while completing their tasks, and are hence better treated as semi-autonomous systems. In order to reduce unnecessary reliance on humans and optimize autonomy, we propose a novel introspective planning model—competence-aware systems (CAS)—that enables a semi-autonomous system to reason about its own competence and allowed level of autonomy by leveraging human feedback or assistance. A CAS learns to adjust its level of autonomy based on experience and interactions with a human authority so as to reduce improper reliance on the human and optimize the degree of autonomy it employs in any given circumstance. To handle situations in which the initial CAS model has insufficient state information to properly discriminate feedback received from humans, we introduce a methodology called iterative state space refinement that gradually increases the granularity of the state space online. The approach exploits information that exists in the standard CAS model and requires no additional input from the human. The result is an agent that can more confidently predict the correct feedback from the human authority in each level of autonomy, enabling it learn its competence in a larger portion of the state space.

Resilience-Aware Mixed-Criticality DAG Scheduling on Multi-cores for Autonomous Systems

Fully- and semi-autonomous systems are complex and safety-critical with strict timing and resource constraints, and have a deep processing pipeline with strong dependencies between different functions. Furthermore, tasks with different criticalities share the same hardware, and the scheduling strategy has to guarantee high criticality tasks' execution irrespective of interference from low criticality tasks whilst respecting the precedence constraints among tasks. Most static scheduling work considering task dependencies does not take into account the survivability of low criticality tasks, instead assuming that all low criticality tasks should be suspended or discarded after a mode change. Consequently, the schedules for high and low modes are different, so that more effort is needed to check the safety of schedules during mode change and with a potential increase in the migration cost as tasks may be executed on a different core after a mode change. This work proposes a novel mixed-criticality DAG-based multi-core static scheduling method considering low criticality tasks' survivability and precedence constraints between tasks with different criticalities. This produces a consistent schedule for different system modes enabling task-level mode change and improving the resilience of the system. Furthermore, the utilisation of computational resources is also improved by avoiding discarding low tasks.

The presence of automation enhances deontological considerations in moral judgments

The rapid deployment of semi-autonomous systems (i.e., systems requiring human monitoring such as Uber AVs) poses ethical challenges when these systems face morally-laden situations. We ask how people evaluate morally-laden decisions of humans who monitor these systems in situations of unavoidable harm. We conducted three pre-registered experiments (total N = 1811), using modified trolley moral dilemma scenarios. Our findings suggest that people apply different criteria when judging morality and deserved punishment of regular-car versus AV drivers. Regular-car drivers are judged according to a consequentialist minimizing harm criterion, whereas AV drivers are judged according to whether or not they took action, with a more favorable prior for acting. Integrating judgment and decision-making research with moral psychology, the current research illuminates how the presence versus absence of automation affects moral judgments.

Flying Washer: Development of High-Pressure Washing Aerial Robot Employing Multirotor Platform with Add-On Thrusters

In this study, we propose a multirotor aerial robot for high-pressure washing tasks at high altitudes. The aerial robot consists of a multirotor platform, an add-on planar translational driving system (ATD), a visual sensing system, and a high-pressure washing system. The ATD consists of three ducted fans, which can generate force in all directions on the horizontal plane. The ATD also allows the multirotor to suppress the reaction force generated by the nozzle of a high-pressure washing system and inject water accurately. In this study, we propose a method to precisely inject water by installing an ATD in the multirotor and using its driving force to suppress the reaction force and move the multirotor while keeping its posture horizontal. The semi-autonomous system was designed to allow the operator to maneuver the multirotor while maintaining a constant distance from the wall by the sensor feedback with onboard LiDAR or stereo camera. In the experiment, we succeeded in performing the high-pressure washing task in a real environment and verified that the reaction force generated from the nozzle was actually suppressed during the task.

Unified Meaning Representation Format (UMRF) - A Task Description and Execution Formalism for HRI

To facilitate continuous development of novel HRI systems, it is beneficial to have tools that enable quick adjustments, flexibility, or re-invention of the human interfaces when system requirements change due to updates in the state-of-the-art, application domain, etc. Thus, modularity is a key design principle which promotes software reuse and scalability, and reduces development time and cost. Hence, a robot’s autonomous capabilities should not depend on the command interface and should be decoupled via a common format that possesses the descriptive capabilities for outlining tasks and has a sensible syntax for HRI. In this paper, we propose the Unified Meaning Representation Format (UMRF) , which provides the syntax and semantics for passing both simple and complex commands modelled as control flow graphs. UMRF is a standalone meaning representation container that supports embedding other meaning representation formalisms, such as predicate-argument semantics and graphical meaning representation formats, making it adoptable as a standard task description format for semi-autonomous systems in HRI domains. In this article, we define the UMRF syntax and semantics, summarize its unique aspects relative to related task description formats, and demonstrate its descriptiveness by navigating a robot via concurrent (e.g., gestures and speech) and interchangeable input systems (e.g., Google Assistant, Amazon Alexa).

Participant Characteristics in Human-in-the-Loop Studies with Multiple Unmanned Vehicles including Aircraft

The characteristics that a supervisor of multiple autonomous and semi-autonomous systems should possess remain unclear. Determination of these qualities would support job performance as well as recruiting and training. To evaluate the human characteristics currently being considered by human-in-the-loop experiments, a review of the multiple remote vehicle supervision literature was conducted. The human characteristics addressed included: gender, domain relevant experience, working memory, supervisory relevant skills and abilities (e.g., visual skills, spatial ability, attentional control, vigilance), and traits related to multi-tasking (e.g., stress, resilience). The discussion identifies gaps in the current state of the art with respect to the consideration of human characteristics for multi-autonomous and semi-autonomous systems supervision where at least one vehicle is an aircraft.

The data analysis and validation engine: an application of artificial intelligence in the improvement of COVID-19 data management

Background: The proper management of healthcare data is fundamental to the health system processes; artificial intelligence has proven its value in these processes. Artificial intelligence can simplify the management of information, improve data security, and automate data flow. It is also useful in the analysis and interpretation of big data. Hence, it has the possibility of screening and diagnosing diseases, categorizing disease severity, detecting therapeutic agents, and forecasting outbreak spots.Methods: A data analysis and validation engine was developed to perform data quality control checks, classify addresses, and generate epidemiology numbers using the index and parse command on the command-line interface of DAVE.Results: DAVE correctly formatted data and created a local copy of the datastore and the index. It also returned previous EPID numbers to each entry and assigned a new EPID number to missed entries. DAVE imported the entries into the data template of the existing data management tool and generated a sample manifest that is then sent to the Laboratory. The data flow from the point of collection to storage and reporting was assessed as 100% accurate without errors and in real-time; there was also the ability to roll back if any error occurred.Conclusions: DAVE is a semi-autonomous system that operates with minimal human intervention; it is automatically faster as it leverages computing power to parse, store, and retrieve data while practically eliminating the need for manual data quality assessment. The DAVE functionality can be extended to incorporate additional features like forecasting outbreaks of emerging/re-emerging diseases, categorizing the severity of diseases and analysis of data in our setting.

Verifiable Surface Disinfection Using Ultraviolet Light with a Mobile Manipulation Robot

Robots are being increasingly used in the fight against highly-infectious diseases such as the Novel Coronavirus (SARS-CoV-2). By using robots in place of human health care workers in disinfection tasks, we can reduce the exposure of these workers to the virus and, as a result, often dramatically reduce their risk of infection. Since healthcare workers are often disproportionately affected by large-scale infectious disease outbreaks, this risk reduction can profoundly affect our ability to fight these outbreaks. Many robots currently available for disinfection, however, are little more than mobile platforms for ultraviolet lights, do not allow fine-grained control over how the disinfection is performed, and do not allow verification that it was done as the human supervisor intended. In this paper, we present a semi-autonomous system, originally designed for the disinfection of surfaces in the context of Ebola Virus Disease (EVD) that allows a human supervisor to direct an autonomous robot to disinfect contaminated surfaces to a desired level, and to subsequently verify that this disinfection has taken place. We describe the overall system, the user interface, how our calibration and modeling allows for reliable disinfection, and offer directions for future work to address open space disinfection tasks.