Human-machine Collaboration Research Articles

Computer vision and optical character recognition for the classification of batteries from WEEE

Today, the sorting of battery packs and pouch cells from waste electrical and electronic equipment (WEEE) is performed manually as the batteries vary in shape and weight. Therefore, the feasibility to classify waste batteries by first recognizing text on an image of the battery with deep learning object character recognition (OCR), and then comparing the extracted text with a database of known text information representing battery types is investigated. In addition, the feasibility to sort beyond what is manually viable today is evaluated by examining the text IEC code on lithium-ion batteries (LIBs) to classify by cathode material. Results are promising, with an achieved precision of 0.98 and recall of 0.67 for classification by battery group, and an achieved precision of 0.83 and recall of 0.34 to classify by cathode material, demonstrating the potential for a human-machine collaboration for battery sorting.

Unmanned and Autonomous Systems: Future of Automation in Process and Energy Industries

Process and energy industries have been recognised as adopters of high levels of automation compared to other sectors. Nonetheless, human cognitive input still plays a critical role in the operation of process plants and replication of these cognitive capabilities remains a key challenge for advancing automation levels. In this paper, we provide an analysis of process and energy industries based on a scenario of reduced availability of skilled labour and increased demands for safety, sustainability, and resilience. We consider the different mechanical, sensing, situational awareness, and decision-making tasks involved in the operation of plants and map them to possible realisations of unmanned and autonomous systems. We discuss the implications of current technology capabilities and future technology development perspectives, the factors influencing the complexity of operation in process plants, and the importance of human-machine collaboration. As part of autonomous system capabilities, we consider adaptation as a key capability and we make a connection to adaptation of model-based solutions. We argue that reaching higher and wider levels of autonomy requires a rethink of the design processes for both the physical plants as well as the way automation, control, and safety solutions are conceptualised.

The future and the role of human resource management in South Africa during the Fourth Industrial Revolution

Orientation: The world of work is evolving at an alarming rate, and human resource (HR) practitioners need to familiarise themselves with the future of human resource management (HRM) in order to add value to their organisations.Research purpose: This article presents South African HR practitioners’ views about the future and the role of HRM in the Fourth Industrial Revolution (4IR) from a qualitative perspective.Motivation for the study: Human resource practitioners play a central role in the 4IR, but theories on how their role is enacted remain insufficient.Research approach/design and method: A qualitative survey design was used to study the views of 105 HR practitioners affiliated with the South African Board of People Practices. Three open-ended questions were sent to participants by means of a SurveyMonkey link. Deductive and inductive coding were used to thematically analyse the data.Main findings: The following themes were identified: technology-driven, data-driven, ethically driven, change driven, business-driven, human–machine collaboration and presilience.Practical/managerial implications: South African HR practitioners should be prepared for the future world of work. If these HR practitioners are not technology-driven, data-driven, ethically driven, change driven, business-driven, human–machine collaboration and presilient, they may have difficulty to add value to the organisation in the 4IR.Contribution/value-add: This study extends the body of knowledge about the future world of work and the role of HRM in South Africa by founding that HR practitioners must have presilience and respect ubuntu. The study also extends contemporary scholarship by using an open-ended qualitative review design to investigate the future of HRM in South Africa during the 4IR.

Beyond thought:

The following is a remixed excerpt from a book-length manuscript titled My Life as an Artificial Creative Intelligence. The book is constructed as a theoretical fiction composed by the “author” Mark Amerika in collaboration with a GPT-2 language model. The book was written as an improvisational call-and-response writing performance with an AI text generator and is arranged as a series of intellectual provocations that investigate the creative process across the human-nonhuman spectrum. Critically reflecting on whether or not creativity itself is, at root, a nonhuman information behavior that emerges from an onto-operational presence experiencing an otherworldly aesthetic sensibility, the author playfully engages with his cyberpunk imagination to simultaneously embrace and problematize human-machine collaborations. Contrary to most contemporary AI research that attempts to build AI systems that perform more like humans, Amerika flips the script and, in My Life as an Artificial Creative Intelligence, questions how his own “psychic automatism” is itself a nonhuman function strategically designed to reveal the poetic attributes of programmable worlds still unimagined.

Visual detection and tracking algorithms for minimally invasive surgical instruments: A comprehensive review of the state-of-the-art

Minimally invasive surgical instrument visual detection and tracking is one of the core algorithms of minimally invasive surgical robots. With the development of machine vision and robotics, related technologies such as virtual reality, three-dimensional reconstruction, path planning, and human–machine collaboration can be applied to surgical operations to assist clinicians or use surgical robots to complete clinical operations. The minimally invasive surgical instrument vision detection and tracking algorithm analyzes the image transmitted by the surgical robot endoscope, extracting the position of the surgical instrument tip in the image, so as to provide the surgical navigation. This technology can greatly improve the accuracy and success rate of surgical operations. The purpose of this paper is to further study the visual detection and tracking technology of minimally invasive surgical instruments, summarize the existing research results, and apply it to the surgical robot project. By reading the literature, the author summarized the theoretical basis and related algorithms of this technology in recent years. Finally, the author compares the accuracy, speed and application scenario of each algorithm, and analyzes the advantages and disadvantages of each algorithm. The papers included in the review were selected through Web of Science, Google Scholar, PubMed and CNKI searches using the keywords: “object detection”, “object tracking”, “surgical tool detection”, “surgical tool tracking”, “surgical instrument detection” and “surgical instrument tracking” limiting results to the year range 1985–2021. Our study shows that this technology will have a great development prospect in the aspects of accuracy and real-time improvement in the future.

Trust in automated vehicles

<p id="p00005">Automated driving (AD) is one of the key directions in the intelligent vehicles field. Before full automated driving, we are at the stage of human-machine cooperative driving: Drivers share the driving control with the automated vehicles. Trust in automated vehicles plays a pivotal role in traffic safety and the efficiency of human-machine collaboration. It is vital for drivers to keep an appropriate trust level to avoid accidents. We proposed a dynamic trust framework to elaborate the development of trust and the underlying factors affecting trust. The dynamic trust framework divides the development of trust into four stages: dispositional, initial, ongoing, and post-task trust. Based on the operator characteristics (human), system characteristics (automated driving system), and situation characteristics (environment), the framework identifies potential key factors at each stage and the relation between them. According to the framework, trust calibration can be improved from three approaches: trust monitoring, driver training, and optimizing HMI design. Future research should pay attention to the following four perspectives: the influence of driver and HMI characteristics on trust, the real-time measurement and functional specificity of trust, the mutual trust mechanism between drivers and AD systems, and ways in improving the external validity of trust studies.

Impact of Skill Requirements on Employees’ Thriving at Work: From the Perspective of Artificial Intelligence Embedding

Artificial intelligence applications such as self-driving car, intelligent customer service robots, and intelligent financial robots have brought many conveniences to the public and brought huge benefits to various industries. Therefore, they are rapidly popularized in life and work scenarios. After the introduction of artificial intelligence into the workplace, this article focuses on the impact of skill requirements with creativity and human-machine collaboration ability on employees’ thriving at work. We believe that in the artificial intelligence application scenario, skill requirements can help employees free themselves from repetitive and complicated work, so that they can better appreciate the value, significance and achievement of work, and obtain a happy psychological experience of job competence, which can stimulate their thriving at work. Through the empirical test, we find that skill requirements triggered by artificial intelligence can enhance employees’ sense of competence and enhance their sense of work vigor. Moreover, the higher the cooperation between employees and intelligent machines is, the higher the human-computer cooperation relationship is. Skill demands will make employees feel a stronger sense of competence, and their energy will be more vigorous. The application of artificial intelligence technology crafts the psychology of employees by changing skill requirements.The research has the following theoretical contributions. First, we introduce artificial intelligence into the field of employee psychology and behavior, and find that in artificial intelligence application scenarios, the increase in skills requirements triggers positive changes of employees’ work enthusiasm, which enriches the research on the relationship between skills requirements and thriving at work. Second, we reveal the inner psychological mechanism of skill requirements affecting thriving at work. In view of the fact that the introduction of artificial intelligence into enterprises has led to an increase in skill requirements, this article finds that skill requirements affect the need for competence, which in turn affects the sense of work vigor, which opens up the internal black box where skill requirements affect the sense of work vigor. Third, from the perspective of human-computer cooperation relationship, the analysis of the boundary conditions that play a role between skill requirements and employees’ work enthusiasm has enriched the research on thriving at work.In short, this article introduces artificial intelligence into the field of organizational psychology, enriches the research on the relationship between skill requirements and employees’ thriving at work, and also provides inspiration for companies to rationally stimulate employees’ thriving at work.

On trajectory tracking control of fluid-driven actuators

Abstract Fluid-driven actuators are not only well-established in automation, but also a promising drive technology for collaborative robots. Their inherent compliance due to the compressibility of suitable fluids such as air, as well as their direct drive properties are advantageous safety features for human-machine collaboration. In this work, we provide an overview of different fluid-driven manipulators, namely fluidic muscle actuated ones, continuum manipulators, and those with rotary joints. For the latter, we introduce the mathematical model including mechanics and pressure dynamics and describe its properties such as strong nonlinearities, which make trajectory tracking control challenging. A model-based nonlinear cascaded controller is presented. Experimental results on a 6 degrees of freedom (DOF) prototype demonstrate the resulting trajectory tracking performance.

Design of Personal Trajectories for Employees’ Professional Development in the Knowledge Society under Industry 5.0

The main feature of Industry 5.0 is “personalization”, linked not only to provide customers with personalized products, but also, in our opinion, to ensure personalization in labor relations with employees, since it increases human value through human–machine collaboration. The human capital quality determines a significant contribution not only to the labor productivity growth, but also to extend a social communication, loyalty and employees’ trust. The study proposes the new methodological approach for corporate human capital assessment and management (CHCM) over new conditions of digital transformation. The CHCM uses methods of system analysis and synthesis, expert assessments, descriptive statistical analysis and survey. The novelty of CHCM is that, firstly, it reflects all the essential features and properties of human capital under emergence of new professions; secondly, it combines and comprehensively uses both quantitative and qualitative methods for human capital assessment, reflecting the subjective and objective aspects of human capital measurement; thirdly, it allows to create warranted management decisions about individual trajectories of professional development of employees, ensuring the continuous growth of individual, corporate and social wealth. It is proved experimentally that the implementation of individual trajectories for employees’ professional development provides 2–3 years’ perspective on companies’ performance growth.

Estimation of Joint Angle From sEMG and Inertial Measurements Based on Deep Learning Approach.

Continuous kinematics estimation from surface electromyography (sEMG) allows more natural and intuitive human-machine collaboration. Recent research has suggested the use of multimodal inputs (sEMG signals and inertial measurements) to improve estimation performance. This work focused on assessing the use of angular velocity in combination with myoelectric signals to simultaneously and continuously predict 12 joint angles in the hand. Estimation performance was evaluated for five functional and grasping movements in 20 subjects. The proposed method is based on convolutional and recurrent neural networks using transfer learning (TL). A novel aspect was the use of a pretrained deep network model from basic joint hand movements to learn new patterns present in functional motions. A comparison was carried out with the traditional method based solely on sEMG. Although the performance of the algorithm slightly improved with the use of the multimodal combination, both strategies had similar behavior. The results indicated a significant improvement for a single task: opening a bottle with a tripod grasp.

Framework for collaborative intelligence in forecasting day-ahead electricity price

Electricity price forecasting in wholesale markets is an essential asset for deciding bidding strategies and operational schedules. The decision making process is limited if no understanding is given on how and why such electricity price points have been forecast. The present article proposes a novel framework that promotes human–machine collaboration in forecasting day-ahead electricity price in wholesale markets. The framework is based on a new model architecture that uses a plethora of statistical and machine learning models, a wide range of exogenous features, a combination of several time series decomposition methods and a collection of time series characteristics based on signal processing and time series analysis methods. The model architecture is supported by open-source automated machine learning platforms that provide a baseline reference used for comparison purposes. The objective of the framework is not only to provide forecasts, but to promote a human-in-the-loop approach by providing a data story based on a collection of model-agnostic methods aimed at interpreting the mechanisms and behavior of the new model architecture and its predictions. The framework has been applied to the Spanish wholesale market. The forecasting results show good accuracy on mean absolute error (1.859, 95% HDI [0.575, 3.924] EUR(MWh)−1) and mean absolute scaled error (0.378, 95% HDI [0.091, 0.934]). Moreover, the framework demonstrates its human-centric capabilities by providing graphical and numeric explanations that augments understanding on the model and its electricity price point forecasts.

Assessing Open-Ended Human-Computer Collaboration Systems: Applying a Hallmarks Approach.

There is a growing desire to create computer systems that can collaborate with humans on complex, open-ended activities. These activities typically have no set completion criteria and frequently involve multimodal communication, extensive world knowledge, creativity, and building structures or compositions through multiple steps. Because these systems differ from question and answer (Q&A) systems, chatbots, and simple task-oriented assistants, new methods for evaluating such collaborative computer systems are needed. Here, we present a set of criteria for evaluating these systems, called Hallmarks of Human-Machine Collaboration. The Hallmarks build on the success of heuristic evaluation used by the user interface community and past evaluation techniques used in the spoken language and chatbot communities. They consist of observable characteristics indicative of successful collaborative communication, grouped into eight high-level properties: robustness; habitability; mutual contribution of meaningful content; context-awareness; consistent human engagement; provision of rationale; use of elementary concepts to teach and learn new concepts; and successful collaboration. We present examples of how we used these Hallmarks in the DARPA Communicating with Computers (CwC) program to evaluate diverse activities, including story and music generation, interactive building with blocks, and exploration of molecular mechanisms in cancer. We used the Hallmarks as guides for developers and as diagnostics, assessing systems with the Hallmarks to identify strengths and opportunities for improvement using logs from user studies, surveying the human partner, third-party review of creative products, and direct tests. Informal feedback from CwC technology developers indicates that the use of the Hallmarks for program evaluation helped guide development. The Hallmarks also made it possible to identify areas of progress and major gaps in developing systems where the machine is an equal, creative partner.

Enhancing human-machine teaming for medical prognosis through neural ordinary differential equations (NODEs)

Machine learning (ML) has recently been demonstrated to rival expert-level human accuracy in prediction and detection tasks in a variety of domains, including medicine. Despite these impressive findings, however, a key barrier to the full realization of ML’s potential in medical prognoses is technology acceptance. Recent efforts to produce explainable AI (XAI) have made progress in improving the interpretability of some ML models, but these efforts suffer from limitations intrinsic to their design: they work best at identifying why a system fails, but do poorly at explaining when and why a model’s prediction is correct. We posit that the acceptability of ML predictions in expert domains is limited by two key factors: the machine’s horizon of prediction that extends beyond human capability, and the inability for machine predictions to incorporate human intuition into their models. We propose the use of a novel ML architecture, Neural Ordinary Differential Equations (NODEs) to enhance human understanding and encourage acceptability. Our approach prioritizes human cognitive intuition at the center of the algorithm design, and offers a distribution of predictions rather than single outputs. We explain how this approach may significantly improve human-machine collaboration in prediction tasks in expert domains such as medical prognoses. We propose a model and demonstrate, by expanding a concrete example from the literature, how our model advances the vision of future hybrid human-AI systems.

The neural resource allocation problem when enhancing human bodies with extra robotic limbs

The emergence of robotic body augmentation provides exciting innovations that will revolutionize the fields of robotics, human–machine interaction and wearable electronics. Although augmentative devices such as extra robotic arms and fingers are informed by restorative technologies in many ways, they also introduce unique challenges for bidirectional human–machine collaboration. Can humans adapt and learn to operate a new robotic limb collaboratively with their biological limbs, without restricting other physical abilities? To successfully achieve robotic body augmentation, we need to ensure that, by giving a user an additional (artificial) limb, we are not trading off the functionalities of an existing (biological) one. Here, we introduce the ‘neural resource allocation problem’ and discuss how to allow the effective voluntary control of augmentative devices without compromising control of the biological body. In reviewing the relevant literature on extra robotic fingers and arms, we critically assess the range of potential solutions available for this neural resource allocation problem. For this purpose, we combine multiple perspectives from engineering and neuroscience with considerations including human–machine interaction, sensory–motor integration, ethics and law. In summary, we aim to define common foundations and operating principles for the successful implementation of robotic body augmentation. The development of extra fingers and arms is an exciting research area in robotics, human–machine interaction and wearable electronics. It is unclear, however, whether humans can adapt and learn to control extra limbs and integrate them into a new sensorimotor representation, without sacrificing their natural abilities. The authors review this topic and describe challenges in allocating neural resources for robotic body augmentation.

ARTIFICIAL INTELLIGENCE (AI) CONTRIBUTIONS TOWARDS BUSINESS INNOVATIONS

We are in the era of human-machine collaboration. Artificial Intelligence, Business Analytics and Internet of Things are the buzzwords. Machines have started simulating ‘human intelligence’ and top companies in the world have started focusing and investing their R&amp;D efforts on Machine learning and Artificial Intelligence. Irrespective of the Industry, Artificial Intelligence is employed extensively as it increases efficiency, reduces cost and solves complex problems in no time. Gartner in its press release on Global Artificial Intelligence dated April 25, 2018 forecasts the business value derived from AI will reach $3.9 trillion in 2022. The report also states that Customer Experience, Cost Reduction and New Revenue Generation are the three major sources of business value delivered by AI. Artificial Intelligence is predicted to be one of the most promising technologies of the future. AI emerging as one of the prominent technologies is driving innovations in various business sectors. In this paper, the evolution of AI, its role in driving innovations in five business sectors and the predicted future of this technology is discussed from Industrial Perspective.

Interpreting quality assessment re-imagined: The synergy between human and machine scoring

Assessment of interpreting quality is a ubiquitous social practice in the interpreting industry and academia. In this article, we focus on both psychometric and social dimensions of assessment practice, and analyse two major assessment paradigms, namely, human rater scoring and automatic machine scoring. Regarding human scoring, we describe five specific methods, including atomistic scoring, questionnaire-based scoring, multi-methods scoring, rubric scoring, and ranking, and critically analyse their respective strengths and weaknesses. In terms of automatic scoring, we highlight four assessment approaches that have been researched and operationalised in cognate disciplines and interpreting studies, including automatic assessment based on temporal variables, linguistic/surface features, machine translation metrics, and quality estimation methodology. Finally, we problematise the socio-technological tension between these two paradigms and envisage human–machine collaboration to produce psychometrically sound and socially responsible assessment. We hope that this article sparks more scholarly discussion of rater-mediated and automatic assessment of interpreting quality from a psychometric-social perspective.

The intelligent, experiential and competitive workplace: Part 1

This two-part paper explores how intelligent automation and the convergence of accelerating technology advancements will shape the future of work and transform the workforce and the workplace of the future. Part 1 examines key intelligent automation advances and challenges with an overview of the emerging workforce and workplace models. Part 2 assesses the impact of intelligent automation and artificial intelligence (AI) upon the workforce and the workplace in greater depth, as well as the societal impacts to consider as these advancing technologies transform business, society and life itself. The paper begins with the premise that the business world is at a major inflection point whereby more businesses than not have completed their first phase of digital transformation. The COVID-19 pandemic accelerated AI, robotics, workplace automation and digital transformation initiatives that were already well underway. Trends that have been gathering momentum for years, such as workplace mobility and diverse ‘hybrid workplace’ behaviours, have rapidly gained adoption to now become mainstream. Just as the office is becoming an ecosystem of workplace options, the workforce is becoming increasingly ‘liquid’ and distributed; the ‘human cloud’ continues to evolve as many organisations turn to contract, on-demand, highly flexible and elastic labour models. Digital workplace technologies — from meeting solution software to enterprise chat platforms and desktop-as-a-service — have enabled the adoption of remote working and creation of workplace ecosystems inclusive of flexible ‘hybrid’ workplaces that can accommodate working in the office, at home or anywhere. As the post-digital era advances, the convergence of AI, robotics, workplace automation and virtual/augmented/extended reality (VR/AR/XR) technologies and 5G mobile networks will enable completely new ways of working and accelerate societal transformation. Digital technologies will enable rich, immersive and distributed virtual collaboration that will power new levels of human performance. The next phase of digital transformation will be driven by businesses willing to make AI investments to improve their competitive advantage. Over the next decade, AI will offer employees unprecedented information awareness and insight, providing greater freedom from low-value-add activities and the ability to easily adopt and use these emerging complex technologies. In the future of work, AI and immersive XR technologies will lead to greater levels of human–machine collaboration; however, policymakers, public and private organisations will need to address the risks and challenges of an increasingly AI-enabled digital world. New ways of working will offer the promise of unlocking greater human potential and may lead to some worker displacement, as well as intensifying the demand for greater workforce reskilling and continuous lifelong learning. Increasingly sophisticated AI applications, including facial recognition and deep learning neural networks, will provide new insights to address complex business problems and societal challenges. These very same advanced AI applications will also raise difficult questions regarding transparency, ethics, equity and privacy.

Anno-Mate: Human–Machine Collaboration Features for Fast Annotation

Large annotated datasets are crucial for training deep machine learning models, but they are expensive and time-consuming to create. There are already numerous public datasets, but a vast amount of unlabeled data, especially video data, can still be annotated and leveraged to further improve the performance and accuracy of machine learning models. Therefore, it is essential to reduce the time and effort required to annotate a dataset to prevent bottlenecks in the development of this field. In this study, we propose Anno-Mate, a pair of features integrated into the Computer Vision Annotation Tool (CVAT). It facilitates human–machine collaboration and reduces the required human effort. Anno-Mate comprises Auto-Fit, which uses an EfficientDet-D0 backbone to tighten an existing bounding box around an object, and AutoTrack, which uses a channel and spatial reliability tracking (CSRT) tracker to draw a bounding box on the target object as it moves through the video frames. Both features exhibit a good speed and accuracy trade-off. Auto-Fit garnered an overall accuracy of 87% and an average processing time of 0.47 s, whereas the AutoTrack feature exhibited an overall accuracy of 74.29% and could process 18.54 frames per second. When combined, these features are proven to reduce the time required to annotate a minute of video by 26.56%.

Research progress and development trend of surgical robot and surgical instrument arm.

In recent years, surgical robots have become an indispensable part of the medical field. Surgical robots are increasingly being used in the areas of gynaecological surgery, urological surgery, orthopaedic surgery, general surgery and so forth. In this paper, the development of surgical robots in different operations is reviewed and analysed. In the type of master-slave surgical robotic system, the robotic surgical instrument arms were located in the execution terminal of a surgical robot system, as one of the core components, and directly contact with the patient during the operation, which plays an important role in the efficiency and safety of the operation. In clinical, the arm function and design in different systems varies. Furtherly, the current research progress of robotic surgical instrument arms used in different operations is analysed and summarised. Finally, the challenge and trend are concluded. According to the classification of surgical types, the development of surgical robots for laparoscopic surgery, neurosurgery, orthopaedics and microsurgery are analysed and summarised. Then, focusing on the research of robotic surgical instrument arms, according to structure type, the research and application of straight-rod surgical instrument arm, joint surgical instrument arm and continuous surgical instrument arm are analysed respectively. According to the discussion and summary of the characteristics of the existing surgical robots and instrument arms, it is concluded that they still have a lot of room for development in the future. Therefore, the development trends of the surgical robot and instrument arm are discussed and analysed in the five aspects of structural materials, modularisation, telemedicine, intelligence and human-machine collaboration. Surgical robots have shown the development trend of miniaturisation, intelligence, autonomy and dexterity. Thereby, in the field of science and technology, the research on the next generation of minimally invasive surgical robots will usher in a peak period of development.

A Bibliometric Analysis of Human-Machine Interaction Methodology for Electric-Powered Wheelchairs Driving from 1998 to 2020.

Electric power wheelchairs (EPWs) enhance the mobility capability of the elderly and the disabled, while the human-machine interaction (HMI) determines how well the human intention will be precisely delivered and how human-machine system cooperation will be efficiently conducted. A bibliometric quantitative analysis of 1154 publications related to this research field, published between 1998 and 2020, was conducted. We identified the development status, contributors, hot topics, and potential future research directions of this field. We believe that the combination of intelligence and humanization of an EPW HMI system based on human-machine collaboration is an emerging trend in EPW HMI methodology research. Particular attention should be paid to evaluating the applicability and benefits of the EPW HMI methodology for the users, as well as how much it contributes to society. This study offers researchers a comprehensive understanding of EPW HMI studies in the past 22 years and latest trends from the evolutionary footprints and forward-thinking insights regarding future research.