Industrial Manipulator Research Articles

New Design Potentials of Non-mimetic Sonification in Human–Robot Interaction

With the increasing use and complexity of robotic devices, the requirements for the design of human–robot interfaces are rapidly changing and call for new means of interaction and information transfer. On that scope, the discussed project—being developed by the Hybrid Things Lab at the University of Applied Sciences Augsburg and the Design Research Lab at Bauhaus-Universität Weimar—takes a first step in characterizing a novel field of research, exploring the design potentials of non-mimetic sonification in the context of human–robot interaction. Featuring an industrial seven-axis manipulator and collecting multiple information (for instance, the position of the end-effector, joint positions and forces) during manipulation, these datasets are being used for creating a novel augmented audible presence and thus allowing new forms of interaction. As such, this article considers (1) research parameters for non-mimetic sonification (such as pitch, volume, and timbre); (2) a comprehensive empirical pursuit, including setup, exploration, and validation; (3) the overall implications of integrating these findings into a unifying human–robot interaction process. The relation between machinic and auditory dimensionality is of particular concern.

Generalized algorithm for designing the mechanical part of an industrial robot manipulator

The paper presents a method that allows you to design and calculate the design of an industrial robot manipulator using modern computer systems (CAD, CAE). Due to the versatility, the scope of application of this method can be different from the design of turntables to flexible production complexes. But the main focus of the method is the design of an industrial robot manipulator. The paper presents an example of the design and calculation of an in-dustrial robot manipulator, which will later be manufactured in a single copy.

Evolutionary Optimization of Fuzzy Reinforcement Learning and Its Application to Time-Varying Tracking Control of Industrial Parallel Robotic Manipulators

This paper contributes to the development of evolutionary optimization of fuzzy reinforcement learning and its application to time-varying tracking control for industrial parallel robotic manipulators. An improved evolutionary social spider optimization (SSO) paradigm with Cauchy mutation is presented to address the optimal fuzzy reinforcement learning problem. The SSO swarm intelligence incorporated with the fuzzy Q-learning strategy, called SSOFQ, is applied to the time-varying tracking control of parallel robotic Stewart manipulators with six degrees of freedom (DOFs). Having the derived kinematics and dynamics of six-DOF Stewart platforms, the SSOFQ is employed to synthesize an intelligent real-time tracking control scheme. More importantly, a custom experimental Stewart platform is constructed using hardware/software codesign and the system-on-a- programmable chip (SoPC) technique. Finally, the simulations, experimental results and comparative works are provided to validate the efficacy and superiority of the presented SSOFQ control method.

IMPLEMENTATION OF A PROTOTYPE FOR CONTROLLING VARIOUS INFORMATION SYSTEMS AND DEVICES USING GESTURE CONTROL

The article provides a brief description of the principles of operation of a prototype device, referred to at the time of development as Glove's Keyboard (GloK), the main function of which is to interact with a wide range of computing equipment (from personal computers to industrial manipulators) and to combine the most common HID devices, such as: mouse, keyboard and joystick. The prototype device is a glove equipped with gyroscope-accelerometer sensors capable of detecting finger positions. The developed cross-platform software provides the ability to conduct communication sessions with devices, obtain data and visualize them using 3D graphics.

Two-Phase Non-Singular Terminal Sliding Mode Control of Nonlinear Systems

This paper presents a two-phase non-singular terminal sliding mode control scheme for a class of nonlinear systems in the presence of external disturbances. A two-phase terminal sliding mode surface is constructed and utilized, such that our control scheme achieves fast finite-duration convergence in regions both close to and far away from the control objective, by which the transition state is set and a less conservative bound of the settling time is obtained and superior to the existing results. Meanwhile, the singularity is avoided by the utilization of the saturation function, with the simplicity of structure and implementation. Extensions to multi-input multi-output systems are carried out further. The numerical simulation of the single inverted pendulum and the industrial manipulator are conducted to verify the effectiveness of the proposed control strategy.

Bistable Stopper Design and Force Prediction for Precision and Power Grasps of Soft Robotic Fingers for Industrial Manipulation

Abstract This article aims at presenting a detailed and practical comparison between three designs of robotic soft fingers for industrial grippers. While the soft finger based on the fin ray effect (FRE) shown here has been proposed long ago, few works in the literature have studied its reliance on the presence of the crossbeams or its precision grasp performance compared to its power grasps. Aiming at addressing these gaps, two novel designs are proposed and compared to FRE fingers in this article. First, the three designs are presented. One of these fingers, named PacomeFlex, embeds changeable grasping modes by relying on two sets of kinematic structures with a bistable stopper design. Then, finite element analyses (FEA) are conducted to simulate power and precision grasps of the three fingers followed by the estimation of the grasp forces produced. These FEA are then used to train neural networks capable of predicting these grasp forces. Finally, the grasp strength and pullout resistance of the fingers are experimentally measured, and experimental results are shown to be in good accordance with the FEA and neural network models. As will also be shown, the PacomeFlex finger introduced in this work provides a noticeably higher performance level than all other fingers with respect to typical metrics in soft grasping.

Self-training based augmented reality for robust 3D object registration and task assistance

Typical augmented reality (AR) technologies superimpose virtual information onto the real environment for task guidance and assistance by tracking fiducial markers. However, these have narrow recognition ranges due to the markers, and it is difficult to track markers when occlusion occurs. Therefore, it is challenging to seamlessly register real objects in AR scenes captured by the camera without requiring fiducial markers. Deep learning–based approaches to 3D object registration and task assistance have recently indicated promising directions for superimposing 3D virtual objects onto corresponding real objects or their surroundings seamlessly. However, since supervised learning-based object registration requires many pose annotations on images for the locations and orientations of real objects, this process is a time-consuming and labor-intensive task. Thus, this paper proposes a new self-training based augmented reality method for seamless real object registration and task assistance without time-consuming pose annotations. As the industrial components or objects have their 3D geometric models, the proposed approach utilizes them as their synthetic data. The automatically generated synthetic dataset is used for training the deep learning model, which outputs a colored coordinate map of objects to be detected. 2D–3D key point matching between the 2D colored coordinate map and 3D colored coordinate model is then performed for 3D object registration in an AR environment. In particular, self-training is performed for fine-tuning the deep learning model by generating pseudo-labels with the highest probability for mask intersection over union (IoU) in the unlabeled real object dataset. We compared the proposed approach with previous studies to confirm its effectiveness and advantages. Also, we performed ablation studies by training a custom dataset based on various industrial 3D virtual models. Finally, we present AR-based task guidance and assistance, such as industrial robot manipulation.

Collision Avoidance for a Selective Compliance Assembly Robot Arm Manipulator Using Topological Path Planning

Industrial applications with robotic manipulators have grown and made production systems increasingly efficient. However, there are still some limitations that can delay production, causing losses. Several factors, such as accidents and collisions of manipulator robots with operators and other machines, can cause unforeseen stops. Thus, this work aims to develop a trajectory planning method to avoid collisions applied to a selective compliance assembly robot arm (SCARA) robotic manipulator in the context of collaborative robotics. The main contribution of this paper is a path planning method based on mathematical morphology, named topological path planning (TPP). Through some evaluation metrics such as the number of path points, computing time, distance, standard deviation of the joint acceleration, and maximum acceleration rate along the path, we show that TPP is a collision-free, deterministic, and predictable route planning. In our experiments, our proposal presented better results for applications in industrial robotic manipulators when compared to the probabilistic roadmap method (PRM) and TPP*, a particular case of TPP that is similar to the generalized Voronoi diagram (GVD).

Bots2ReC – Analysis of Key Findings for the Application Development of Semi-Autonomous Asbestos Removal

The removal of asbestos from existing buildings is a socially important and challenging renovation process due to the handling of hazardous materials. The EU project Robots to ReConstruction (Bots2ReC) has developed a robotic system for semi-autonomous asbestos removal for apartment buildings. The removal of plaster and tiles and the associated tile adhesive is implemented as a semi-autonomous construction task by mobile manipulators and integrated into the existing construction process. The development is based on a test-driven approach in order to obtain information about the achievable performance and the acceptance of the human operators as early as possible. The identification of the control variables for the respective tools and the embedding in the on-site process flow are named as the main reasons for the test-driven approach. The analysis of selected key findings of the project shows for the developed semantic mapping via mobile robot that a minimal, application-specific data format favours an efficient overall process control via task planning up to execution. In the development of mobile bases for the application, it can be shown that a holonomic drive is preferable to a differential drive due to the partly narrow and angled parts of the building. To carry out the intended processes and to guide the automated power tools, the in-house development of serial manipulators is necessary, since the combination of the required working space with the occurring weight and process forces cannot be solved within the requirements by an industrial manipulator. Finally, the main benefit of this papers contribution is seen in the transferable outcomes to other on-site automation projects.

Context-Aware Robotic Assistive System: Robotic Pointing Gesture-Based Assistance for People with Disabilities in Sheltered Workshops

People with disabilities are severely underrepresented in the open labor market. Yet, pursuing a job has a positive impact in many aspects of life. This paper presents a possible approach to improve inclusion by including a robotic manipulator into context-aware Assistive Systems. This expands the assistance possibilities tremendously by adding gesture-based feedback and aid. The system presented is based on the intelligent control system of behavior trees, which—together with a depth camera, specifically designed policies, and a collaborative industrial robotic manipulator—can assist workers with disabilities in the workplace. A developed assistance node generates personalized action sequences. These include different robotic pointing gestures, from simple waving, to precisely indicating the target position of the workpiece during assembly tasks. This paper describes the design challenges and technical implementation of the first Context-Aware Robotic Assistive System. Moreover, an in-field user study in a Sheltered Workshop was performed to verify the concept and developed algorithms. In the assembly task under consideration, almost three times as many parts could be assembled with the developed system than with the baseline condition. In addition, the reactions and statements of the participants showed that the robot was considered and accepted as a tutor.

Mobile Manipulators in Industry 4.0: A Review of Developments for Industrial Applications.

In the realm of Industry 4.0, diverse technologies such as AI, Cyber-Physical Systems, IoT, and advanced sensors converge to shape smarter future factories. Mobile manipulators (MMs) are pivotal, fostering flexibility, adaptability, and collaboration in industrial processes. On one hand, MMs offer a remarkable level of flexibility, adaptability, and collaboration in industrial processes, facilitating swift production line changes and efficiency enhancements. On the other hand, their integration into real manufacturing environments requires meticulous considerations, such as safety, human-robot interaction, and cybersecurity. This article delves into MMs' essential role in achieving Industry 4.0's automation and adaptability by integrating mobility with manipulation capabilities. The study reviews MMs' industrial applications and integration into manufacturing systems. The most observed applications are logistics (49%) and manufacturing (33%). As Industry 4.0 advances, the paper emphasizes updating and aligning MMs with the smart factory concept by networks of sensors and the real-time analysis of them, especially for an enhanced human-robot interaction. Another objective is categorizing considerations for MMs' utilization in Industry 4.0-aligned manufacturing. This review methodically covers a wide range of considerations and evaluates existing solutions. It shows a more comprehensive approach to understanding MMs in Industry 4.0 than previous works. Key focus areas encompass perception, data analysis, connectivity, human-robot interaction, safety, virtualization, and cybersecurity. By bringing together different aspects, this research emphasizes a more integrated view of the role and challenges of MMs in the Industry 4.0 paradigm and provides insights into aspects often overlooked. A detailed and synthetic analysis of existing knowledge was performed, and insights into their future path in Industry 4.0 environments were provided as part of the contributions of this paper. The article also appraises initiatives in these domains, along with a succinct technology readiness analysis. To sum up, this study highlights MMs' pivotal role in Industry 4.0, encompassing their influence on adaptability, automation, and efficiency.

SISG-Net: Simultaneous instance segmentation and grasp detection for robot grasp in clutter

Robots have always found it challenging to grasp in cluttered scenes because of the complex background information and changing operating environment. Therefore, in order to enable robots to perform multi-object grasping tasks in a wider range of application scenarios, such as object sorting on industrial production lines and object manipulation by home service robots, we innovatively integrated segmentation and grasp detection into the same framework, and designed a simultaneous instance segmentation and grasp detection network (SISG-Net). By using the network, robots can better interact with complex environments and perform grasp tasks. In order to solve the problem of insufficient fusion of the existing RGBD fusion strategy in the robot field, we propose a lightweight RGB-D fusion module called SMCF to make modal fusion more efficient. In order to solve the problem of inaccurate perception of small objects in different scenes, we propose the FFASP module. Finally, we use the AFF module to adaptively fuse multi-scale features. Segmentation can remove noise information from the background, enabling the robot to grasp in different backgrounds with robustness. Using the segmentation result, we refine grasp detection and find the best grasp pose for robot grasping in complex scenes. Our grasp detection model performs similarly to state-of-the-art grasp detection algorithms on the Cornell Dataset. Our model achieves state-of-the-art performance on the OCID Dataset. We show that the method is stable and robust in real-world grasping experiments. The code and video of our experiment used in this paper can be found at: https://github.com/meiguiz/SISG-Net.

Adaptive technique for physical human–robot interaction handling using proprioceptive sensors

The work focuses on the development of an adaptive technique for the physical interaction handling between a human and a robot, as well as its experimental validation. The proposed technique is based on the deep residual neural network and dedicated finite state machine, where the states are the robot behavior modes and transitions are the switchings between the states that depend on the interaction parameters and characteristics. It ensures the human operator safety and improves the human–robot collaboration performance by implementing various scenarios. In the scope of this technique, the parameters of human–robot interaction are used to select an appropriate robot reaction strategy using data from internal robot sensors only, i.e. proprioceptive sensors. These parameters define the interaction force vector and its application point on the robot surface, which allow to classify the interaction within the set of predefined categories. This classification distinguishes interactions applied at the tool or intermediate link (Tool/Link), having soft or hard nature (Soft/Hard), as well as having different intention (Intl/Accd) or duration (Short/Long). Based on identified category and the current robot state, the algorithm chooses an appropriate robot reaction. To confirm the efficiency the developed technique, an experimental study was conducted, which involved the collaboration between the real industrial manipulator KUKA LBR iiwa and the human operator.

Pieces Separation Applied to a RV-2AJ Industrial Manipulator

The objective of this work is to create a system for separating types of pieces for didactic purposes. An artificial vision system applied to a manipulator was used to perform this task of classifying and separating the pieces, through the use of digital image processing. The vision system uses a camera attached to the robot's end-effector, whose objective is to identify the area, perimeter and position of the centroid of the pieces to be separated. The separation of objects is performed between two classes: with round and non-round shapes. The tests were carried out with objects such as screws and nuts. After obtaining the positions of the objects to be separated by the vision system, some characteristics of these related to the reference system of the robot base are determined. Some difficulties were observed in the recognition of objects related to the colours and lighting conditions of the separation environment. To reduce these problems, black colour objects were used in order to reduce the reflectance. Another problem encountered in the separation process is related to the relative position of objects. In this case, two or more objects can be identified as a single object, depending on their relative positions. The system classifies with a rate of hits greater than 95%.

‘Odds Are: They Win’: a disruptive messaging innovation for challenging harmful products and practices of the gambling industry

ObjectiveThis paper presents an evidence informed rationale for focussing on harmful gambling products and industry practices in public health messaging through the example of a recent innovation called ‘Odds Are: They Win’. Methods‘Odds Are: They Win’ was initially developed through coproduction involving public health professionals and people with lived experience of gambling harms and implemented across a city-region area. A review of relevant evidence was undertaken, upon which the research team reflected to draw out the implications of ‘Odds Are: They Win’ for gambling harms messaging. ResultsEvidence is mounting that safer gambling campaigns framed in terms of individual responsibility are ineffective and can generate stigma. ‘Odds Are: They Win’ presents an alternative focus that is not anti-gambling but raises awareness of industry manipulation of the situational and structural context of gambling. This is in-keeping with historical lessons from the stop smoking field and emerging research in critical health literacy. The latter highlights the potential of education on the social and commercial determinants of health to stimulate behaviour change and collective action. Conclusion‘Odds Are: They Win’ is a potentially disruptive innovation for the gambling harms field. Research is required to robustly evaluate this intervention across diverse criteria, target audiences, and delivery settings.

High-Precision Control of Industrial Robot Manipulator Based on Extended Flexible Joint Model

High-precision industrial manipulators are essential components in advanced manufacturing. Model-based feedforward is the key to realizing the high-precision control of industrial robot manipulators. However, traditional feedforward control approaches are based on rigid models or flexible joint models which neglect the elasticities out of the rotational directions and degrade the setpoint precision significantly. To eliminate the effects of elasticities in all directions, a high-precision setpoint feedforward control method is proposed based on the output redefinition of the extended flexible joint model (EFJM). First, the flexible industrial robots are modeled by the EFJM to describe the elasticities in joint rotational directions and out of the rotational directions. Second, the nonminimum-phase EFJM is transformed into a minimum-phase system by using output redefinition. Third, the setpoint control task is transformed from Cartesian space into joint space by trajectory planning based on the EFJM. Third, a universal recursive algorithm is designed to compute the feedforward torque based on the EFJM. Moreover, the computational performance is improved. By compensating the pose errors caused by elasticities in all directions, the proposed method can effectively improve the setpoint control precision. The effectiveness of the proposed method is illustrated by simulation and experimental studies. The experimental results show that the proposed method reduces position errors by more than 65% and the orientation errors by more than 62%.

Inverted Pendulum Control: A Comparative Study from Conventional Control to Reinforcement Learning

The rise of deep reinforcement learning in recent years has led to its usage in solving various challenging problems, such as chess and Go games. However, despite its recent success in solving highly complex problems, a question arises on whether this class of method is best employed to solve control problems in general, such as driverless cars, mobile robot control, or industrial manipulator control. This paper presents a comparative study between various classes of control algorithms and reinforcement learning in controlling an inverted pendulum system to evaluate the performance of reinforcement learning in a control problem. A test was performed to test the performance of root locus-based control, state compensator control, proportional-derivative (PD) control, and a reinforcement learning method, namely the proximal policy optimization (PPO), to control an inverted pendulum on a cart. The performances of the transient responses (such as overshoot, peak time, and settling time) and the steady-state responses (namely steady-state error and the total energy) were compared. It is found that when given a sufficient amount of training, the reinforcement learning algorithm was able to produce a comparable solution to its control algorithm counterparts despite not knowing anything about the system’s properties. Therefore, it is best used to control plants with little to no information regarding the model where testing a particular policy is easy and safe. It is also recommended for a system with a clear objective function.

Trajectory planning and control algorithm of industrial robot manipulator

As the industrial robot task becomes more complex, the difficulty of trajectory planning and tracking control of manipulator is gradually increasing. To minimize the vibration during the manipulator motion and improve the planning accuracy, the method of quintic polynomial combined with non-uniform B-spline interpolation is studied for joint space (JS) planning. The trajectory tracking system is easily affected by friction nonlinearity and parameters. So a JS trajectory tracking controller based on based on fuzzy neural network (FNN) is designed. Through simulation experiments, the curve obtained by the planning method studied is smoother and the planning error is minimum. The maximum position error is 0.09 rad, and the speed error is not more than 0.1 rad/s. The controller performance test results under different parameters show that the W^, c^, κ^ parameter in FNN can be adjusted in real time, and the value will not affect the performance of the controller. The fluctuation range of trajectory error of different joints is within ±0.2×10-5rad, which indicates that the performance of AFNNC controller studied is better. And its response time is the shortest and its robustness is better when the load changes suddenly.

A robotized 6-DOF dry test rig for wave power

Wave power has the potential to contribute significantly to sustainability by reducing our global dependence on fossil fuels. Due to harsh conditions and high costs associated with offshore testing, lab experiments are favourable for resource-efficient validation and optimization in developing Wave Energy Converter (WEC) technologies. The limited scale and availability of existing wave tanks, and the limited flexibility of existing dry test rigs does however put significant restraints on such experiments. In this paper we introduce an alternative novel robotized dry test rig concept for wave power, evaluate its performance and discuss its potential. A full-scale robotized dry test rig demonstrator is constructed and used for experiments with a WEC prototype device. High motion flexibility and accuracy is thereby validated, also for repeating recorded wave and buoy motions. Compared to other dry test rigs, no special components were used and the motion trajectories were defined in full 6-Degrees-Of-Freedom. Two real-time hydrodynamic motion response methods are also demonstrated in the test rig, enabling emulation of actual offshore operation as well as development of advanced WEC control strategies. With a larger industrial robot manipulator, the introduced test rig concept could achieve realistic scaled force and power experiments with most point absorber WECs.

A speech-enabled virtual assistant for efficient human–robot interaction in industrial environments

This paper presents a natural language-enabled virtual assistant (VA), named Max, developed to support flexible and scalable human–robot interactions (HRI) with industrial robots. Regardless of the numerous natural language interfaces already proposed for intuitive HRI on the industrial shop floor, most of those interfaces remain tightly bound with a specific robotic system. Besides, the lack of a natural and efficient human–robot communication protocol hinders the user experience. Therefore three key elements characterize the proposed framework. First, a Client–Server style architecture is introduced so Max can provide a centralized solution for managing and controlling various types of robots deployed on the shop floor. Second, inspired by human–human communication, two conversation strategies, lexical-semantic and general diversion strategies, are used to guide Max’s response generation. These conversation strategies were embedded to improve the operator’s engagement with the manufacturing tasks. Third, we fine-tuned the state-of-the-art (SOTA) pre-trained model, Bidirectional Encoder Representations from Transformers (BERT), to support a highly accurate prediction of requested intents from the operator and robot services. Multiple experiments were conducted using the latest iteration of our autonomous industrial mobile manipulator, “Little Helper (LH)”, to validate Max’s performance in a real manufacturing environment.