Vision-guided System Research Articles

Vision and Radar Steering Reduces Agricultural Sprayer Operator Stress without Compromising Steering Performance

HighlightsStress was measured in professional sprayer operators who, while working, drove manually and with vision or radar steering.Vision and radar steering reduced the average operator stress rate by 48% relative to manual steering.The use of automatic guidance could have a dramatic positive effect on the health of sprayer operators.Sprayer steering performance was reported for professional operators and both vision and radar guidance for the first time.Abstract. Self-propelled agricultural sprayer operators work an average of 15 h d-1 in peak season, and steering is the task that causes the operator the most stress because of the large number of stimuli involved. Automatic guidance systems help reduce stress and fatigue for operators by allowing them to focus on tasks other than steering. Physiological signals like skin conductance (electrodermal activity, EDA) change with stress and can be used to identify stressful events. The objective of this study was to determine if using a commercially available vision and radar guidance system (VSN®, Raven Industries) reduces agricultural sprayer operators’ stress compared to when they are steering manually. Four male professional sprayer operators participated in this study. Each operator performed his job duties normally in GPS-guidance-planted fields, at his self-selected speed, except to drive some passes manually and others with VSN in the same field. EDA was measured with an Empatica E4 wristband, and stressful events were quantified. Machine data (e.g., speed, RTK-GPS, and VSN metrics) were collected from each sprayer via CAN logs. The steering type, stress rate (e.g., stressful events min-1), and steering performance (cross-track error standard deviation, XTE SD) were determined for each pass. In total, 51 passes (23 manual, 28 VSN) in six fields were analyzed. Operators using VSN had a significant reduction (48% lower, p < 0.001) in their stress rate compared to when they were steering manually. There was no significant difference in the XTE SD for the steering type. The use of an automatic guidance system such as VSN could have a dramatic positive effect on the health of sprayer operators, especially during the long workdays of the peak spraying season, and could reduce the negative effects that stress and fatigue have on steering performance, mistakes, and accidents. Keywords: Electrodermal activity, Guidance systems, Machine vision, Precision agriculture, Radar, Skin conductance, Vehicle guidance.

Fire evacuation supported by centralized and decentralized visual guidance systems

In the event of fires and other hazards, visual guidance systems that support evacuation are critical for the safety of individuals. Current visual guidances for evacuations are typically non-adaptive signs in that they always indicate the same exit route independently of the hazard’s location. Adaptive signage systems can facilitate wayfinding during evacuations by optimizing the route towards the exit based on the current emergency situation. In this paper, we demonstrate that participants that evacuate a virtual museum using adaptive signs are quicker, use shorter routes, suffer less damage caused by the fire, and report less distress compared to participants using non-adaptive signs. Furthermore, we develop both centralized and decentralized computational frameworks that are capable of calculating the optimal route towards the exit by considering the locations of the fire and automatically adapting the directions indicated by signs. The decentralized system can easily recover from the event of a sign malfunction because the optimal evacuation route is computed locally and communicated by individual signs. Although this approach requires more time to compute than the centralized system, the results of the simulations show that both frameworks need less than two seconds to converge, which is substantially faster than the theoretical worst case. Finally, we use an agent-based model to validate various fire evacuation scenarios with and without adaptive signs by demonstrating a large difference in the survival rate of agents between the two conditions.

Low-cost and high-performance visual guidance and navigation system for space debris removal

A highly intelligent visual guidance navigation system that can autonomously identify target space debris and realize a controlled rendezvous maneuver is required to mitigate space debris. In this study, such a system is developed by integrating autonomous control software technologies and commercial off-the-shelf (COTS) devices. The orbital performance of the COTS-based intelligent camera system of the end-of-life services is demonstrated during an Astroscale-demonstration mission, a technology demonstration mission for satellite end-of-life services. For this mission, Astroscale collaborated with the Tokyo University of Science to develop a visual guidance and navigation system for space debris Removal. In this regard, this paper reports the design and performance of a COTS-based intelligent camera system for space debris Removal.

Rancang Bangun Visual Docking Guidance System (VDGS) Sebagai Pendeteksi Arah Gerak Longitudinal Pesawat Pada Sistem Parkir Pesawat Terbang

The growth of air transportation and technological developments is getting faster every year. This causes airport services to exceed the ability to provide facilities to meet growth adequately. In this case, it cannot be denied that there are always undesirable things that happen unexpectedly such as airplane accidents which are not only always in the air when flying, there are also many cases of plane accidents at the ground whether it's at the landing or parking process. Therefore, a Visual Docking Guidance System (VDGS) tool was designed using the TF Mini LiDAR sensor and programmed for the aircraft parking system at the airport to identify and guide pilots to find the right position when parking. This tool is able to give guidance information on the aircraft position. A <<<< sign that is, the plane must move to the left, the >>>> sign, the plane must move to the right and the // \\ sign as an indicator the aircraft is in the middle position of parking stand. The results showed that the Visual Docking Guidance System (VDGS) using the LiDAR sensor with a distance specification of 10m is a fairly good level of accuracy and the error obtained from testing the distance measurement tool and actual distance has an error of 0-0,7368% with an average error of 0.0972%.

Towards Laparoscopic Visual AI: Development of a Visual Guidance System for Laparoscopic Surgical Palpation

Currently, there are numerous obstacles to performing palpation during laparoscopic surgery. The laparoscopic interface does not allow access into a patient’s body anything other than the tools that are inserted through the trocars. Palpation is usually done with the surgeon’s hands to detect lumps and certain anomalies underneath the skin, muscle, or tissues. It can be useful technique for augmenting surgical decision-making during laparoscopic surgery, especially when discerning operations involving cancerous tumors. Previous research demonstrated the use of tactile sensors and mechanical sensors placed at the end-effectors for palpating laparoscopically. In this study, a visual guidance system is proposed for use during laparoscopic palpation, specifically engineered to be part of a motion-based laparoscopic palpation system. In particular, the YOLACT++ model is used to localize a target organ, the gall bladder, on a custom dataset of laparoscopic cholecystectomy. Our experiments showed an AP score of 90.10 for bounding boxes and 87.20 on masks. In terms of the speed performance, the model achieved a playback speed of approximately 20 fps, which translates to approximately 48 ms video latency. The palpation path guides are guidelines that are computer-generated within the identified organ, and they show potential in helping the surgeon implement the palpation more accurately. Overall, this study demonstrates the potential of deep learning-based real-time image processing models to complete our motion-based laparoscopic palpation system, and to realize the promising role of artificial intelligence in surgical decision-making. Visual presentation of our results can be seen on our project page: https://kerwincaballas.github.io/lap-palpation.

MyPlate, Half-Plate, and No Plate: How Visual Plate-related Dietary Benchmarks Influence What People Serve

Can visual plate-related dietary guidance systems – such as the MyPlate guideline or the Half-Plate Rule – help people eat better when dining at home or in restaurants? To help explore this, 104 young adults completed a food diary study after having been randomly assigned to follow either 1) USDA MyPlate guidelines, 2) the Half-Plate Rule, or 3) no guidelines (control condition). Both of the visual dietary guidance systems were considered easy to understand, to follow, and left people with fewer questions about what to eat (all p<.01). Moreover, people who rated a system “easy to follow” indicated they had consumed less (meat (r = .268), but this was uncorrelated with fruit and vegetable intake (r =.092) and carbohydrate intake (r = .069). There are three key conclusions to these and other findings: First, the simplest guidance system may be more effective than no system. Second, even the most perfect dietary guidance system will not change behavior if the foods are not available or it is not followed. Third, guidance systems could over-increase the consumption of any food they specifically mention.

Visual guidance for auditory brainstem implantation with modular software design

Abstract An auditory brainstem implant (ABI) attempts to restore hearing at patients with bilateral damaged hearing nerve. The optimal placement of the ABI is challenging even with available auditory measures on the brainstem. We present a visual guidance system that aims to assist during intraoperative ABI placement. As a starting point, a surgical probe is navigated and intuitively visualized in the microscope oculars. The system is developed using modular and agile software design techniques. In a usability study, the participants were able to detect invisible targets marked in a phantom image with a millimetric precision. To the best of our knowledge, this is the first time that this kind of visual guidance is presented. In the future, the system will be expanded with surgical instruments used for the ABI placement.

Multi-View-Based Pose Estimation and Its Applications on Intelligent Manufacturing.

Pose estimation is a typical problem in the field of image processing, the purpose of which is to compare or fuse images acquired under different conditions. In recent years, many studies have focused on pose estimation algorithms, but so far there are still many challenges, such as efficiency, complexity and accuracy for various targets and conditions, in the field of algorithm research and practical applications. In this paper, a multi-view-based pose estimation method is proposed. This method can solve the pose estimation problem effectively for large-scale targets and achieve good performance accuracy and stability. Compared with existing methods, this method uses different views (positions and angles), each of which only observes some features of large-size parts, to estimate the six-degree-of-freedom pose of the entire large-size parts. Experimental results demonstrate that the accurate six-degree-of-freedom pose for different targets can be obtained by the proposed method which plays an important role in many actual production lines. What is more, a new visual guidance system, applied into intelligent manufacturing, is presented based on this method. The new visual guidance system has been widely used in automobile manufacturing with high accuracy and efficiency but low cost.

An Automatic Machine Vision-Guided System for the Propagation of Potato Test-Tube Plantlets

In manually propagating potato test-tube plantlets (PTTPs), the plantlet is usually grasped and cut at the node point between the cotyledon and stem, which is hardly located and is easily damaged by the gripper. Using an agricultural intelligent robot to replace manual operation will greatly improve the efficiency and quality of the propagation of PTTPs. An automatic machine vision-guided system for the propagation of PTTPs was developed and tested. In this paper, the workflow of the visual system was designed and the image acquisition device was made. Furthermore, the image processing algorithm was then integrated with the image acquisition device in order to construct an automatic PTTP propagation vision system. An image processing system for locating a node point was employed to determine a suitable operation point on the stem. A binocular stereo vision algorithm was applied to compute the 3D coordinates of node points. Finally, the kinematics equation of the three-axis parallel manipulator was established, and the three-dimensional coordinates of the nodes were transformed into the corresponding parameters X, Y, and Z of the three driving sliders of the manipulator. The experimental results indicated that the automatic vision system had a success rate of 98.4%, 0.68 s time consumed per 3 plants, and approximate 1 mm location error in locating the plantlets in an appropriate position for the medial expansion period (22 days).

Application of Artificial Intelligence Algorithm in Visual Guidance of Industrial Robot

With the rapid development of science and technology and manufacturing industry, the position of industrial robots in the advanced manufacturing industry is also improving, the application of industrial robots has been expanding. Visual guidance technology can help industrial robots to complete the production work with high flexibility and high intelligence, and promote the fine development of industrial robot technology. The purpose of this paper is to realize the good application of artificial intelligence algorithm in the visual guidance of industrial robots and promote the continuous improvement of its visual guidance system. This paper first discusses the concepts of artificial intelligence, industrial robots and visual guidance, then analyzes the specific application of artificial intelligence algorithm in visual guidance system based on Apriori algorithm, and detects the visual guidance system under artificial intelligence algorithm through simulation experiments. The experimental results show that the industrial robot vision system based on artificial intelligence algorithm in this paper has a great advantage in calibration accuracy and calibration time.

Vision Guiding System Based on Strong Positioning and Hand-eye Calibration

In order to solve the current localization algorithm big background interference affects calibration methods exist damage, physical contact with the workpiece material on the surface of the problem, hand-eye calibration was proposed based on strong orientation and three visual guidance system. First, deepen the Hessian matrix, using Gaussian filter, using the maximum inhibition precisely determine the feature points and direction, in order to construct surf feature points description, finish strong positioning operator based on surf, to remove the background and the ambient light interference, the purpose of accurate positioning center of the target with Angle. And then put forward three points calibration method, calculation scale rotation matrix and translation matrix parameters, complete the Robot world coordinates and binding mapping camera image coordinates. Finally, program implementation algorithm and system, the experiment results showed that compared with the current workpiece material transfer system, this system has higher goal orientation and calibration transfer success rate.

Ekonomska opravdanost kabina kranova opremljenih računarskim sistemom vizuelnog navođenja u realnom vremenu

Cranes’ operations are extremely risky and hazardous tasks, as demonstrated in various surveys in the field of incidents’ causes and effects. The most frequent cause of unsafe crane operation task execution is human error. Accordingly, innovation which enables cranes’ safe operations, prevents “blind lifts” and helps crane operator to avoid potential error is urgent. Project SPRINCE has proposed computer-aided Visual Guidance System as a real time solution aimed to prevent crane accidents, caused by obstructed view. This paper analyses the economic feasibility of Visual Guidance System application in various scenarios in aim to connect human, technology and organization issues through cost-benefit framework. There are two proposed investment scenarios analysed: A) Producing and selling of crane cabins with real-time computer-aided Visual Guidance System and B) Usage (purchase) of crane cabins with real-time computer-aided Visual Guidance System. The economic appraisal has proved that the total economic benefit in both scenarios in the complete utilization period is several times higher than the buying price, while the internal rate of return is few times higher than the aggregate rate of interest paid. Also, the amount of time which takes to recover the cost of an investment is less than four years. Moreover, both scenarios belong to the category of innovation projects with very low risk.

Sight-to-Sound Human-Machine Interface for Guiding and Navigating Visually Impaired People

Visually impaired people often find it hard to navigate efficiently in complex environments. Moreover, helping them to navigate intuitively is not a trivial task. In sighted people, cognitive maps derived from visual cues play a pivotal role in navigation. In this paper, we present a sight-to-sound human-machine interface (STS-HMI), a novel machine vision guidance system that enables visually impaired people to navigate with instantaneous and intuitive responses. The proposed STS-HMI system extracts visual context from scenes and converts them into binaural acoustic cues for users to establish cognitive maps. A series of experiments were conducted to evaluate the performance of the STS-HMI system in a complex environment with difficult navigation paths. The experimental results confirm that the STS-HMI system improves visually impaired people’s mobility with minimal effort.

Intelligent Path Recognition against Image Noises for Vision Guidance of Automated Guided Vehicles in a Complex Workspace

Applying computer vision to mobile robot navigation has been studied more than twodecades. The most challenging problems for a vision-based AGV running in a complex workspaceinvolve the non-uniform illumination, sight-line occlusion or stripe damage, which inevitably resultin incomplete or deformed path images as well as many fake artifacts. Neither the fixed thresholdmethods nor the iterative optimal threshold methods can obtain a suitable threshold for the pathimages acquired on all conditions. It is still an open question to estimate the model parameters ofguide paths accurately by distinguishing the actual path pixels from the under- or oversegmentationerror points. Hence, an intelligent path recognition approach based on KPCA–BPNNand IPSO–BTGWP is proposed here, in order to resist the interferences from the complexworkspace. Firstly, curvilinear paths were recognized from their straight counterparts by means of apath classifier based on KPCA–BPNN. Secondly, an approximation method based on BTGWP wasdeveloped for replacing the curve with a series of piecewise lines (a polyline path). Thirdly, a robustpath estimation method based on IPSO was proposed to figure out the path parameters from a set ofpath pixels surrounded by noise points. Experimental results showed that our approach caneffectively improve the accuracy and reliability of a low-cost vision-guidance system for AGVs in acomplex workspace.

SVM-based image partitioning for vision recognition of AGV guide paths under complex illumination conditions

Abstract Applying computer vision to mobile robot navigation has been studied more than two decades. For the commercial off-the-shelf (COTS) automated guided vehicles (AGV) products, the cameras are still not widely used for the acquisition of guidance information from the environment. One of the most challenging problems for a vision guidance system of AGVs lies in the complex illumination conditions. Compared to the applications of computer vision where on-machine cameras are fixed in place, it is difficult to structure the illumination circumstance for an AGV that needs to travel through a large work space. In order to distinguish the original color features of path images from their illumination artifacts, an illumination-adaptive image partitioning approach is proposed based on the support vector machine (SVM) classifier with the slack constraint and the kernel function, which is utilized to divide a path image to low-, normal-, and high-illumination regions automatically. Moreover, an intelligent path recognition method is developed to carry out guide color enhancement and adaptive threshold segmentation in different regions. Experimental results show that the SVM-based classifier has the satisfactory generalization ability, and the illumination-adaptive path recognition approach has the high adaptability to the complex illumination conditions, when recognizing the path pixels in the field of view with both high-reflective and dark-shadow regions. The 98% average rate of path recognition will significantly facilitate the subsequent operation of path fitting for vision guidance of AGVs.

Rancang Bangun Alat Ukur Jarak dan Peringatan pada Visual Docking Guidance System menggunakan Sensor Lidar

The limited ground marshall, yellow line and others at an airport will affect when parking aircraft. Aviation technology has provided solutions, among others, by attaching to aircraft equipment known as the Visual Docking Guidance System (VDGS), is a tool provided by the airport to help a pilot park his plane without the help of a marshaller to direct the movement of planes to the correct parking. However, not all airports are equipped with sophisticated equipment. So this research aims to create a system that can make it easier for pilots to park aircraft while in Apron. To overcome this problem, a VDGS tool was designed which was designed using a TF Mini Lidar sensor and programmed for aircraft parking systems at pioneering airports. This tool is able to provide information on the distance of the plane as far as 12 meters given a condition that the aircraft will display identification of guidance or collision prevention that will STOP if the aircraft is at a distance of 1-3 meters, and GO identification if the aircraft is at a distance of 4-12 meters so that this tool can serves as a guide to the position of the aircraft when parking right in the designated parking area and prevent aircraft crashes when parking when entering the apron area at the pioneer airport.

Design, Fabrication, and Testing a Semiautomatic Sewing Device for Personalized Stent Graft Manufacturing

For the treatment of abdominal aortic aneurysm, a personalized stent graft is used to ensure that it fits tightly to the patients vessel geometry. A personalized stent graft is usually handmade, which requires thousands of stitches and can take weeks or even months to complete. This delay may expose the patient to the risk of aneurysm rupture. This paper presents a robotic sewing device that can enhance the stent graft sewing speed by providing automated needle manipulation. It simplifies the sewing process and has the potential to achieve fully automated stent graft manufacturing via a vision-guided system. The device features a sewing probe that can switch a double-pointed semicircular needle between two movable jaws. This forgoes the need for manual needle handling including grasping, driving rotation, releasing, and regrasping, which requires a high level of manual dexterity and attention. This paper presents the design of the device, its mechanical synthesis and experimental validation. The focus of the paper is on the linkage parameter optimization and needle locking mechanism design. The proposed device has been fabricated using 3-D rapid prototyping techniques, and its performance has been compared with the conventional manual sewing method. The experimental results show that the device can achieve a 30% reduction of the completion time for a stitching task while achieving better consistency and quality of the stitches.

A new strategy for enteral nutrition using a deflection flexible visual gastric tube: A randomized crossover manikin trial.

Enteral nutrition via gastric tube insertion is a routine clinical practice for critically ill patients, although complications due to blind manipulation are occasionally reported. An 8.4Fr deflection flexible ureteroscope was delivered into a 15Fr conventional gastric tube to create a gastric visual guidance system. Twenty inexperienced physicians were randomly assigned to perform 5 repeated orogastric tube placements in a manikin using both the conventional method and the deflection visual gastric tube, for a total of 10 procedures per physician. Placement time, procedure-related complications, and participants' experience with both methods were recorded. Under real-time guidance, the visual gastric tube successfully reached the stomach. The procedure provided additional information on the anatomy of the esophagus and stomach. Placement time was significantly less in the visual group than in the conventional group (39.39 ± 2.11 seconds vs 49.82 ± 3.11 seconds; P < .001). Procedure-related complications were not observed in the visual group; however, the gastric tube was misplaced into the airway in 19 out of 100 cases (19%) in the conventional group. Furthermore, 17 out of 20 participants (85%) preferred the visual gastric tube guide over the standard method. Results of this manikin model demonstrate that it is feasible to use the deflection flexible visual gastric tube to create a route for enteral nutrition and that such a procedure decreases placement time and procedure-related complications compared to the conventional procedure. These findings may point to a new strategy for gastric tube insertion in the future.

A Node-Based Method for SLAM Navigation in Self-Similar Underwater Environments: A Case Study

This work deals with the development of a node-based monocular visual methodology for autonomous vehicle navigation which has the goal of exploring unknown regions of the sea bottom with a posterior safe revisiting of them. The work accentuates characteristics of the seabed like self-similarity and backscattering. In a stepwise fashion, a visual guidance system constructs a shape similar to a narrow corridor by optimally creating a heading function on the basis of keypoints threads, which ensures future revisits. The corridor is composed of nodes and paths in between. Each path is composed of a visual-odometry-based trail which is generated in feature-poor environments, in combination with a feature-based trail which emerges in feature-rich regions. A probabilistic analysis of the uncertainties and their impact in the success rate on loop closings is carried out. We work out two case studies, the first employing an ad-hoc benchmark and the second a series of experiments in the real world. From here qualitative conclusions can be drawn out that enable us to anticipate potential applications of the approach in the field of autonomous navigation underwater.

PERANCANGAN TAMPILAN VISUAL DOCKING GUIDANCE SYSTEM (VDGS) PADA SISTEM PARKIR PESAWAT TERBANG

Visual Docking Guidance System (VDGS), is a device provided by the airport service to assist a pilot parked the plane on the righr place. On Aircraft parking control activities in the airport, the pilot is assisted by a marshaller that aims to be steering the movement of aircraft to the correct parking spot. The more density of the movement of aircraft on the apron activity requires the use of alternative technology to substitute marshaller, this technology is called Visual Docking Guidance Syatem (VDGS). VDGS display design uses three pieces o f board o f dot matrix display that is made into one to form a combination in accordance with the appearance on the VDGS on the market. Display used is a superbright LED dot matrix. The controller is a three pieces of. PIC 16F84A microcontroller and a series o f shift registers to display the movement of the aircraft to the right, to the left, straight and stops and also display the aircraft identity.