Visual Assistance System Research Articles

Design of visual symbol-aided system based on wireless network sensor and embedded system

Abstract In order to solve the problem of the low transmission rate of wireless network sensors (WNSs), this article proposes a WNS visual assistance system based on embedded systems. The system uses WNSs instead of traditional wired sensors, which can achieve synchronous transmission of multi-node data, enabling nodes to work together better, thereby improving the real-time and reliability of the entire system. This article conducts in-depth research on feature extraction algorithms and tests the visual assistance system in the experimental section. The results show that the recognition rate and stability of the visual symbol assistance system implemented using WNSs are higher than those of ordinary systems. In the satisfaction survey, it was found that 87 people were very satisfied with the visual symbol assistance system, accounting for 87%, while only 57 people were very satisfied with the traditional visual symbol assistance system, accounting for 57%. The experimental results show that the system output stability of the design method is good, and the response time and reliability are better.

OrCam MyEye 2.0, A Dire Need to Revolutionize the Lives of Congenital or Acquired visually Impaired Patients in Pakistan.

I am writing to highlight the pressing issues in the management of congenital or acquired visually impaired patients, and to accentuate the dire need for an alternative approach and attitude towards their rehabilitation in Pakistan. To lessen the socio-emotional impacts and dependency of patients who are blind or visually impaired, as well as to improve their quality of life, ophthalmologists must pay close attention to the inadequate use of AI visual assistance systems and contemporary technology.1

APPLICATION OF OBJECT DETECTION TECHNOLOGY IN AORTOGRAPHY KEYPOINT TRACKING

HighlightsThe practical significance of the work lies in the fact that the presented aortography data visualization system is an effective tool for visually assisting surgeons during transcatheter aortic valve implantation interventions, supporting real-time operation mode. The proposed data preprocessing algorithm, which improves image quality with minimal performance costs, complements the system, allowing specialists to achieve the best result. AbstractAim. The aim of this study is to develop a visual assistance system for transcatheter aortic valve implantation procedures.Methods. To address the stated objective, our own dataset consisting of 35 videos of the intervention was used. The visualization system is based on the approach of detecting key points in aortography, utilizing “Object detection” technology with the application of YOLO family artificial neural networks. To achieve the best result, we proposed a method to enhance the quality of input data using convolutional neural networks, specifically the «Autoencoder» technology.Results. The results of the study revealed that the convolutional autoencoder model is capable of restoring the informativeness of noisy input images from 40 to 75%, thereby increasing the accuracy of object detection in images. The presented real-time tracking system for facilitating TAVI procedures achieves a final accuracy of 51.9% according to the Mean Average Precision (MAP) quality metric. The visual assistance system can recognize and track key points indicating the location of the aortic root, delivery system, and heart valve prosthesis during surgery. The practical significance of the work lies in the fact that the presented aortography data visualization system is an effective tool for visually assisting surgeons during interventions, supporting real-time operation mode.Conclusion. The proposed data preprocessing algorithm, which improves image quality with minimal performance costs, complements the visualization system, allowing specialists to achieve the best results.

Aortography Keypoint Tracking for Transcatheter Aortic Valve Implantation Based on Multi-Task Learning.

Currently, transcatheter aortic valve implantation (TAVI) represents the most efficient treatment option for patients with aortic stenosis, yet its clinical outcomes largely depend on the accuracy of valve positioning that is frequently complicated when routine imaging modalities are applied. Therefore, existing limitations of perioperative imaging underscore the need for the development of novel visual assistance systems enabling accurate procedures. In this paper, we propose an original multi-task learning-based algorithm for tracking the location of anatomical landmarks and labeling critical keypoints on both aortic valve and delivery system during TAVI. In order to optimize the speed and precision of labeling, we designed nine neural networks and then tested them to predict 11 keypoints of interest. These models were based on a variety of neural network architectures, namely MobileNet V2, ResNet V2, Inception V3, Inception ResNet V2 and EfficientNet B5. During training and validation, ResNet V2 and MobileNet V2 architectures showed the best prediction accuracy/time ratio, predicting keypoint labels and coordinates with 97/96% accuracy and 4.7/5.6% mean absolute error, respectively. Our study provides evidence that neural networks with these architectures are capable to perform real-time predictions of aortic valve and delivery system location, thereby contributing to the proper valve positioning during TAVI.

Development of a high resolution scanning RGB laser headlamp

Abstract Recent rapid progress in automotive lighting technology led to the emergence of headlamps featuring a large variety of light distributions that are highly adapted to provide best possible road illumination in particular traffic situations. Visual assistance systems which highlight relevant objects or project symbols to communicate with other traffic participants, further improve road safety. Implementing all these functions within a single headlamp usually requires the use of several additional modules with high and low resolution. This multitude of modules impacts the installation space, styling and cost of automotive headlamps. This paper presents a high-resolution red-green-blue (RGB) laser headlamp module which provides a good illumination of the road and can also be used for external communication purposes. A specially developed laser scanning unit, based on a bi-axial resonant micro-electro-mechanical systems (MEMS) scanner and a high-power RGB laser, serves as the technical basis. Three module concepts are designed using light simulation, constructed as computer-aided design (CAD) models and integrated into a serial headlamp package. The concepts are based on design, simulation and measurement data of the scanning unit.

Use of Neural Networks in Visual Assistance Systems for Transcatheter Implantation of Aortic Valve Prostheses

The current state of approaches to analysis of medical angiographic images obtained during interventions on the circulatory system is analyzed. The main approaches — image-based algorithms, machine learning algorithms, and deep neural network learning — are considered from the point of view of individual scientific studies and from the applied point of view, i.e., in terms of development of visual assistance systems for medical procedures. In addition, the authors’ own views on the potential for solving difficulties in creating systems for cardiovascular surgery (in particular, transcatheter replacement of the aortic valve) are presented.

Adaptive Visual Assistance System for Enhancing the Driver Awareness of Pedestrians

ABSTRACTIn the past decade, Pedestrian Collision Warning Systems have been proposed to detect pedestrians and warn drivers of imminent collision. However, such systems are often limited by eye-off-road and cognitive overload problems. Head Up displays with augmented reality are being considered as a key technology for changing drivers’ user experiences. In this paper, we propose a new visual assistance system that can enhance drivers’ perception by dynamically directing attention to pedestrians to avoid collisions using Augmented Reality cues. The proposed system takes into account driver behaviors through vehicle driving signals analysis, in order to warn it at the right moments. To that end, we statistically model correct and incorrect driver behaviors in situations with pedestrians. Based on this model, a warning visual metaphor is displayed if unawareness is detected and a driving simulator was used to evaluate the concept. The experimental results suggest that our proposed adaptive visual aids can enhance driver awareness of pedestrians in critical situations.

The Assist-By-X system: Calibration and application of a modular production equipment for visual assistance

Shorter production lifecycles, new challenges due to the advancing globalization, and increasing demand for individualized products are only some of the general conditions that internationally operating manufacturing companies have to face. Therefore, adaptable production systems are required which can be changed to the current requirements of the production process. In this article, a modular visual assistance system is presented which can be integrated in different production systems and environments. For a successful integration of the assistance system a calibration is indispensable, thus a calibration method for one component of the assistance system is discussed in detail. In order to control the system, a visual data flow programming tool is used in combination with MQTT as a standard communication protocol. By doing so, the programming effort of the system is reduced and new components can easily be added to the system by creating new flows.

Evaluation of Wearable Visual Assistance System for Manual Automotive Assembly

Automotive manual assembly cannot avoid operations with limited sight for the associate. To complete the process, the associate often must rely on tactile feel or to assume ergonomically challenging positions. This work aims to provide a wearable camera system which can record a field of view near an associate's hand and display it remotely to the associate and determine user preference and acceptance. We present three camera systems: (1) wearable camera with arm wearable display, (2) wearable camera with external stationary display, and (3) wearable camera with smart glasses screen. The conducted evaluations included novice user evaluation of accuracy, timing, user preferences, and other performance results conducted on and off the assembly line providing insight on accuracy, timing, user acceptance, preferences, and performance potential in the production environment. This work is intended to assist assembly associates to better view their work area and to complete tasks faster, more reliably, and with higher quality. Our results show an improvement in performance metrics for novice users, with them experiencing increased accuracy and reduced timing with a positive acceptance rate for assistive technologies.

Visual support system for remote control by adaptive ROI selection of monitoring robot

A visual assistance system has become attractive as a technique to improve the efficiency and stability of remote control. While an operator controls a working robot, another autonomous monitoring robot evaluates a suitable viewpoint to observe the work site, and dynamically moves to the optimal viewpoint for monitoring. Choosing the observation region (ROI: region of interest) is equivalent to deciding the action of the following autonomous monitoring system. Therefore, we focus on ROI detection in our visual support system. We propose an ROI selection method to identify the most suitable observation point and interobject relations. The monitoring robot detects a gestalt of the scene in order to identify the relations between objects. Such an adaptive ROI in real time improves the efficiency of the remote control. The experimental results indicate the effectiveness of the proposed system in terms of execution time and number of errors.

A Multitask Grocery Assist System for the Visually Impaired: Smart glasses, gloves, and shopping carts provide auditory and tactile feedback

According to the World Health Organization, "285 million people are estimated to be visually impaired worldwide" [1]. Several technologies such as automatic text readers, Braille note makers, and navigation assistance canes have been developed to assist the visually impaired. Concurrent advances in computer vision and hardware technologies provide opportunities for a visual-assistance system that can be used in multiple contexts. As part of the Visual Cortex on Silicon program, we have been developing interfaces, algorithms, and hardware platforms to assist the visually impaired with a focus on grocery shopping. This article describes the various features that we have incorporated into this visual-assistance system so that it can be used in multiple contexts.

Robust Localization of Texts in Real-World Images

Localization of texts in natural images could be an important stage in many applications such as content-based image retrieval, visual impairment assistance systems, automatic robot navigation in urban environments and tourist assistance systems. However due to the variations of font, script, scale, orientations, color, shadow and lighting conditions, robust scene text localization is still a challenging task. In this paper, we propose a novel method to localize not only Farsi/Arabic and Latin texts with different sizes, fonts and orientations but also low luminance contrast and poor quality ones in the natural images taken with uneven illumination conditions. Firstly, fast weighted median filtering as a nonlinear edge-preserving smoothing filter and then color contrast preserving decolorization are exploited to make the text localization system more robust for low luminance contrast and poor quality texts. In order to extract the Farsi/Arabic and Latin scene texts and also filter the nontext ones, a unified framework is proposed incorporating the maximally stable extremal regions and a novel proposed region detector called Stable Width Stroke Regions which is based on closed boundary regions. Phase congruency and Laplacian operators are exploited to extract the closed boundary regions. Finally, to extract the single text lines, the Meanshift clustering and radon transform were used. Experimental results show that the proposed method localize low luminance contrast and low quality scene texts for both Farsi/Arabic and Latin scripts encouragingly.

Competence Development for the Holistic Design of Collaborative Work Systems in the Logistics Learning Factory

Shorter product life cycles and emerging technologies in the field of industrial equipment are changing the prerequisites and circumstances under which the design of assembly and logistics systems takes place. Planners have to adapt the production in accordance with the underlying product at a higher pace, oversee a more complex system and – most importantly – find the ideal solution for functional as well as social interaction between humans and machines in a cyber-physical system. Such collaborative work systems consider the individual capabilities and potentials of humans and machines to combine them in a manner that assists the operator during his daily work routine towards more productive, less burdening work. To be able to design work systems which act on that maxim, specific competences such as the ability of integrated process and product planning as well as systems and interface competence are required. The ESB Logistics Learning Factory trains students as well as professionals to gain such qualification by providing a close-to-reality learning environment based on a didactical concept which covers all relevant methods for ergonomic work system design and a state-of-the-art infrastructure composed of an manual assembly system, service robots, visual assistance systems, sensor-based work load monitoring and logistical resources. Group-based, activity oriented scenarios enable the participants to put the learnings into practice within their professional environments. By this, Learning Factories have an indirect impact on the transfer of proven best practices to the industry and thereby on the diffusion of the idea of a human-centric working environment.

Image Segmentation using Learning Vector Quantization of Artificial Neural Network

This paper presents color image segmentation application for road-sign detection and recognition system based on Learning vector Quantization (LVQ). In traffic-sign maintenance and in a visual driver assistance system, road-sign detection and recognition are two of the most important functions. Our system is able to detect and recognize the signs. Road signs provide drivers important information and help them to drive more safely and more easily by guiding and warning them and thus regulating their actions. The RGB images are converted into HSV color space, and segmented using LVQ depending on the hue values of each pixel in the HSV color space. LVQ neural network is used to detect the color from the images and recognize them with high accuracy and speed. The detection and recognition of traffic signs represents an important issue in the field of traffic sign analysis. The paper researches the existing traffic sign detection and recognition methods. Special attention has been given to the methods based on artificial neural networks. Some techniques extracted various features from the images containing traffic signs and used various detectors and classifiers to perform detection and recognition. The following examples of these methods can be mentioned: Histogram of Oriented Gradient (2) or (1) and (6) or Support Vector Machines (SVM) (5). Some authors have also used artificial neural networks (4). This paper, however, uses a method of neural network (NN) which has given successful results. For individual object identification in humans, Color discrimination plays an important role. Image segmentation is the basic and first step regarding image analysis and pattern recognition. In image processing, image segmentation is not only the critical and essential component but also it is a very difficult task. The actual operation of the algorithm determines the accuracy and quality of image analysis. Color segmentation is a process of extracting one or more connected regions from the image domain. These connected regions are satisfying the uniformity (homogeneity) criterion which is derived from the spectral components. These components are then defined with respect to some color spaced model such as RGB model. RGB is the most common model. In RGB model, a color point is defined by the color components of the corresponding pixel which are defined in the form of Red (R), Green (G) and Blue (B). This paper considers a color image segmentation problem as a pixel classification problem. .

Multimodal user interface for a semi-robotic visual assistance system for image guided neurosurgery

We developed a visual assistance system for image guided neurosurgery, consisting of a stereoscopic digital camera (exoscope) mounted on a semi-robotic manipulator. In order to minimize the operation time, the application-specific multimodal user interface enables hands-free manipulation of the exoscope. The surgeon wears thereby a head-mounted unit with a binocular display, a head tracker, a microphone and earphones. Different modes of view positioning and adjustment can be selected by voice and controlled by head rotation while pressing a miniature confirmation button mounted on a finger ring or suction device. Apart from the development of the mechatronic and software modules of the semi-robotic manipulator, initial studies focused on the evaluation and optimization of the intuitiveness, comfort and precision of different modes of operation. For the user based evaluation of different control modes, a simulation has been implemented in stereoscopic virtual reality. Results of user tests with neurosurgeons are presented in this paper. Based on these findings, the multimodal user interface has been implemented into the first labtype. The system has been demonstrated in the operating room in first tests on a phantom together with the clinical partner.