Robust Vision Research Articles

COVID-19 and Vision Therapy: An Academic Experience

Background: The COVID-19 pandemic required unprecedented changes to health care delivery. The University Eye Center (UEC) at the State University of New York College of Optometry has a robust vision rehabilitation practice. However, during the pandemic via state mandate, it could no longer provide in-office care. Development of novel approaches to manage patients with binocular vision disorders, especially to care for the vulnerable traumatic brain injury and pediatric patient populations, was essential and required rapid response. This included a combination of launching telehealth visits and a home-based vision therapy app, as well as innovative programs developed internally. While telehealth is not a substitute for in-office vision therapy, the combination of multiple platforms enabled the UEC to continue patient care despite a stay-at-home order during the pandemic.

Edge‐aware image filtering using a structure‐guided CNN

Image filtering is a fundamental preprocessing step for accurate, robust computer vision applications such as image segmentation, object classification, and reconstruction. However, many convolutional neural network (CNN)-based methods tend to lose significant edge information in the output layer, and generate undesired artefacts in the feature extraction layers. This study presents a deep CNN model for edge-aware image filtering. The proposed network model consists of three sub-networks: (i) feature extraction, (ii) convolution artefact removal, and (iii) structure extraction networks. The proposed network model has an end-to-end trainable architecture that does not need any post-processing steps. Especially, the structure extraction network can successfully preserve significant edges. The proposed filter outperforms state-of-the-art denoising filters in terms of both objective and subjective measures, and can be used for various image enhancement and restoration problems such as edge-preserving smoothing, image denoising, deblurring, and deblocking.

Enhancing Cross-Modal Retrieval Based on Modality-Specific and Embedding Spaces

A new approach that drastically improves cross-modal retrieval performance in vision and language (hereinafter referred to as “vision and language retrieval”) is proposed in this paper. Vision and language retrieval takes data of one modality as a query to retrieve relevant data of another modality, and it enables flexible retrieval across different modalities. Most of the existing methods learn optimal embeddings of visual and lingual information to a single common representation space. However, we argue that the forced embedding optimization results in loss of key information for sentences and images. In this paper, we propose an effective utilization of representation spaces in a simple but robust vision and language retrieval method. The proposed method makes use of multiple individual representation spaces through text-to-image and image-to-text models. Experimental results showed that the proposed approach enhances the performance of existing methods that embed visual and lingual information to a single common representation space.

WepSIM: An Online Interactive Educational Simulator Integrating Microdesign, Microprogramming, and Assembly Language Programming

Our educational project has three primary goals. First, we want to provide a robust vision of how hardware and software interplay, by integrating the design of an instruction set (through microprogramming) and using that instruction set for assembly programming. Second, we wish to offer a versatile and interactive tool where the previous integrated vision could be tested. The tool we have developed to achieve this is called WepSIM and it provides the view of an elemental processor together with a microprogrammed subset of the MIPS instruction set. In addition, WepSIM is flexible enough to be adapted to other instruction sets or hardware components (e.g., ARM or x86). Third, we want to extend the activities of our university courses, labs, and lectures (fixed hours in a fixed place), so that students may learn by using their mobile device at any location, and at any time during the day. This paper presents how WepSIM has improved the teaching of Computer Architecture courses by empowering students with a more dynamic and guided learning process. In this paper, we show the results obtained during the experience of using the simulator in the Computer Structure course of the Bachelor's Degree in Computer Science and Engineering, University Carlos III of Madrid.

Novel and Robust Vision- and System-on-chip-based Sensor for Fall Detection

Novel and Robust Vision- and System-on-chip-based Sensor for Fall Detection

Attending to the public understanding of science education: A response to Furtak and Penuel

AbstractThis commentary is a response to Furtak and Penuel (2019), and raises four key points. The first is to reframe Furtak and Penuel's argument as an appeal for a public understanding of science education, and by extension, science education reform. The second is to note that such a public understanding of science education must include attention to more than the structure and content of science curricula and the enactment of a “practice turn”; it must also include a robust vision of science pedagogy—something that is markedly absent from the recent Next Generation Science Standards and Framework documents. The third is to make the case that the use of terminology like “hands‐on” is really a marker for a conception of science education, and by attempting to communicate a competing model of science education, reformers are aiming to effect conceptual change among the public about science education itself. Finally, I argue that logical and rational efforts to communicate a public understanding of science education are likely to fall short if affective factors—which are highly activated when discussing science and science education publicly—are not taken into account.

Slantwise Moves: Games, Literature, and Social Invention in Nineteenth-Century America. By Douglas A. Guerra (Philadelphia: University of Pennsylvania Press, 2018. 264 pp. $69.95)

Douglas Guerra positions his study of nineteenth-century American board games within a broader account of the history of the book. Like the “semantic vectors” arrayed across the printed page of modern novels, the sequences of rules governing leisure-time amusements functioned as a medium for the “circulation of ideas through the arteries of culture” (197), the most important idea of which was the sovereignty of the self. Any skepticism regarding this ambitious argument is convincingly offset by Guerra’s close reading of Milton Bradley’s Checkered Game of Life, which sold more than forty thousand copies soon after its invention in 1860. Play was organized around an opportunistic grid of “wealth,” “matrimony,” “school,” “fame,” “idleness,” “poverty,” “disgrace,” and “jail,” supplemented with behavioral norms that included “truth,” “honor,” and “ambition” and which together encompassed most of the human condition. Navigating this checkerboard of personal prospects required “a frequent choice of moves involving the exercise of judgment,” as Bradley himself exclaimed upon applying for a patent (20). The game of life was accordingly designed as a trial run for life off the board, synopsized by Guerra with an epigraph from Whitman’s “Song of Myself” (1855) that presents a similarly robust vision of the individual’s newly open field of play. As such, Milton Bradley and Walt Whitman each promoted “life” into the exclusive subject of those who lived it, a creed that inverted America’s traditional moral economy by re-constituting public order on the basis of private prerogative. The subsequent destruction of fixed hierarchies meant that “I” became the singular expression of my own talent and tenacity, a distinctly modern ethic which gave birth to a distinctly modern cultural hero in these same years, the self-made man. Or, as Guerra quotes Emerson’s pithy characterization of this self-centered gestalt: “You think me the child of my circumstances: I make my circumstances” (47).

Motion and appearance based background subtraction for freely moving cameras

With increasing popularity of smart phone cameras and wearable cameras, it is imperative to develop robust vision systems in analyzing videos captured by these freely moving cameras. In this paper, we propose a novel motion and appearance based algorithm for foreground/background segmentation of these videos. Unlike existing methods, it does not require any prior information nor does it restrict camera motion or scene geometry. The proposed algorithm first estimates a dense motion field between two consecutive frames, and obtains a motion-based foreground probability estimate for each pixel by comparing the motion field with its low-rank approximation. In parallel, color features are extracted by sliding a fixed-size neighborhood window over the entire image. Using the motion-based probability estimates, highly probable foreground and background color features are identified and used to learn foreground and background appearance models. These models then generate appearance-based probability estimate for each pixel. To overcome the inaccuracies in appearance modeling and background motion approximation, we incorporate an innovative Mega-pixel denoising process that uses color segmentation to smooth out the probability estimates. Finally, the denoised probability estimates are combined with the image gradient map to produce the output foreground mask under the Graph-Cut optimization framework. To cope with non-stationary dynamic scenes, the foreground and background appearance models are continuously updated with highly probable foreground and background color features. Extensive evaluations on publicly available test sequences show that the proposed technique outperforms six state-of-the-art algorithms.

Adaptive Autonomous Technology in Unmanned Aerial Vehicles for Parcel Delivery

An Unmanned Aerial Vehicle (often called a Drone) is an aircraft that is not operated by a pilot on board. Drones are rapidly growing market and increasingly leverage. The Applications of these flying robots are limitless. This paper reviews and discusses about embedded vision and sensor technology which culminates in several innovative applications over drones. It introduces an industry alliance available to help product creators incorporate robust vision capabilities into their drones designs. This paper mainly deals on Autonomous Delivery of light weighted items along with the collision avoidance technology and also tracks the whereabouts of the vehicle in a cost effective approach. This can also be used for carrying out complex assignments such as rescue operations during disasters, mines surveying.

Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation.

We investigate two crucial and closely-related aspects of CNNs for optical flow estimation: models and training. First, we design a compact but effective CNN model, called PWC-Net, according to simple and well-established principles: pyramidal processing, warping, and cost volume processing. PWC-Net is 17 times smaller in size, 2 times faster in inference, and 11 percent more accurate on Sintel final than the recent FlowNet2 model. It is the winning entry in the optical flow competition of the robust vision challenge. Next, we experimentally analyze the sources of our performance gains. In particular, we use the same training procedure for PWC-Net to retrain FlowNetC, a sub-network of FlowNet2. The retrained FlowNetC is 56 percent more accurate on Sintel final than the previously trained one and even 5 percent more accurate than the FlowNet2 model. We further improve the training procedure and increase the accuracy of PWC-Net on Sintel by 10 percent and on KITTI 2012 and 2015 by 20 percent. Our newly trained model parameters and training protocols are available on https://github.com/NVlabs/PWC-Net.

Date Fruit Classification for Robotic Harvesting in a Natural Environment Using Deep Learning

An accurate vision system to classify and analyze fruits in real time is critical for harvesting robots to be cost-effective and efficient. However, practical success in this area is still limited, and to the best of our knowledge, there is no research in the area of machine vision for date fruits in an orchard environment. In this work, we propose an efficient machine vision framework for date fruit harvesting robots. The framework consists of three classification models used to classify date fruit images in real time according to their type, maturity, and harvesting decision. In the classification models, deep convolutional neural networks are utilized with transfer learning and fine-tuning on pre-trained models. To build a robust vision system, we create a rich image dataset of date fruit bunches in an orchard that consists of more than 8000 images of five date types in different pre-maturity and maturity stages. The dataset has a large degree of variations that reflects the challenges in the date orchard environment including variations in angles, scales, illumination conditions, and date bunches covered by bags. The proposed date fruit classification models achieve accuracies of 99.01%, 97.25%, and 98.59% with classification times of 20.6, 20.7, and 35.9 msec for the type, maturity, and harvesting decision classification tasks, respectively.

Formative assessment framework proposal for transversal competencies: Application to analysis and problem-solving competence

Purpose: In the last years, there is an increasing interest in the manner that transversal competences (TC) are introduced in the curricula. Transversal competences are generic and relevant skills that students have to develop through the several stages of the educational degrees. This paper analyses TCs in the context of the learning process of undergraduate and postgraduate courses. The main aim of this paper is to propose a framework to improve results. The framework facilities the student's training and one of the important pieces is undoubtedly that he has constant feedback from his assessments that allowing to improve the learning. An applying in the analysis and problem solving competence in the context of Master Degree in Advanced Engineering Production, Logistics and Supply Chain at the UPV is carried out.Design/methodology/approach: The work is the result of several years of professional experience in the application of the concept of transversal competence in the UPV with undergraduate and graduate students. As a result of this work and various educational innovation projects, a team of experts has been created, which has been discussing some aspects relevant to the improvement of the teaching-learning process. One of these areas of work has been in relation to the integration of various proposals on the application and deployment of transversal competences. With respect to this work, a conceptual proposal is proposed that has subsequently been empirically validated through the analysis of the results of several groups of students in a degree.Findings: The main result that is offered in the work is a framework that allows identifying the elements that are part of the learning process in the area of transversal competences. Likewise, the different items that are part of the framework are linked to the student's life cycle, and a temporal scope is established for their deployment.Practical implications: One of the most noteworthy practical implications is that the proposed framework includes a tool that allows a clear measurement of the student's evolution throughout his / her formative life cycle. In this way the student has a more consistent and robust vision of his / her training and the academic directors of the titles can have a vision of the impact of the decisions on the learning processes.Originality/value: The analysis of transversal competences is usually presented in the context of a subject. In this paper we propose an approach to cross-curricular competences but in the scope of the student's complete life cycle. The consideration of the entire formative process as well as the identification of the relevant elements that are part of this process are the most original aspects of the work.

A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard

Reliable and robust vision algorithms to detect the cutting points on peduncles of overlapping grape clusters in the unstructured vineyard are essential for efficient use of a harvesting robot. In this study, we designed an approach to detect these cutting points in three main steps. First, the areas of pixels representing grape clusters in vineyard images were obtained using a segmentation algorithm based on k-means clustering and an effective color component. Next, the edge images of grape clusters were extracted, and then a geometric model was used to obtain the contour intersection points of double overlapping grape clusters. Profile analysis was used to separate the regional pixels of double grape clusters by a line connecting double intersection points. Finally, the region of interest of the peduncle for each grape clusters was determined based on the geometric information of each pixel region, and a computational method was used to determine the appropriate cutting point on the peduncle of each grape cluster by use of a geometric constraint method. Thirty vineyard images that were captured from different perspectives were tested to validate the performance of the presented approach in a complex environment. The average recognition accuracy was 88.33%, and the success rate of visual detection of the cutting point on the peduncle of double overlapping grape clusters was 81.66%. The demonstrated performance of this developed method indicated that it could be used by harvesting robots.

Robust vision sensor for multi-point displacement monitoring of bridges in the field

Computer vision sensors have great potential for accurate remote displacement monitoring in the field. This paper presents InnoVision, a video image processing technique developed by the authors to address a number of difficulties associated with the application of the vision sensors to monitoring structural displacement responses in the outdoor condition that are rarely comprehensively studied in literatures. First, limited lighting condition in the field presents a challenge to tracking low contrast features on the structural surface using intensity-based template matching algorithms. For tackling this challenge, a gradient based template matching algorithm is formulated. Second, to cost-effectively monitor structural displacements at multiple points using one camera, widely used interpolation subpixel methods are investigated and incorporated into InnoVision. Third, camera vibration in the field causes displacement measurement errors. A practical solution is proposed by applying the multi-point monitoring to track both the structure and a stationary reference point. The effect of the camera vibration can be canceled by subtracting the reference displacement from the structural displacements. Several laboratory and field tests are conducted to evaluate the InnoVision’s performance. One of the field tests is conducted in a challenging low lighting condition at night on a steel girder bridge to validate the robustness of InnoVision in comparison with two other vision sensing methods. Another field test is carried out on the Manhattan Bridge to demonstrate the efficacy of the proposed technique for canceling camera vibration and the capability of InnoVision to simultaneously monitor multiple points under the effect of camera vibration.

Robust industrial vision system for mechanical parts recognition

Robust industrial vision system for mechanical parts recognition

Fast and Precise Glass Handling Using Visual Servo With Unscented Kalman Filter Dual Estimation

In the protective glass manufacturing industry for cell phones, placing glass pieces into the slots of the grinder requires submillimeter accuracy which only can be achieved by human workers, leading to a bottle neck in the production line. To address such issue, industrial robot equipped with vision sensors is proposed to support human workers. The high placing performance is achieved by a two step approach. In the first step, an eye-to-hand camera is installed to detect the glass piece and slot with robust vision, which can put the glass piece close to the slot and ensures a primary precision. In the second step, a closed-loop controller based on visual servo is adopted to guide the glass piece into the slot with dual eye-in-hand cameras. However, vision sensor suffers from a very low frame rate and slow image processing speed resulting in a very slow placing performance. In addition, the placing performance is substantially limited by the system parameter uncertainty. To compensate for these limitations, a dual-rate unscented Kalman filter (UKF) with dual-estimation is adopted for sensor data filtering and online parameter identification without requiring any linear parameterization of the model. Experimental results are presented to confirm the effectiveness of the proposed approach.

Visual Detection and Two-Dimensional Rotation Control in Zebrafish Larva Heart Microinjection

Zebrafish larva heart microinjection has emerged as an important technology that enables delivery of a wide variety of vehicles into hearts of small individuals. However, traditional manual microinjection is laborious and time consuming, and most existing automated injection systems can be costly or complicated to assemble. In this paper, a semiautomated and simple-structure vision servosystem of Zebrafish larva heart microinjection is presented. A robust vision recognition algorithm is developed to locate heart of anesthetized larvae with different postures. A rolling model that utilizes slice figures of larva body is constructed to estimate larva heart height. The two-dimensional rotation control that positions and orients larva by a motorized translational stage and a glass capillary is also proposed to meet the requirement of successful holding. Experimental results show that the developed visual detection method can achieve a success rate of 100% when larva heart is towards left or right and 90% in other postures. The rotation control approach also achieves a high success rate of 97% with rotation resolution of 0.5 ${}^{\circ}$ . As a high-efficiency and high-cost-performance microinjection system, our system has a potential application in other organs injection of Zebrafish larvae.

Does philosophy contribute to an invasion complex? Sloterdijk the antagonist and the agonism of Mouffe

ABSTRACTThe political backlash against multiculturalism alongside the media portrayal of the global refugee crisis would suggest that the spaces for cultural difference have contracted and moved into a mode of transnational crisis management. This article addresses the moral panic over cultural difference by challenging some of the philosophical frameworks that have justified naturalized negative attitudes towards migrants and dismissed the viability of cosmopolitan perspectives. In particular, the author will critically evaluate the antagonistic perspective developed in Peter Sloterdijk’s writings and Chantal Mouffe’s theory of agonism. To grasp the complex and hybrid forms of cross-cultural exchanges, the author argues that a more robust vision of cosmopolitanism is necessary.

Stereo vision during adverse weather — Using priors to increase robustness in real-time stereo vision

Stereo vision can deliver a dense 3D reconstruction of the environment in real-time for driver assistance as well as autonomous driving. Semi-Global Matching (SGM) is a popular method of choice for solving this task which is already in use for production vehicles. Despite the enormous progress in the field and the high level of performance of modern stereo methods, one key challenge remains: robust stereo vision in automotive scenarios during rain, snow and darkness. Under these circumstances, current methods generate strong temporal noise, many disparity outliers and false positives on object level. These problems are addressed in this work by regularizing stereo vision via prior information. We formulate a temporal prior and a scene prior, which we apply to SGM in order to overcome the deficiencies. The temporal prior integrates knowledge from the previous disparity map to exploit the high temporal correlation, the scene prior exploits knowledge of a representative traffic scene. Using these priors, the object detection rate improves significantly on a driver assistance dataset of 3000 frames including bad weather while reducing the rate of erroneous object detections. We also outperform the ECCV Robust Vision Challenge 2012 winner, iSGM, on this dataset. In addition, results are presented for the KITTI dataset, even showing improvements under good weather conditions when exploiting the temporal prior.We also show that the temporal and scene priors are easy and efficient to implement on a hybrid CPU/reconfigurable hardware platform. The use of these priors can be extended to other application areas such as mobile robotics.

High-Speed Target Tracking System Based on a Hierarchical Parallel Vision Processor and Gray-Level LBP Algorithm

Visual target tracking has made significant advances in past decades. However, fast and robust vision target tracking systems are still greatly demanded. This paper proposes a novel high-speed target tracking system based on hierarchical parallel vision processor architecture. This system contains three main parts: 1) a CMOS image sensor; 2) a vision processor; and 3) an actuator with two degrees of freedom. The vision processor integrates a pixel-parallel processing element (PE) array, a row-parallel row processor (RP) array, dual-core microprocessor unit (MPU) and motor controller. The PE array and RP array can speed up low-level and middle-level image processing operations by ${O(M^{2})}$ and ${O}$ ( ${M}$ ), respectively. The MPU is responsible for the high-level image processing and the overall chip management. A novel tracking algorithm based on a gray-level local binary pattern descriptor is proposed. The descriptor describes not only local texture feature but also distribution of luminance. The algorithm increases the robustness of the tracking system under low resolution scenery and complex background. It can be carried out by the vision processor with very high efficiency. Experiment results demonstrate that the system can track a fast moving target under complex conditions and the vision processor can achieve over 2000 frames/s processing speed of the target tracking algorithm with $ {750\times 480}$ image resolution.