Optical Vision System Research Articles

Tree-PointNet: A Novel Neural Network for Dormant Apple Trees Using Semi-Circle-Based Point Cloud for Robotic Pruning

HighlightsTree-PointNet, a novel neural network, integrates geometric down-sampling for efficient pruning.A Semi-Circle-Based point cloud dataset of apple trees was collected to aid in automated pruning.Enhanced algorithm performance was investigated, showing significant improvements in model accuracy. Abstract. Pruning during the dormant season is one of the most costly and labor-intensive operations in the production of specialty crops. In winter, many trained seasonal workers have to carefully remove branches from hundreds of trees according to pre-defined rules. The aim of automated pruning is to reduce this reliance on the large amount of labor currently required for the operation. The first step in the automatic application of pruning rules is the acquisition of point cloud data from trees. To match the optical vision system of an ideal automatic pruning robot, a brand new and challenging apple tree dataset was collected in which the point clouds were all based on semicircular shapes, rather than the carefully crafted point clouds of complete apple trees. 1000 Semi-Circle-Based point clouds of apple trees were used. For the brand-new structure of point cloud part segmentation, a neural network model (namely Tree-PointNet) was proposed. A Semi-Circle-Based point cloud of apple tree was segmented by the Tree-PointNet with the segmentation result consisting of a trunk and primary branches. With a weight size of only 7.9% of PointStack, the best OA of 95.6% was obtained by the Tree-PointNet network proposed in this paper. The GDS was also used, which could obtain more key points in the Branch-to-Trunk connection region as input to the proposed network. On the other hand, the OA and mIoU of the model improved by 10.2% and 7.98%, respectively, compared to the PointNet++. Keywords: Dormant apple trees, Part segmentation, PointNet++, Semi-circle-based point cloud.

Исследование телескопической системы с трехступенчатой сменой увеличения и стабильным положением выходного зрачка

The visual optical systems with three fields of view are considered in the article. The removal of the exit pupil of the considered systems is from 1/3 to 1/4 of its total length. The problems that arise during a three-stage change of magnification are considered in the paper. An important feature of such systems is the need to match the pupils of the system when changing the magnification to ensure the stability of the position of the exit pupil. The complexity of the implementation while observing weight and size restrictions are also discussed. A comprehensive solution to these problems should be laid at the circuit level and should be based on the search for a fundamental structural optical scheme. During the design process, the issues of modeling and determining the area of solutions are revealed, which make it possible to ensure high image quality in the transition to finite apertures and thicknesses. The scientific hypothesis is that a three-stage change of magnification can be realized by presence of the rotary afocal system of magnification change. At the same time, for such a solution, internal alignment and competitive characteristics of the optical system of the product as a whole are also provided. The purpose of this article is to present the results of the analysis of the circuit solution in the paraxial approximation, to determine the areas of solutions that provide the smallest relative apertures of the circuit components under the introduced overall restrictions on the length of the system, the diameters of the components and the removal of the exit pupil. The results of the study are confirmed by the development of an optical system with interchangeable magnifications of 1x, 3x and 9x with an eye relief of 80 mm with the total system length of 250 mm.

RETRACTED ARTICLE: Underwater image segmentation based on computer vision and research on recognition algorithm

Due to the continuous growth of the world’s population, the development and utilization of marine resources have received great attention. At present, marine fishing relies heavily on divers for underwater operations, which have the disadvantages of high risk, low efficiency, and high cost. Therefore, the development of underwater capture robot which can automatically detect, locate, and capture targets is of great significance for the development of marine economy. Underwater robot is an indispensable equipment for deep-sea operation and plays an irreplaceable role in the development of the ocean. When autonomous underwater vehicles perform underwater operations, they can use computer vision system to obtain clear underwater images and accurate target category information, which can help the manipulator select different grasping parts for different shapes and categories and improve work efficiency. The current underwater vision technology includes “acoustic vision” and “light vision.” Due to the influence of multichannel effect and blind area, the acoustic vision research in detecting and tracking underwater targets is not deep enough. Compared with the acoustic image processing system, the underwater optical vision system has the advantages of image and video capture and has higher real-time performance, which can aim at the target faster and more conveniently. Underwater vision optical system plays an important leading role in the detailed research of underwater vehicle sensing system, which can further improve the autonomous performance of underwater vehicle. In addition, considering the nature of underwater image imaging, we are developing an underwater image segmentation and recognition system based on image processing.

Optimization of Underwater Marker Detection Based on YOLOv3

The research on the detection and recognition technology of marker is of great significance for some underwater operations, such as marine resource exploration, underwater robot operation and so on. The existing image processing methods can effectively detect and recognize the markers in the air. Nevertheless, in the underwater environment, the complex imaging environment of the ocean leads to serious degradation of underwater images obtained by the optical vision system. Due to the lack of effective information for object recognition, the severely degraded underwater images increases the difficulty of detection and recognition of underwater objects. With the development of high-tech underwater imaging equipment, the quality of underwater images has been improved to a certain extent, but there are still some phenomena such as color fading, low contrast and blurred details. Solutions to overcome these problems are important for the exploration of the ocean. In this paper, we introduce a deep learning model to optimize the performance of detection, and make a unique marker dataset for the application scene of our experiment. We first use the deep learning network to pre-train the marker images in the air. Next, we use the underwater marker images for fine-tuning. Finally, after the target marker is detected, the traditional image processing method is used to recognize the marker. Experimental results show that the optimization method we proposed achieves better performance on the dataset.

Discomfort of Visual Perception in Virtual and Mixed Reality Systems

The importance of the topic of this paper is caused by the rapid development of virtual, mixed, and augmented reality systems. A problem in designing virtual reality (VR), augmented reality (AR) and mixed reality (MR) systems is the assessment of the visual perception of the result, in particular, the assessment of discomfort caused either by differences in the illumination conditions of objects in the real and virtual worlds or by the conflict of the vergence and accommodation of vision of the device user. The standard optical characteristics, such as aberration, frequency and contrast characteristic, and others are not applicable to evaluating the discomfort of visual perception. On the other hand, the discomfort can be assessed using a real proptotype; however, its fabrication can be labor consuming and expensive. We propose methods for virtual prototyping of visual optical systems that can be used in designing AR, VR, and MR devices for the analysis of discomfort caused, for example, by the vergence–accommodation conflict in virtual and augmented reality systems or by differences in the illumination conditions of real and virtual scene objects in mixed reality systems.

Robust PCANet on target recognition via the UUV optical vision system

Inspired by the importance of preserve the robustness of algorithm, in this paper, a simple and effective underwater target recognition algorithm called robust principal component analysis network (RPCANet) is established, which has further enhanced the operational ability of unmanned underwater vehicle (UUV) with optical vision systems. Several advantages of RPCANet are summarized as follows. First, RPCANet is superior to traditional underwater target recognition methods, such as principal component analysis (PCA), because it is not sensitive to severe outliers. Second, RPCANet employs ....Fp.... -norm as the distance metric for obtaining a filter. The proposed method retains the desirable properties of the PCA, which includes a solution related to the covariance matrix and rotational invariance. In addition, RPCANet is a simple deep learning network used for image recognition and classification. Finally, the performance of the proposed method is illustrated by video image data collected by the optical vision system of the UUV. The experimental results show the advantages and effectiveness of proposed recognition scheme.

Target recognition based on dynamic (2D)2PCA for UUV optical vision system

In this paper, a new method coined dynamic two-directional two-dimensional principal component analysis (D(2D)2PCA) was established, which has further developed the feature expression of the video image data collected by the optical vision system of unmanned underwater vehicle (UUV). The contributions are as follows: (1) D(2D)2PCA is also based on the dynamic characteristics of image data in addition to being relied on the static relationship among data; (2) An image covariance matrix that directly uses the sample image data in the dynamic time series augmented matrix is constructed, and the projection vector for image feature extraction is derived; (3) To test and performance estimation of D(2D)2PCA, a series of experiments were performed on the video image data collected by underwater optical vision system. Experimental results show that, compared with the traditional underwater target recognition method, D(2D)2PCA has more effective feature extraction ability and higher target recognition accuracy, which improves the autonomous underwater operation ability of UUV.

OVERVIEW OF REMOTE SENSING APPLICATION FOR SEARCH OF MH370

This paper summarizes some remote sensing technologies in use by different search and rescue operators for the search of the missing Malaysia Airline (MH370). Studies are on-going using detected products and complex imagery. Some of the detected imagery currently does not provide reliable information, though the techniques themselves are promising. The introduction of microwave remote sensing systems will provide the much needed assistance when both visual searches and optical imaging systems do not help. However, under normal atmospheric conditions, airborne or space borne imaging systems are developed to obtain further information. The paper looked into the airplanes, flight data recorder and cockpit voice recorder, as well as the possible path.

X-ray and optical stereo-based 3D sensor fusion system for image-guided neurosurgery.

In neurosurgery, an image-guided operation is performed to confirm that the surgical instruments reach the exact lesion position. Among the multiple imaging modalities, an X-ray fluoroscope mounted on C- or O-arm is widely used for monitoring the position of surgical instruments and the target position of the patient. However, frequently used fluoroscopy can result in relatively high radiation doses, particularly for complex interventional procedures. The proposed system can reduce radiation exposure and provide the accurate three-dimensional (3D) position information of surgical instruments and the target position. X-ray and optical stereo vision systems have been proposed for the C- or O-arm. Two subsystems have same optical axis and are calibrated simultaneously. This provides easy augmentation of the camera image and the X-ray image. Further, the 3D measurement of both systems can be defined in a common coordinate space. The proposed dual stereoscopic imaging system is designed and implemented for mounting on an O-arm. The calibration error of the 3D coordinates of the optical stereo and X-ray stereo is within 0.1 mm in terms of the mean and the standard deviation. Further, image augmentation with the camera image and the X-ray image using an artificial skull phantom is achieved. As the developed dual stereoscopic imaging system provides 3D coordinates of the point of interest in both optical images and fluoroscopic images, it can be used by surgeons to confirm the position of surgical instruments in a 3D space with minimum radiation exposure and to verify whether the instruments reach the surgical target observed in fluoroscopic images.

Mesoscale Robotic Assembly With Orthogonal Optical Alignment for UV-LIGA Parts

This paper proposes an assembly system for ultraviolet-lithographie galvanik abformung (UV-LIGA) parts with a robotic manipulator. Both images of base part and object part could be obtained simultaneously from an in-house orthogonal optical alignment vision system. Two microgrippers were introduced to realize the reliable clamping. An initial calibration method was presented to ensure assembly accuracy. Assembly experiments were conducted with success rates of 80% and the time consumption of 20 min for all four parts assembly. Suspected causes of failure are motion mechanisms' uncertainty, part dislocation resulted from inertia force when microgripper is moving, and the error which is produced in the detection process because of random factors.

Data Registration Method Based on Three Dimensional Target

Data registration method using special three dimensional target to track the structured light measurement system is discussed. Optical scanning device, tracking target and stereo vision system are integrated together to fulfill profile inspection of large-scale free-form surface objects without extra mark points. System architecture and processing steps are introduced and layout optimization methods of three dimensional target are illustrated. Experimental results are showed to evaluate the validity of the registration method and suggests are given to improve the accuracy of the system.

Optical transfer properties of external channels and image isoplanarity in vision systems

Some aspects of the application of the linear systems approach and the method of the Green’s functions to the theory of vision in scattering media are refined. The physical meaning of the influence function and the optical transfer function of scattering media and vision systems are discussed. It is stated that the physical meaning of the concepts of the optical transfer function and the impulse response (point spread function or influence function) of scattering media or an imaging channel in scattering media is uncertain if the reflection (emission) properties of the object plane (surface) are undetermined. The main differences between the characteristics of linear systems for optical imaging and vision systems are emphasized. A criterion is suggested for estimating the size of the peripheral image isoplanarity zones in vision systems.

Virtual microscopy http://vmic.unibas.ch/

Triangulation laser rangefinders using a laser sheet of light and a video camera are one of the most popular active optical vision systems for short range (indoor) autonomous robot applications. Conventional implementations of this system are based on digitising the video signal which is output from the camera, and processing the digitised images in order to obtain the points of incidence of the reflection of the laser sheet of light on the objects in front of the robot on the camera, from which the range information can be directly acquired. In this paper we propose a system is based on direct hardware detection of the pixels that receive maximum illumination (i.e. those corresponding to the intersection points between the laser beam and the objects in the scene). In this way, only the co-ordinates of these pixels are fed into the vision computer, and range information is obtained in a straightforward manner, without any need to digitise full images. Experimental results and performance tests of a vision system prototype based on this idea, are presented in the paper.

112 Development of a Sub-pixel Optical Vision System for Precision Tool Setting

Precise and efficient tool setting technique and accurate tool shape monitoring are of essential importance in ultra-precision diamond turning operation. The traditional way of tool setting are typically laborious, inefficient and rely heavily on experience. A big part of the tool setting is done by using a contact probe such as LVDT. The contact tool setting station can normally, depending on the resolution of the probes, place the tool tip to within a 1∿10μm positioning accuracy. However, it is running the risk of damage the delicate tool tip and has the ambiguity introduced by contact point of tool and touch probe. The optical/non-contact way of setting the tool do have the advantage of not having to touch the tool, but its resolution is limited by the optical diffraction limit and the resolution of the CCD device used (mm/pixel). A non-contact precision tool setting system is developed and built in this study using edge-detection image processing and sub-pixel dividing techniques in conjunction with CNC controller of the precision turning machine to improve the system presently available. The results showed that tools of different shapes namely round, half-round and sharp tool could all be positioned to within an error band of ±0.1μm by using the developed tool setting system.

Extraction of roughness properties from captured images of surfaces

Surfaces of industrial parts need to be specified on the basis of their use and application environment. Hence, surface characterization is vital for design, manufacturing and inspection. Furthermore, the development of new industries has led to a requirement for super-smooth surfaces and for the ability to measure such surfaces accurately; therefore, the measurement of engineering surface roughness is becoming increasingly important. Many methods of measuring surface finish have been developed ranging from the simple touch comparator to sophisticated optical techniques (light scattering). The latest technology for measuring surface roughness is the computer vision systems. Unlike the stylus instruments, the optical techniques and computer vision systems have the advantages of being non-contact and are capable of measuring an area from the surface rather than a single line. This paper introduces a new technique to capture an image for a surface and then to extract the roughness heights from the captured image. A set-up consists of a charge-coupled device (CCD) video colour camera, a frame grabber, IBM-compatible PC and a microscope used to capture the images. The images were stored using the bitmap image file format (BMP). A program was written in Visual C++ to convert the captured colour images into grey scale images and then to extract the grey level for each pixel and to store it into a two-dimensional array sized to the image width and the image length. The program is capable of reading six different image file formats: these are BMP, TIFF, GIF, PCX, JPG and WMF. The grey levels range between 0 and 255. The grey level 0 represents the black colour, which represents the minimum height in the image, and the grey level 255 represents the white colour, which represents the maximum height in the image. The grey levels between 0 and 255 represent different heights according to their grey level values. After extraction of the grey levels from the image, a three-dimensional view is displayed to view the topography of the surface. The three-dimensional view can be displayed, virtually, from any point of view using the same basics as those of three-dimensional computer aided design (CAD) systems. From the three-dimensional surface topography it is easy to assess any scratches or cracks in the surface. The developed software works independently and does not need any other software. The package includes the unique feature of calculating a multitude of surface roughness parameters (59 in all) that have not been included in any other package hitherto.

Visible-band testbed projector with a replicated diffractive optical element

Raytheon has designed, fabricated, and tested a diffractive-optical-element-based (DOE-based) testbed projector for direct and indirect visual optical applications. By use of a low-cost replicated DOE surface from Rochester Photonics Corporation for color correction the projector optics bettered the modular transfer function of an equivalent commercial camera lens. The testbed demonstrates that a practical DOE-based optical system is suitable for both visual applications (e.g., head-mounted displays) and visual projection (e.g., tactical sensors). The need for and the proper application of DOE's in visual optical systems, the nature and the performance of the projector optical design, and test results are described.

A New Laser Triangulation Processor for Mobile Robot Applications: Preliminary Results

Triangulation laser rangefinders using a laser sheet of light and a video camera are one of the most popular active optical vision systems for short range (indoor) autonomous robot applications. Conventional implementations of this system are based on digitising the video signal which is output from the camera, and processing the digitised images in order to obtain the points of incidence of the reflection of the laser sheet of light on the objects in front of the robot on the camera, from which the range information can be directly acquired. In this paper we propose a system is based on direct hardware detection of the pixels that receive maximum illumination (i.e. those corresponding to the intersection points between the laser beam and the objects in the scene). In this way, only the co-ordinates of these pixels are fed into the vision computer, and range information is obtained in a straightforward manner, without any need to digitise full images. Experimental results and performance tests of a vision system prototype based on this idea, are presented in the paper.

Relative Positions of the Thermal Convection Column and Smoke Plume Generated from a Smoldering Cigarette

Abstract The relative positions of the thermal convection column and the smoke plume from a variety of smoldering cigarettes were measured using a combination of schlieren and visual optical systems. The schlieren technique is an optical method used to observe refractive index gradients in gases and other clear media. The refractive index gradients can be caused by variations in pressure, composition or temperature. The convection column of heated air and combustion gases rising from a cigarette coal was observed with a two mirror schlieren system. A video camera was used as the observing device rather than the usual photographic camera. A second video camera was arranged to view the smoke plume rising from the coal region. The two video images were combined with a video special effects generator and were viewed on a single monitor. The behaviour and relative positions of both columns were thus observed in real time with two non-invasive optical methods. The schlieren images of the convection column were compared to those of model systems such as a heated cylinder and a small flame. Results for experimental cigarettes with paper porosities of 12 cm/min to 48 cm/min (Coresta) during smolder showed that the thermal convection column was centered 2 mm to 3 mm in front of the paper char line directly over the hottest part of the coal as determined by infra-red imaging. The smoke plume was centered 2 mm behind the paper char line and the position did not change with paper porosity. Results for experimental cigarettes made with a commercially available low sidestream paper showed that the position of the convection column did not change. However, the position of the smoke plume changed considerably. In addition to being markedly decreased in visibility, the plume now appeared to be centered directly over the paper char line. This change in position provides a valuable insight into the mechanism of smoke reduction. The low sidestream papers seem to work by preventing the escape of smoke forming condensibles through the paper behind the paper char line. The condensibles are therefore forced to escape at the paper char line and undergo increased combustion and pyrolysis. As a result, some are converted to lower molecular weight materials and are unable to condense as readily to form the smoke. In addition, the remaining condensibles are released into a hotter and faster rising gas stream. This serves to reduce smoke formation by suppressing condensation and increasing dilution.

Transformation of image positions, rotations, and sizes into shift parameters

An optical image processing system is described that converts orientation and size to shift properties and simultaneously preserves the positional information as a shift. The system is described analytically and experimentally. The transformed image can be processed further with a classical correlator working with a rotational and size-invariant. multiplexed match filter. An optical robot vision system designed on this concept would be able to look at several objects simultaneously and determine their shape, size, orientation, and position with two measurements on the input scene at different rotation angles.

Aberration tolerances for visual optical systems.

The effects of aberrations on images formed by a total system, including the eye, are assessed in terms of modulation threshold, grating detectability, and modulation transfer functions. The effects of approximately one and two waves of third-order spherical aberration, coma, and astigmatism on the total system modulation transfer function are investigated using three-bar resolution targets and Ronchi rulings in two separate psychophysical experiments. The results of both experiments indicate that the experimentally determined system modulation transfer function is closely approximated by the theoretically calculated modulation transfer function of the total system optics (dioptrics included) in the presence of instrument aberrations. The data indicate that the eye accomodates of a "best focus" when spherical aberration and astigmatism (but not coma) are present in the instrument. The quantitative data are used to determine aberration tolerances for instruments that are designed for specific visual tasks and that employ simple targets similar to those investigated.