Computer Vision Tools Research Articles

An Analysis on Very Deep Convolutional Neural Networks: Problems and Solutions

Neural Networks have become a powerful tool in computer vision because of the recent breakthroughs in computation time and model architecture. Very deep models allow for better deciphering of the hidden patterns in the data; however, training them successfully is not a trivial problem, because of the notorious vanishing/exploding gradient problem. We illustrate this problem on VGG models, with 8 and 38 hidden layers, on the CIFAR100 image dataset, where we visualize how the gradients evolve during training. We explore known solutions to this problem like Batch Normalization (BatchNorm) or Residual Networks (ResNets), explaining the theory behind them. Our experiments show that the deeper model suffers from the vanishing gradient problem, but BatchNorm and ResNets do solve it. The employed solutions slighly improve the performance of shallower models as well, yet, the fixed deeper models outperform them.

MACHINE LEARNING APPLICATION: IMAGE PROCESSING

-In the current scenario of the data world, the data holds significant information if processed correctly. The data can be in the form of images which can prove to be a boon in deriving the useful insights from it in order to get the knowledge of things at an early stage itself. But the matter of concern is deriving the information from the images will be a tedious task for human beings and would incur a heavy cost and time. So, an easy and cheaper technique is to teach a machine efficiently to do the task for us. The concept of using Machines to do human tasks is known as Machine Learning. In this paper, I present various literature reviews regarding image processing in Machine learning and how image processing has helped in identifying the issues at early stages so that they can be resolved easily without causing much harm. Also, image processing has been a helpful tool in computer vision.

Computer Vision Tagging the Metropolitan Museum of Art's Collection

Computer vision algorithms are increasingly being applied to museum collections to identify patterns, colors, and subjects by generating tags for each object image. There are multiple off-the-shelf systems that offer an accessible and rapid way to undertake this process. Based on the highlights of the Metropolitan Museum of Art's collection, this article examines the similarities and differences between the tags generated by three well-known computer vision systems (Google Cloud Vision, Amazon Rekognition, and IBM Watson). The results provide insights into the characteristics of these taxonomies in terms of the volume of tags generated for each object, their diversity, typology, and accuracy. In consequence, this article discusses the need for museums to define their own subject tagging strategy and selection criteria of computer vision tools based on their type of collection and tags needed to complement their metadata.

Skin Lesions Detection and Classification Using Deep Learning

The skin lesion is considered the most widespread malignant disease in individuals, and melanoma is the deadliest form of the disease. Early detection affects the prognosis of the disease and improves the chances of survival. Dermatologists use scientific calculation tools such as ABCD to diagnose melanoma through visual inspection of the mole. However, computer vision tools have been presented to support the quantitative scrutiny of skin lesions. Significant improvements to deep learning algorithms in image recognition tasks should be very successful in medical image examination, particularly in the classification of skin lesions used to diagnose melanoma. In this research, a deep learning simulation with 38 layers to detect and classify skin lesions was proposed. Two datasets were used for training and testing i.e., the HAM10000 dataset & the ISIC2019 dataset. Experimental results show that the model outperforms on both the datasets hence making it nondependent of the dataset. 94.45% of validation top 3 accuracies are achieved on the HAM10000 dataset & 93.06% of validation top 3 accuracies are achieved on the ISIC2019 dataset.

Using Cameras To Understand What Is Happening While Drilling

Can a camera do a better job than humans at creating a minute-by-minute record of drilling activity? Or evaluate damaged drill bits better? Based on two papers at the SPE/IADC International Drilling Conference and Exhibition, the answer might soon be yes. Schlumberger researchers showed that video from a camera on the drill floor and on the drilling cuttings shaker kept an accurate record of what the rig was doing at the time, also known as the rig state (SPE 204086). A paper from The University of Texas-Austin (UT-Austin) showed a camera phone could create images for use in their bit-evaluation program to assess the damage and offer a reasonable explanation for what caused it (SPE 204124). A year ago Teradata announced its Bit-Box, which houses cameras and lights to create 3D bit images. It is now on the market; other companies are working on competing products. Cameras have gotten small and cheap at a time when innovations like self-driving cars have driven tech companies to speed development of computer-vision tools. Schlumberger’s goal is to convert video from rig cameras into a constant record of the rig state. It is being done with data gathered by sensors while drilling, or by people if the sensors stop working or generate bad data. The system leaves gaps when sensor issues are missed or workers are otherwise occupied, which is a maddening problem for those who need to know what is going on at any moment to interpret drilling data. “We need to understand what the rig was doing when we look at and try to solve problems” using digital analysis, said Crispin Chatar, a drilling subject matter expert for Schlumberger, who presented the paper at the virtual conference. The bit-evaluation program used phone camera images as a lower-cost substitute for 3D imaging, like the system sold by Teradata. Mobile phone images come with the risk of bad lighting and varying angles, which the authors said affected the analysis. But those devices are small, and there are plenty of backups available. “Everybody has a mobile phone at a rig site,” said Pradeep Ashok, a research scientist in the petroleum engineering department at UT-Austin. He believes cameras on mobile devices are the future because “if you could build something that requires no added equipment it will allow more rapid adoption.” Mobile phone owners also worry about breaking their phone. They may not be so concerned about rig equipment, which is a concern to those selling more-complex imaging devices. Both projects were early efforts to test what can be done with camera data, which is likely to expand over time. Schlumberger’s goal of tracking the rig state more reliably and cheaper than when using sensors may not be enough to convince users to change their ways at a time when sensors are proliferating to support more digitally controlled activities. The Schlumberger data were gathered at a test well for an automated rig.

Edge detection algorithm for omnidirectional images, based on superposition laws on Blach’s sphere and quantum entropy

This paper presents an edge detection algorithm for omnidirectional images based on superposition law on Bloch’s sphere and quantum local entropy. Omnidirectional vision system has become an essential tool in computer vision, duo to its large field of view. However, classical image processing algorithms are not suitable to be applied directly in this type of images without taking into account the spatial information around each pixel. To show the performance of the proposed method, a set of experimentation was done on synthetic and real images devoted to agriculture applications. Later, Fram & Deutsh criterion has been adopted to evaluate its performance against three algorithms proposed on the literature and developed for omnidirectional images. The results show a good performance of the proposed method in term of edge quality, edge community and sensibility to noise.

Behavioral Biometric Data Analysis for Gender Classification Using Feature Fusion and Machine Learning.

Biometric security applications have been employed for providing a higher security in several access control systems during the past few years. The handwritten signature is the most widely accepted behavioral biometric trait for authenticating the documents like letters, contracts, wills, MOU’s, etc. for validation in day to day life. In this paper, a novel algorithm to detect gender of individuals based on the image of their handwritten signatures is proposed. The proposed work is based on the fusion of textural and statistical features extracted from the signature images. The LBP and HOG features represent the texture. The writer’s gender classification is carried out using machine learning techniques. The proposed technique is evaluated on own dataset of 4,790 signatures and realized an encouraging accuracy of 96.17, 98.72 and 100% for k-NN, decision tree and Support Vector Machine classifiers, respectively. The proposed method is expected to be useful in design of efficient computer vision tools for authentication and forensic investigation of documents with handwritten signatures.

Development and Validation of Automated Visual Field Report Extraction Platform Using Computer Vision Tools

Purpose: To introduce and validate hvf_extraction_script, an open-source software script for the automated extraction and structuring of metadata, value plot data, and percentile plot data from Humphrey visual field (HVF) report images.Methods: Validation was performed on 90 HVF reports over three different report layouts, including a total of 1,530 metadata fields, 15,536 value plot data points, and 10,210 percentile data points, between the computer script and four human extractors, compared against DICOM reference data. Computer extraction and human extraction were compared on extraction time as well as accuracy of extraction for metadata, value plot data, and percentile plot data.Results: Computer extraction required 4.9-8.9 s per report, compared to the 6.5-19 min required by human extractors, representing a more than 40-fold difference in extraction speed. Computer metadata extraction error rate varied from an aggregate 1.2-3.5%, compared to 0.2-9.2% for human metadata extraction across all layouts. Computer value data point extraction had an aggregate error rate of 0.9% for version 1, <0.01% in version 2, and 0.15% in version 3, compared to 0.8-9.2% aggregate error rate for human extraction. Computer percentile data point extraction similarly had very low error rates, with no errors occurring in version 1 and 2, and 0.06% error rate in version 3, compared to 0.06-12.2% error rate for human extraction.Conclusions: This study introduces and validates hvf_extraction_script, an open-source tool for fast, accurate, automated data extraction of HVF reports to facilitate analysis of large-volume HVF datasets, and demonstrates the value of image processing tools in facilitating faster and cheaper large-volume data extraction in research settings.

A critical comparison analysis between human and machine-generated tags for the Metropolitan Museum of Art's collection

PurposeBased on the highlights of The Metropolitan Museum of Art's collection, the purpose of this paper is to examine the similarities and differences between the subject keywords tags assigned by the museum and those produced by three computer vision systems.Design/methodology/approachThis paper uses computer vision tools to generate the data and the Getty Research Institute's Art and Architecture Thesaurus (AAT) to compare the subject keyword tags.FindingsThis paper finds that there are clear opportunities to use computer vision technologies to automatically generate tags that expand the terms used by the museum. This brings a new perspective to the collection that is different from the traditional art historical one. However, the study also surfaces challenges about the accuracy and lack of context within the computer vision results.Practical implicationsThis finding has important implications on how these machine-generated tags complement the current taxonomies and vocabularies inputted in the collection database. In consequence, the museum needs to consider the selection process for choosing which computer vision system to apply to their collection. Furthermore, they also need to think critically about the kind of tags they wish to use, such as colors, materials or objects.Originality/valueThe study results add to the rapidly evolving field of computer vision within the art information context and provide recommendations of aspects to consider before selecting and implementing these technologies.

Application of a Computer Vision Tool for Automated Glottic Tracking to Vocal Fold Paralysis Patients.

(1) Demonstrate true vocal fold (TVF) tracking software (AGATI [Automated Glottic Action Tracking by artificial Intelligence]) as a quantitative assessment of unilateral vocal fold paralysis (UVFP) in a large patient cohort. (2) Correlate patient-reported metrics with AGATI measurements of TVF anterior glottic angles, before and after procedural intervention. Retrospective cohort study. Academic medical center. AGATI was used to analyze videolaryngoscopy from healthy adults (n = 72) and patients with UVFP (n = 70). Minimum, 3rd percentile, 97th percentile, and maximum anterior glottic angles (AGAs) were computed for each patient. In patients with UVFP, patient-reported outcomes (Voice Handicap Index 10, Dyspnea Index, and Eating Assessment Tool 10) were assessed, before and after procedural intervention (injection or medialization laryngoplasty). A receiver operating characteristic curve for the logistic fit of paralysis vs control group was used to determine AGA cutoff values for defining UVFP. Mean (SD) 3rd percentile AGA (in degrees) was 2.67 (3.21) in control and 5.64 (5.42) in patients with UVFP (P < .001); mean (SD) 97th percentile AGA was 57.08 (11.14) in control and 42.59 (12.37) in patients with UVFP (P < .001). For patients with UVFP who underwent procedural intervention, the mean 97th percentile AGA decreased by 5 degrees from pre- to postprocedure (P = .026). The difference between the 97th and 3rd percentile AGA predicted UVFP with 77% sensitivity and 92% specificity (P < .0001). There was no correlation between AGA measurements and patient-reported outcome scores. AGATI demonstrated a difference in AGA measurements between paralysis and control patients. AGATI can predict UVFP with 77% sensitivity and 92% specificity.

RETRACTED: Design of embedded digital image processing system based on ZYNQ

Zynq- real time edge detection system based on execution of the image reconstructed from the sequence. Although the object edge detection is an essential tool in computer vision, edge detection results in the negative image frame noise significantly. Further, due to the high computational complexity of an image filtering operation, implemented in the hardware configuration of the re-configurable hardware it is necessary. In this portion, the proposed dynamic Zynq embedded system reconfiguration capability to detect according to the input image frame to the noise level of the different density filters perform during runtime bit stream re Configuration.The results show that the accuracy of edge detection is highly evaluated. We analyze various noise density levels, and the results show that the edge detection results of the proposed filter bit stream are more accurate, while effectively providing computing capacity to support the real-time processing of the image frame. In the configuration time, CPU usage, and hardware resource utilization performance test results were compared.

An assistive computer vision tool to automatically detect changes in fish behavior in response to ambient odor

The analysis of fish behavior in response to odor stimulation is a crucial component of the general study of cross-modal sensory integration in vertebrates. In zebrafish, the centrifugal pathway runs between the olfactory bulb and the neural retina, originating at the terminalis neuron in the olfactory bulb. Any changes in the ambient odor of a fish’s environment warrant a change in visual sensitivity and can trigger mating-like behavior in males due to increased GnRH signaling in the terminalis neuron. Behavioral experiments to study this phenomenon are commonly conducted in a controlled environment where a video of the fish is recorded over time before and after the application of chemicals to the water. Given the subtleties of behavioral change, trained biologists are currently required to annotate such videos as part of a study. This process of manually analyzing the videos is time-consuming, requires multiple experts to avoid human error/bias and cannot be easily crowdsourced on the Internet. Machine learning algorithms from computer vision, on the other hand, have proven to be effective for video annotation tasks because they are fast, accurate, and, if designed properly, can be less biased than humans. In this work, we propose to automate the entire process of analyzing videos of behavior changes in zebrafish by using tools from computer vision, relying on minimal expert supervision. The overall objective of this work is to create a generalized tool to predict animal behaviors from videos using state-of-the-art deep learning models, with the dual goal of advancing understanding in biology and engineering a more robust and powerful artificial information processing system for biologists.

Shoulder kinematics plus contextual target information enable control of multiple distal joints of a simulated prosthetic arm and hand

BackgroundProsthetic restoration of reach and grasp function after a trans-humeral amputation requires control of multiple distal degrees of freedom in elbow, wrist and fingers. However, such a high level of amputation reduces the amount of available myoelectric and kinematic information from the residual limb.MethodsTo overcome these limits, we added contextual information about the target’s location and orientation such as can now be extracted from gaze tracking by computer vision tools. For the task of picking and placing a bottle in various positions and orientations in a 3D virtual scene, we trained artificial neural networks to predict postures of an intact subject’s elbow, forearm and wrist (4 degrees of freedom) either solely from shoulder kinematics or with additional knowledge of the movement goal. Subjects then performed the same tasks in the virtual scene with distal joints predicted from the context-aware network.ResultsAverage movement times of 1.22s were only slightly longer than the naturally controlled movements (0.82 s). When using a kinematic-only network, movement times were much longer (2.31s) and compensatory movements from trunk and shoulder were much larger. Integrating contextual information also gave rise to motor synergies closer to natural joint coordination.ConclusionsAlthough notable challenges remain before applying the proposed control scheme to a real-world prosthesis, our study shows that adding contextual information to command signals greatly improves prediction of distal joint angles for prosthetic control.

Demonstration of background rejection using deep convolutional neural networks in the NEXT experiment

Convolutional neural networks (CNNs) are widely used state-of-the-art computer vision tools that are becoming increasingly popular in high-energy physics. In this paper, we attempt to understand the potential of CNNs for event classification in the NEXT experiment, which will search for neutrinoless double-beta decay in 136Xe. To do so, we demonstrate the usage of CNNs for the identification of electron-positron pair production events, which exhibit a topology similar to that of a neutrinoless double-beta decay event. These events were produced in the NEXT-White high-pressure xenon TPC using 2.6 MeV gamma rays from a 228Th calibration source. We train a network on Monte Carlo-simulated events and show that, by applying on-the-fly data augmentation, the network can be made robust against differences between simulation and data. The use of CNNs offers significant improvement in signal efficiency and background rejection when compared to previous non-CNN-based analyses.

Edge-Aware Filter Based on Adaptive Patch Variance Weighted Average

Edge-aware smoothing is an essential tool for computer vision, graphics and photography. In this paper, we develop a new and efficient local weighted average filter for edge-aware smoothing. The proposed filter can use guidance information which permits an iterative filtering process. Since the weights of the proposed filter depend on the local variance, the implementation requires linear filters only, leading to $\mathcal {O}(N_{pix})$ computational complexity. We also present statistical analysis and simulations which provide new insights into its computational efficiency and its relationship with the bilateral filter. The performance of the proposed filter is comparable to those state-of-the-art filters in many applications including: edge-preserving smoothing, compression artifact removal, structure separation, edge extraction, non-photo realistic image rendering, salience detection, detail magnification and multi-focus image fusion.

Cell Junction Mechanics beyond the Bounds of Adhesion and Tension.

Cell-cell junctions, in particular adherens junctions, are major determinants of tissue mechanics during morphogenesis and homeostasis. In attempts to link junctional mechanics to tissue mechanics, many have utilized explicitly or implicitly equilibrium approaches based on adhesion energy, surface energy, and contractility to determine the mechanical equilibrium at junctions. However, it is increasingly clear that they have significant limitations, such as that it remains challenging to link the dynamics of the molecular components to the resulting physical properties of the junction, to its remodeling ability, and to its adhesion strength. In this perspective, we discuss recent attempts to consider the aspect of energy dissipation at junctions to draw contact points with soft matter physics where energy loss plays a critical role in adhesion theories. We set the grounds for a theoretical framework of the junction mechanics that bridges the dynamics at the molecular scale to the mechanics at the tissue scale.

Toward a scalable framework for reproducible processing of volumetric, nanoscale neuroimaging datasets.

BackgroundEmerging neuroimaging datasets (collected with imaging techniques such as electron microscopy, optical microscopy, or X-ray microtomography) describe the location and properties of neurons and their connections at unprecedented scale, promising new ways of understanding the brain. These modern imaging techniques used to interrogate the brain can quickly accumulate gigabytes to petabytes of structural brain imaging data. Unfortunately, many neuroscience laboratories lack the computational resources to work with datasets of this size: computer vision tools are often not portable or scalable, and there is considerable difficulty in reproducing results or extending methods.ResultsWe developed an ecosystem of neuroimaging data analysis pipelines that use open-source algorithms to create standardized modules and end-to-end optimized approaches. As exemplars we apply our tools to estimate synapse-level connectomes from electron microscopy data and cell distributions from X-ray microtomography data. To facilitate scientific discovery, we propose a generalized processing framework, which connects and extends existing open-source projects to provide large-scale data storage, reproducible algorithms, and workflow execution engines.ConclusionsOur accessible methods and pipelines demonstrate that approaches across multiple neuroimaging experiments can be standardized and applied to diverse datasets. The techniques developed are demonstrated on neuroimaging datasets but may be applied to similar problems in other domains.

NZESPA: A Near-3D-Memory Zero Skipping Parallel Accelerator for CNNs

Convolutional neural networks (CNNs) are one of the most popular machine learning tools for computer vision. The ubiquitous use in several applications with its high computation-cost has made it lucrative for optimization through accelerated architecture. State-of-the-art has either exploited the parallelism of CNNs, or eliminated computations through sparsity or used near-memory processing (NMP) to accelerate the CNNs. We introduce NMP-fully sparse architecture, which acquires all three capabilities. The proposed architecture is parallel and hence processes the independent CNN tasks concurrently. To exploit the sparsity, the proposed system employs a dataflow, namely, Near-3D-Memory Zero Skipping Parallel dataflow or nZESPA dataflow. This dataflow maintains the compressed-sparse encoding of data that skips all ineffectual zero-valued computations of CNNs. We design a custom accelerator which employs the nZESPA dataflow. The grids of nZESPA modules are integrated into the logic layer of the hybrid memory cube. This integration saves a significant amount of off-chip communications while implementing the concept of NMP. We compare the proposed architecture with three other architectures which either do not exploit sparsity (NMP-dense) or do not employ NMP (traditional-fully sparse) or do not include both (traditional-dense). The proposed system outperforms the baselines in terms of performance and energy consumption while executing CNN inference.

“And We Will Fight for Our Race!” A Measurement Study of Genetic Testing Conversations on Reddit and 4chan

Progress in genomics has enabled the emergence of a booming market for “direct-to-consumer” genetic testing. Nowadays, companies like 23andMe and AncestryDNA provide affordable health, genealogy, and ancestry reports, and have already tested tens of millions of customers. At the same time, alt- and far-right groups have also taken an interest in genetic testing, using them to attack minorities and prove their genetic “purity.” In this paper, we present a measurement study shedding light on how genetic testing is being discussed on Web communities in Reddit and 4chan. We collect 1.3M comments posted over 27 months on the two platforms, using a set of 280 keywords related to genetic testing. We then use NLP and computer vision tools to identify trends, themes, and topics of discussion. Our analysis shows that genetic testing attracts a lot of attention on Reddit and 4chan, with discussions often including highly toxic language expressed through hateful, racist, and misogynistic comments. In particular, on 4chan's politically incorrect board (/pol/), content from genetic testing conversations involves several alt-right personalities and openly antisemitic rhetoric, often conveyed through memes. Finally, we find that discussions build around user groups, from technology enthusiasts to communities promoting fringe political views.

Detection and Location of Dead Trees with Pine Wilt Disease Based on Deep Learning and UAV Remote Sensing

Pine wilt disease causes huge economic losses to pine wood forestry because of its destructiveness and rapid spread. This paper proposes a detection and location method of pine wood nematode disease at a large scale adopting UAV (Unmanned Aerial Vehicle) remote sensing and artificial intelligence technology. The UAV remote sensing images were enhanced by computer vision tools. A Faster-RCNN (Faster Region Convolutional Neural Networks) deep learning framework based on a RPN (Region Proposal Network) network and the ResNet residual neural network were used to train the pine wilt diseased dead tree detection model. The loss function and the anchors in the RPN of the convolutional neural network were optimized. Finally, the location of pine wood nematode dead tree was conducted, which generated the geographic information on the detection results. The results show that ResNet101 performed better than VGG16 (Visual Geometry Group 16) convolutional neural network. The detection accuracy was improved and reached to about 90% after a series of optimizations to the network, meaning that the optimization methods proposed in this paper are feasible to pine wood nematode dead tree detection.