Linear Image Processing Research Articles

Analog image processing with nonlinear nonlocal flat optics

Digital signal processing has revolutionized many fields of science and engineering, but it still shows critical limits, mainly related to the complexity, power consumption, and limited speed of analogue-to-digital converters. A long-sought solution to overcome these hurdles is optical analog computing. In this regard, flat optics has been recently unveiled as a powerful platform to perform data processing in real-time, with low power consumption and a small footprint. So far, these explorations have been mainly limited to linear optics. Arguably, significantly more impact may be garnered from pushing this operation towards nonlinear processing of the incoming signals. In this context, we demonstrate here that nonlinear phenomena combined with engineered nonlocality in flat optics devices can be leveraged to synthesize Volterra kernels able to outperform linear optical analog image processing.

Opening and closing in morphological image processing

Morphological Image Processing is a useful tool in digital image processing because it can rigorously quantify many features of the geometrical structure of a way that is consistent with human intuition and perception. Morphological processing may remove noise and clutter from images, as well as alter them according to the size and shape of the items of interest. Linear Image Processing is sometimes used instead of Morphological Image Processing since it changes the underlying geometric form of an image, although Morphological Image Processing does not lose the image's information.

High-contrast phase imaging based on nonlinear holographic hot image model

Imaging for weak-phase objects is a challenging issue in the linear imaging process. Here, we demonstrate a high-contrast phase imaging method based on a nonlinear holographic hot image model. Due to the nonlinear Kerr effect, the holographic hot image can transform a weak phase into strong amplitude as a signal amplifier. The phase information is iteratively obtained from the light field distribution of the holographic hot image. The strong signal-to-noise ratio helps improve the imaging contrast. Using a tunable photorefractive crystal, we numerically and experimentally demonstrate the advantage of this method for imaging weak-phase objects. For the determined sample, our method doubles the imaging contrast. As far as we know, this is the first report using the nonlinear holographic hot image for imaging technology. This study can provide a potential strategy to achieve high-contrast imaging for various weak-phase objects applied in biomedical imaging, semiconductor metrology, and photolithography.

The area method for measuring acetabular cup anteversion: An accurate and autonomous solution

Several radiological methods of measuring anteversion of the acetabular component after total hip arthroplasty have been described, all time-consuming and with varying reproducibility. This study aimed to compare the recently proposed Area method to true cup anteversion as determined by an accelerometer. This study further applied this method programmatically to autonomously determine radiographic cup orientation using two computer programs, then compared these results to hand and accelerometer measurements. 160 anteroposterior pelvis radiographs were taken of a standard Sawbones® pelvis fitted with a total hip arthroplasty system. The acetabular cup was re-oriented between each radiograph, with anteversion ranging from 0° to 90°. An accelerometer was mounted to the cup to measure true cup anteversion. Radiographic anteversion was independently measured via three methods: by hand, linear image processing, and machine learning. Measurements were compared to triaxial accelerometer recordings. Coefficient of determination (R2) was found to be 0.997, 0.991, and 0.989 for hand measurements, the machine learning, and linear image processing, respectively. The machine learning program and hand measurements overestimated anteversion by 0.70° and 0.02° respectively. The program using linear techniques underestimated anteversion by 5.02°. Average runtime was 0.03 and 0.59 s for the machine learning and linear image processing program, respectively. The machine learning program averaged within 1° of cup orientation given a true cup anteversion less than 51°, and within 2° given an anteversion less than 85°. The Area method showed great accuracy and reliability with hand measurements compared to true anteversion. The results of this study support the use of machine learning for accurate, timely, autonomous assessment of cup orientation.

Limitations of CNNs for Approximating the Ideal Observer Despite Quantity of Training Data or Depth of Network.

The performance of a convolutional neural network (CNN) on an image texture detection task as a function of linear image processing and the number of training images is investigated. Performance is quantified by the area under (AUC) the receiver operating characteristic (ROC) curve. The Ideal Observer (IO) maximizes AUC but depends on high-dimensional image likelihoods. In many cases, the CNN performance can approximate the IO performance. This work demonstrates counterexamples where a full-rank linear transform degrades the CNN performance below the IO in the limit of large quantities of training data and network layers. A subsequent linear transform changes the images' correlation structure, improves the AUC, and again demonstrates the CNN dependence on linear processing. Compression strictly decreases or maintains the IO detection performance while compression can increase the CNN performance especially for small quantities of training data. Results indicate an optimal compression ratio for the CNN based on task difficulty, compression method, and number of training images.

SMOaaS: a Scalable Matrix Operation as a Service model in Cloud

Matrix operations are fundamental to a wide range of scientific applications such as Graph Theory, Linear Equation System, Image Processing, Geometric Optics, and Probability Analysis. As the workload in these applications has increased, the sizes of matrices involved have also significantly increased. Parallel execution of matrix operations in existing cluster-based systems performs effectively for relatively small matrices but significantly suffers as matrices become larger due to limited resources. Cloud Computing offers scalable resources to handle this limitation; however, the benefits of having access to almost-infinite scalable resources in the Cloud also come with challenges of ensuring time and resource-efficient matrix operations. To the best of our knowledge, there is no specific Cloud service that optimizes the efficiency of matrix operations on Cloud infrastructure. To address this gap and offer convenient service of matrix operations, the paper proposes a novel scalable service framework called Scalable Matrix Operation as a Service. Our framework uses Dynamic Matrix Partition techniques, based on matrix operation and sizes, to achieve efficient work distribution, and scales based on demand to achieve time and resource-efficient operations. The framework also embraces the basic features of security, fault tolerance, and reliability. Experimental results show that the adopted dynamic partitioning technique ensures faster and better performance when compared to the existing static partitioning technique.

CNN performance dependence on linear image processing

Generative adversarial networks (GANs) have been significantly investigated in the past few years due to its outstanding data generation capacity. The extensive use of the GANs techniques is dominant in the field of computer vision, for example, plausible image generation, image to image translation, facial attribute manipulation, improving image resolution, and image to text translation. In spite of the significant success achieved in these domains, applying GANs to various other problems still presents important challenges. Several reviews and surveys for GANs are available in the literature. However, none of them present short but focused review about the most significant aspects of GANs. In this paper, we address these aspects. We analyze the basic theory of GANs and the differences among various generative models. Then, we discuss the recent spectrum of applications covered by the GANs. We also provide an insight into the challenges and future directions.

OEC-cnn: a simple method for over-exposure correction in photographs

This work reports on convolutional neural network (CNN) performance on an image texture classification task as a function of linear image processing and number of training images. Detection performance of single and multi-layer CNNs (sCNN/mCNN) are compared to optimal observers. Performance is quantified by the area under the receiver operating characteristic (ROC) curve, also known as the AUC. For perfect detection AUC = 1.0 and AUC = 0.5 for guessing. The Ideal Observer (IO) maximizes AUC but is prohibitive in practice because it depends on high-dimensional image likelihoods. The IO performance is invariant to any fullrank, invertible linear image processing. This work demonstrates the existence of full-rank, invertible linear transforms that can degrade both sCNN and mCNN even in the limit of large quantities of training data. A subsequent invertible linear transform changes the images’ correlation structure again and can improve this AUC. Stationary textures sampled from zero mean and unequal covariance Gaussian distributions allow closed-form analytic expressions for the IO and optimal linear compression. Linear compression is a mitigation technique for high-dimension low sample size (HDLSS) applications. By definition, compression strictly decreases or maintains IO detection performance. For small quantities of training data, linear image compression prior to the sCNN architecture can increase AUC from 0.56 to 0.93. Results indicate an optimal compression ratio for CNN based on task difficulty, compression method, and number of training images.

Microfluidic-based linear-optics label-free imager.

Linear optics based nanoscopy previously reached resolution beyond the diffraction limit, illuminating samples in the visible light regime while allowing light to interact with freely moving metallic nanoparticles. However, the hydrodynamics governing the nanoparticle motion used to scan the sample is very complex and has low probability of achieving appropriate and fast mapping in practice. Hence, an implementation of the technique on real biological samples has not been demonstrated so far. Moreover, a suitable way to perform controlled nanoparticle scanning of biological samples is required. Here we show a solution where a microfluidic channel is used to flow and trap biological samples inside a water droplet along with suspended nanoparticles surrounded by silicone oil. The evanescent light scattered from the sample and is rescattered by the nanoparticles in the vicinity. This encodes the sub-wavelength features of the sample which can later on be decoded and reconstructed from measurements in the far field. The microfluidic system-controlled flow allows better nanoparticle scanning of the sample and maintains an isolated system for each sample in each droplet. A more localized scan at the droplet water/oil interface is also conducted using amphiphilic nanoparticles where their hydrophilic side is constrained to the droplet and their hydrophobic side is constrained to the oil. This allows higher probability of capturing evanescent fields closer to their origin, yielding better resolution and a higher signal to noise ratio. Using this system, we obtained images of an E. coli sample and demonstrated how the method yield fine resolution of the sample contours. To the best of our knowledge, this is the first time that a linear and label free optics imaging process was performed using a micro-fluidic device.

Multiplication of medium-density matrices using TensorFlow on multicore CPUs

Matrix multiplication is an essential part of many applications, such as linear algebra, image processing and machine learning. One platform used in such applications is TensorFlow, which is a machine learning library whose structure is based on dataflow programming paradigm. In this work, a method for multiplication of medium-density matrices on multicore CPUs using TensorFlow platform is proposed. This method, called tbt_matmul, utilizes TensorFlow built-in methods tf.matmul and tf.sparse_matmul. By partitioning each input matrix into four smaller sub-matrices, called tiles, and applying an appropriate multiplication method to each pair depending on their density, the proposed method outperforms the built-in methods for matrices of medium density and matrices of significantly uneven distribution of non-zeros.

Environmental and social-demographic predictors of the southern house mosquito Culex quinquefasciatus in New Orleans, Louisiana

BackgroundUnderstanding the major predictors of disease vectors such as mosquitoes can guide the development of effective and timely strategies for mitigating vector-borne disease outbreaks. This study examined the influence of selected environmental, weather and sociodemographic factors on the spatial and temporal distribution of the southern house mosquito Culex quinquefasciatus Say in New Orleans, Louisiana, USA.MethodsAdult mosquitoes were collected over a 4-year period (2006, 2008, 2009 and 2010) using CDC gravid traps. Socio-demographic predictors were obtained from the United States Census Bureau, 2005–2009 American Community Survey and the City of New Orleans Department of Code Enforcement. Linear mixed effects models and ERDAS image processing software were used for statistical analysis and image processing.ResultsOnly two of the 22 predictors examined were significant predictors of Cx. quinquefasciatus abundance. Mean temperature during the week of mosquito collection was positively associated with Cx. quinquefasciatus abundance while developed high intensity areas were negatively associated with Cx. quinquefasciatus abundance.ConclusionThe findings of this study illustrate the power and utility of integrating biophysical and sociodemographic data using GIS analysis to identify the biophysical and sociodemographic processes that increase the risk of vector mosquito abundance. This knowledge can inform development of accurate predictive models that ensure timely implementation of mosquito control interventions.

A Statistical Framework for Generalized Linear Image Processing Systems

Generalized linear image processing systems have been developed from physical image formation models, human visual perception models, and mathematical models. Although there have been many papers on the extension, parameterization, and symmetrization of some of these systems, what is lacking is a unified framework such that the development and study of such systems can be performed based on a common ground. In this paper, we propose a conceptual image sensor which models how the light energy is converted into the sensor data. In the proposed sensor model, the input energy is regarded as a random variable and the conversion is through the cumulative distribution function. Based on the sensor model, we suggest a statistical framework by which new systems can be derived, and existing and seemingly unrelated systems can be studied from a unified perspective. The proposed statistical framework not only provides a principled way to symmetrizing systems through the even extension of the probability distribution function (PDF) and a natural way for the parameterization of systems through parameters of PDF. In this paper, we demonstrate new applications of the statistical framework through a numerical approximation of the lower incomplete gamma function, through the enhancement of the dynamic range and manipulation of the sharpness of images by using the scalar multiplication operation of the parametric system, through an application of a new system in fusion of multi-exposure images, and through an application of the three new systems for the correction of incorrect exposure.

Enhanced learning-based imaging with thin-film luminescent concentrators.

LumiConSense, a transparent, flexible, scalable, and disposable thin-film image sensor has the potential to lead to new human-computer interfaces that are unconstrained in shape and sensing-distance. In this article we make four new contributions: (1) A new real-time image reconstruction method that results in a significant enhancement of image quality compared to previous approaches; (2) the efficient combination of image reconstruction and shift-invariant linear image processing operations; (3) various hardware and software prototypes which, realize the above contributions, demonstrating the current potential of our sensor for real-time applications; and finally, (4) a further higher quality offline reconstruction algorithm.

Effect of linear image processing on the depth of correlation in micro PIV

This work investigates the effect of selected linear image processing methods on the depth of correlation (DoC) in micro particle image velocimetry using a single camera. In practice, band-pass and high-pass filters (background subtraction) are commonly applied to micro particle image velocimetry images. This work provides analytical models describing the effect of the parameters of low-, high-, and band-pass filters on the DoC and verifies the models by experiments. Furthermore, we propose a scheme that allows computing the weighting function and DoC for more complicated cases numerically.

Feasibility study of a novel methodology for solar radiation prediction on an hourly time scale: A case study in Plymouth, United Kingdom

Accurate forecasting of renewable energies provides a suitable means for efficient and secure operation of electricity grids and enables optimal energy market planning. This paper presents a number of novel prediction models for solar radiation prediction; and their performances are compared to those of conventional prediction models such as Neural Networks (NNs). All the developed models are based on linear predictive coding principles and image processing fundamentals using one and two-dimensional (1-D & 2-D) Finite Impulse Response (FIR) filters. The studies of highly correlated solar radiations among different hours of a day and also among the identical hours of different days are important parts of the developed prediction models. The meteorological data was collected for the duration of May 1, 2011 to April 30, 2012 from the Casella automatic weather station at Plymouth, UK. To prove the efficiency of the proposed prediction models, some similar data are used for all of the comparing predictive algorithms. It was observed that the performance of the 2-D FIR filters were better than that of the 1-D representations and NNs. However, their prediction quality greatly depends on the filter size and their related optimal tap weights.

Linear interferometric image processing for analysis of a particle in a volume

A method of extracting the number of fringes/fringe spacing of compression interferogram based on a wavelet-matched filter and Fourier transform technique is proposed, and sub-pixel accuracy of the extracted frequency is achieved by the modified Rife algorithm. Both numerical simulation and experimental measurement are carried out to verify the performance of the method. The measurement uncertainty is ± 1.118 μm and the absolute error is 1.873 μm for standard particles of diameter 45 μm. The research results show that the algorithm presented in this paper is feasible, and is promising for application to a high-density spray field.

Parametric logarithmic type image processing for contrast based auto-focus in extreme lighting conditions

Abstract While most of state-of-the-art image processing techniques were built under the so-called classical linear image processing, an alternative that presents superior behavior for specific applications comes in the form of Logarithmic Type Image Processing (LTIP). This refers to mathematical models constructed for the representation and processing of gray tones images. In this paper we describe a general mathematical framework that allows extensions of these models by various means while preserving their mathematical properties. We propose a parametric extension of LTIP models and discuss its similarities with the human visual system. The usability of the proposed extension model is verified for an application of contrast based auto-focus in extreme lighting conditions. The closing property of the named models facilitates superior behavior when compared with state-of-the-art methods.

A High-Frequency Multipeak Model for Wide-Angle SAR Imagery

A new modeling method for representing distributed scattering centers in wide-angle synthetic aperture radar (SAR) is presented. The proposed multipeak model approximates amplitudes of localized image peaks that typically appear at a single pixel or as an in-line set of pixels in a SAR image. In this way, the multipeak model is an improvement over existing peak models which poorly represent distributed canonical scatterers, such as the common dihedral with a fold line oriented parallel to the imaging plane. The model is derived from a wide-angle approximation of the well-known attributed scattering center or parametric models when under the action of a linear imaging operator. It is shown that, under typical imaging conditions of 10 <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$^{ \circ}$</tex></formula> or more in the synthetic aperture, the multipeak model approximates the image peak amplitudes due to distributed canonical scatterers as if they are due to an equivalent point scatterer with an azimuth-independent dispersive amplitude function in the spectral domain. This improves parameter estimation and scatterer classification, and it is also shown that the imaging relative error due to the approximation is less than 2% for other common image processing conditions such as tapered windowing in azimuth and when the canonical scatterer is at least ten wavelengths in size. A distinct advantage of the multipeak model over point scatterer models is that parameter estimation and scatterer classification can be performed solely in the spatial domain on a pixel-by-pixel basis and efficiently integrated within a linear SAR imaging process. To illustrate the benefits and limitations of the approach, parameter estimation and scatterer classification experiments are presented using simulated SAR data.

A novel compiler support for automatic parallelization on multicore systems

Abstract The widespread use of multicore processors is not a consequence of significant advances in parallel programming. In contrast, multicore processors arise due to the complexity of building power-efficient, high-clock-rate, single-core chips. Automatic parallelization of sequential applications is the ideal solution for making parallel programming as easy as writing programs for sequential computers. However, automatic parallelization remains a grand challenge due to its need for complex program analysis and the existence of unknowns during compilation. This paper proposes a new method for converting a sequential application into a parallel counterpart that can be executed on current multicore processors. It hinges on an intermediate representation based on the concept of domain-independent kernel (e.g., assignment, reduction, recurrence). Such kernel-centric view hides the complexity of the implementation details, enabling the construction of the parallel version even when the source code of the sequential application contains different syntactic variations of the computations (e.g., pointers, arrays, complex control flows). Experiments that evaluate the effectiveness and performance of our approach with respect to state-of-the-art compilers are also presented. The benchmark suite consists of synthetic codes that represent common domain-independent kernels, dense/sparse linear algebra and image processing routines, and full-scale applications from SPEC CPU2000.

Theory and application of general linear image processing

Image modeling in terms of physical/mathematical formation and the human visual system plays an important role in developing new image processing techniques. Parameters such as scale, orientation, texture, morphology, and color must be taken into consideration in order to solve complicated image processing problems efficiently. Although the classical linear image processing (CLIP) approach has played a central role in image processing, it is not necessarily the best and even the right choice. General linear image processing (GLIP) approaches based on different image modeling techniques have been studied to overcome some of the problems associated with CLIP. Indeed, using general linear algebra, it is possible to develop entirely new general linear operators and transforms. It is even possible to define entirely new general linear operations (addition, scalar multiplication, convolution, etc.) in order to describe how images are combined, amplified, transformed, analyzed, compared, and measured.