Uncontrolled Imaging Conditions Research Articles

Coupled image and kernel prior learning for high-generalized super-resolution

Deep convolution neural networks (DCNNs) have demonstrated great success on single image super resolution, where most existing methods aim to construct a deep model in an fully-supervised way using large amount of synthetic image pairs under ideal degradation assumption. The super-resolution performance would be significantly degraded for the low-resolution images captured under uncontrolled imaging conditions with complicated degradation procedures. To handle the above limitations, this work proposes a high-generalized image super-resolution framework by synergistically learning the latent image and the degradation process to achieve the specific priors for an under-studying observation in an unsupervised manner. Specifically, we incorporate dual branches of networks to configure our framework, where one is structured with both convolution and transformer blocks to learn local-to-global prior to generate high-quality image while the other aims to learn the kernel prior with a simple convolution-based architecture. The kernel prior subnet is pre-trained to enhance the stability of the joint optimization for the overall unsupervised learning procedure. With the estimated degradation kernel and target image, we produce the approximated low-resolution version with a convolution-based degradation block to formulate the loss function for the specific learning. Moreover, to pursue visually plausible image generation, we further incorporate a perceptual loss by leveraging a pre-trained discriminator in Gan-based SR model. Extensive experiments on several benchmark datasets have demonstrated the effectiveness of our proposed high-generalized super-resolution model, manifesting superiority over both supervised and unsupervised state-of-the-art SR models.

Zero-Shot Blind Learning for Single-Image Super-Resolution

Deep convolutional neural networks (DCNNs) have manifested significant performance gains for single-image super-resolution (SISR) in the past few years. Most of the existing methods are generally implemented in a fully supervised way using large-scale training samples and only learn the SR models restricted to specific data. Thus, the adaptation of these models to real low-resolution (LR) images captured under uncontrolled imaging conditions usually leads to poor SR results. This study proposes a zero-shot blind SR framework via leveraging the power of deep learning, but without the requirement of the prior training using predefined imaged samples. It is well known that there are two unknown data: the underlying target high-resolution (HR) images and the degradation operations in the imaging procedure hidden in the observed LR images. Taking these in mind, we specifically employed two deep networks for respectively modeling the priors of both the target HR image and its corresponding degradation kernel and designed a degradation block to realize the observation procedure of the LR image. Via formulating the loss function as the approximation error of the observed LR image, we established a completely blind end-to-end zero-shot learning framework for simultaneously predicting the target HR image and the degradation kernel without any external data. In particular, we adopted a multi-scale encoder–decoder subnet to serve as the image prior learning network, a simple fully connected subnet to serve as the kernel prior learning network, and a specific depthwise convolutional block to implement the degradation procedure. We conducted extensive experiments on several benchmark datasets and manifested the great superiority and high generalization of our method over both SOTA supervised and unsupervised SR methods.

Robust beam width estimation with sub-pixel resolution in uncontrolled imaging conditions.

Estimations of laser beam widths can yield vastly different results depending on the used beam width measurement method and the measurement conditions. We expand upon an imaging analysis method, proposed in the field of laser eye safety, to determine a laser's beam width based on the ratio between encircled energy and diameter. We show this method to be highly valuable when determining beam widths in noisy imaging conditions and in the presence of background illumination. Furthermore, this method enables estimations with sub-pixel resolution on beam widths with a spatial extent of a few pixels.

Segmentation of backscattered electron images of geopolymers using convolutional autoencoder network

Segmentation of backscattered electron (BSE) images of cementitious materials is often used to quantify different microstructural features for the sake of performance estimation at macro-scale levels. However, the heterogeneous nature of cementitious matrices compounds with varying imaging conditions can lead the conventional segmentation methods to a processing bottleneck for largescale experiments. To overcome these challenges, in this study, we evaluate the potential of deep autoencoder convolutional networks, specifically SegNet, for automatic segmentation of fly ash-based geopolymer images. We present the SegNet power in achieving a comparable accuracy to the human performance even with a few BSE images in the model’s training. The SegNet demonstrates magnification independent training that adapts itself with both seen and unseen magnification levels. A comparative study shows that SegNet outperforms the Gaussian method on uncontrolled imaging conditions such as background brightness levels. In addition, we demonstrate the self-learning capability of SegNet in poorly annotated areas.

An intelligent machine vision-based smartphone app for beef quality evaluation

Beef tenderness is the most important attribute correlated with beef quality, consumer satisfaction, and purchasing decisions. Nowadays, a rapid, non-invasive, and non-destructive evaluation and prediction of beef tenderness and quality from fresh product attributes is desired in industries, laboratories, and markets dealing with beef handling, processing, analyzing, and buy and sell. In this study, a new machine vision-based smartphone app was developed and verified for the first time in order to predict beef tenderness from fresh beef image captured under uncontrolled conditions. In order to eliminate the effects of uncontrolled imaging conditions, an illumination-, rotation-, scale-, and translation-invariant image processing algorithm was developed so that a common user can easily capture the image of the beef sample with more degree of freedom in terms of luminance, rotation, scale, and translation with no worries about the accuracy of the results. The obtained preprocessed image textural features were well correlated with instrumental data obtained using Warner-Bratzler shear force measurement through artificial neural network technique. The developed android app was installed on a LG G4 H815 smartphone and its performance was assessed using thirty unseen beef samples. The probability of occurrence of 2-D correlation coefficients obtained from the analyses of all the beef samples subjected to the image processing algorithm showed the average probability of 0.92, which strongly supported the robustness of the developed algorithm. The best obtained neural network model could predict the tenderness values with mean absolute percentage error (MAPE) of 3.28% and coefficient of determination (R2) of 0.97. The app promisingly predicted the beef tenderness values of the unseen samples with mean squared error (MSE) of 3.34, MAPE of 3.74%, and R2 of 0.99. Accordingly, the developed app can be a low-cost and user-friendly tool for predicting beef tenderness and quality from its real-world image.

Markov random fields and facial landmarks for handling uncontrolled images of face sketch synthesis

Face sketch synthesis has drawn great attention in many computer vision applications such as law enforcement and digital entertainment. The majority of existing face sketch synthesis techniques are exemplar-based techniques, where a set of training photo–sketch pairs are first divided into patches. For an input photo patch from the face to be synthesized, k similar photo patches are found from the training set. The corresponding sketch patch of the best match is then selected to be synthesized. In such techniques, a multiscale Markov random fields (MRF) model is utilized for synthesizing a sketch using candidate sketch patches; having observed that techniques tend to fail with face photos acquired in uncontrolled imaging conditions like pose and lighting variations. For example, some structures along the lower part of the face sketch contour get lost due to ignoring the global face shape information and illumination changes. In this paper, we propose a reliable face sketch synthesis method based on MRF model and facial landmarks, called MRF-FL that can maintain further structures with uncontrolled face photos. Besides matching the input photo with training photo, the input photo and training sketch are also matched based on the facial landmarks so as to enhance face sketch structures around the lower part of face sketch contour. Experimental results showed that the proposed MRF-FL achieves superior performance compared with recent face sketch synthesis methods on CUHK and AR face sketch databases.

Image Alignment by Piecewise Planar Region Matching

Robust image registration is a challenging problem, especially when dealing with severe changes in illumination and viewpoint. Previous methods assume a global geometric model (e.g., homography) and, hence, are only able to align images under predefined constraints (e.g., planar scenes and parallax-free camera motion). However, these constraints may not hold for natural scenes and uncontrolled imaging conditions. Therefore, this paper proposes a novel method which approximates image regions with planes by incorporating piecewise local geometric models. The approximated planar regions are obtained by exploiting a hierarchical figure-ground segmentation method. Each such planar region assumes an affine transformation. To achieve the alignment of the planar regions, an energy function is defined which employs intensity, a key-point descriptor, and geometric information under a global constraint. By re-segmenting and re-merging planar regions iteratively in an energy minimization framework, the method is able to align images even under significant changes in illumination and viewpoint. Experiments on two datasets show that the proposed method outperforms state-of-the-art, especially in the case of large appearance variations and it is, therefore, applicable to web-images (i.e., unconstrained setting) which are taken from the same scene with different viewpoints.

Detecting siblings in image pairs

In everyday life, face similarity is an important kinship clue. Computer algorithms able to infer kinship from pairs of face images could be applied in forensics, image retrieval and annotation, and historical studies. So far, little work in this area has been presented, and only one study, using a small set of low quality images, tackles the problem of identifying siblings pairs. The purpose of our paper is to present a comprehensive investigation on this subject, aimed at understanding which are, on the average, the most relevant facial features, how effective can be computer algorithms for detecting siblings pairs, and if they can outperform human evaluation. To avoid problems due to low quality pictures and uncontrolled imaging conditions, as for the heterogeneous datasets collected for previous researches, we prepared a database of high quality pictures of sibling pairs, shot in controlled conditions and including frontal, profile, expressionless, and smiling faces. Then we constructed various classifiers of image pairs using different types of facial data, based on various geometric, textural, and holistic features. The classifiers were first tested separately, and then the most significant facial data, selected with a two stage feature selection algorithm were combined into a unique classifier. The discriminating ability of the automatic classifier combining features of different nature has been found to outperform that of a panel of human raters. We also show the good generalization capabilities of the algorithm by applying the classifier, in a cross-database experiment, to a low quality database of images collected from the Internet.

High Accuracy and Visibility-Consistent Dense Multiview Stereo

Since the initial comparison of Seitz et al., the accuracy of dense multiview stereovision methods has been increasing steadily. A number of limitations, however, make most of these methods not suitable to outdoor scenes taken under uncontrolled imaging conditions. The present work consists of a complete dense multiview stereo pipeline which circumvents these limitations, being able to handle large-scale scenes without sacrificing accuracy. Highly detailed reconstructions are produced within very reasonable time thanks to two key stages in our pipeline: a minimum s-t cut optimization over an adaptive domain that robustly and efficiently filters a quasidense point cloud from outliers and reconstructs an initial surface by integrating visibility constraints, followed by a mesh-based variational refinement that captures small details, smartly handling photo-consistency, regularization, and adaptive resolution. The pipeline has been tested over a wide range of scenes: from classic compact objects taken in a laboratory setting, to outdoor architectural scenes, landscapes, and cultural heritage sites. The accuracy of its reconstructions has also been measured on the dense multiview benchmark proposed by Strecha et al., showing the results to compare more than favorably with the current state-of-the-art methods.