Elementary Image Research Articles

Stacking Deep-Learning Model, Stories and Drawing Properties for Automatic Scene Generation

Text-image mapping is of great interest to the scientific community, especially for educational purposes. It helps young learners, mainly those with learning difficulties, to better understand the content of stories. In this paper, we propose to capture the teacher’s experience in manually building relevant scenes for animal behavior stories. This manual work, which consists of a pair of texts and a set of elementary images, is fed into a Long Short-Term Memory (LSTM) followed by a Conditional Random Field (CRF) that aims to associate the relevant words in the text with their corresponding elementary image while preserving the drawing properties. This association is then used for scene construction. Several experiments were con-ducted to show how better the constructed scenes convey textual information than the scenes constructed from the competitor’s models.

4D Epanechnikov Mixture Regression in LF Image Compression

With the emergence of light field imaging in recent years, the compression of its elementary image array (EIA) has become a significant problem. Our coding framework includes modeling and reconstruction. For the modeling, the covariance-matrix form of the 4-D Epanechnikov kernel (4-D EK) and its correlated statistics were deduced to obtain the 4-D Epanechnikov mixture models (4-D EMMs). A 4-D Epanechnikov mixture regression (4-D EMR) was proposed based on this 4-D EK, and a 4-D adaptive model selection (4-D AMLS) algorithm was designed to realize the optimal modeling for a pseudo video sequence (PVS) of the extracted key-EIA. A linear function based reconstruction (LFBR) was proposed based on the correlation between adjacent elementary images (EIs). The decoded images realized a clear outline reconstruction and superior coding efficiency compared to high-efficiency video coding (HEVC) and JPEG 2000 below approximately 0.05 bpp. This work realized an unprecedented theoretical application by (1) proposing the 4-D Epanechnikov kernel theory, (2) exploiting the 4-D Epanechnikov mixture regression and its application in the modeling of the pseudo video sequence of light field images, (3) using 4-D adaptive model selection for the optimal number of models, and (4) employing a linear function-based reconstruction according to the content similarity.

Deep networks may capture biological behavior for shallow, but not deep, empirical characterizations

We assess whether deep convolutional networks (DCN) can account for a most fundamental property of human vision: detection/discrimination of elementary image elements (bars) at different contrast levels. The human visual process can be characterized to varying degrees of “depth,” ranging from percentage of correct detection to detailed tuning and operating characteristics of the underlying perceptual mechanism. We challenge deep networks with the same stimuli/tasks used with human observers and apply equivalent characterization of the stimulus–response coupling. In general, we find that popular DCN architectures do not account for signature properties of the human process. For shallow depth of characterization, some variants of network-architecture/training-protocol produce human-like trends; however, more articulate empirical descriptors expose glaring discrepancies. Networks can be coaxed into learning those richer descriptors by shadowing a human surrogate in the form of a tailored circuit perturbed by unstructured input, thus ruling out the possibility that human–model misalignment in standard protocols may be attributable to insufficient representational power. These results urge caution in assessing whether neural networks do or do not capture human behavior: ultimately, our ability to assess “success” in this area can only be as good as afforded by the depth of behavioral characterization against which the network is evaluated. We propose a novel set of metrics/protocols that impose stringent constraints on the evaluation of DCN behavior as an adequate approximation to biological processes.

3-D Epanechnikov Mixture Regression in integral imaging compression

Integral imaging is a kind of 3D display with no glasses, which represents the future developments. Elementary image array (EIA) is an essential component of integral imaging. Our coding framework includes pre-processing, modeling, and reconstruction. We acquire the sub-EIA from the original EIA and get the offsets between adjacent elementary images (EIs) through pre-processing. As for modeling, we get the optimal combination of 3-D Epanechnikov Mixture Regression (3-D EMR) or 3-D Gaussian Mixture Regression (3-D GMR) by Elementary Image Adaptive Model Selection (EI-AMLS) algorithm to achieve the best modeling of sub-EIA. Finally, the linear-based reconstruction is completed according to the correlation between adjacent EIs. Our decoded images realize a clearer outline reconstruction and more superior coding efficiency than HEVC and JPEG2000 below about 0.05bpp. Furthermore, the proposed method can achieve the same visual effect as HEVC with only 15% to 80% time consumed.

Multi-Block Color-Binarized Statistical Images for Single-Sample Face Recognition.

Single-Sample Face Recognition (SSFR) is a computer vision challenge. In this scenario, there is only one example from each individual on which to train the system, making it difficult to identify persons in unconstrained environments, mainly when dealing with changes in facial expression, posture, lighting, and occlusion. This paper discusses the relevance of an original method for SSFR, called Multi-Block Color-Binarized Statistical Image Features (MB-C-BSIF), which exploits several kinds of features, namely, local, regional, global, and textured-color characteristics. First, the MB-C-BSIF method decomposes a facial image into three channels (e.g., red, green, and blue), then it divides each channel into equal non-overlapping blocks to select the local facial characteristics that are consequently employed in the classification phase. Finally, the identity is determined by calculating the similarities among the characteristic vectors adopting a distance measurement of the K-nearest neighbors (K-NN) classifier. Extensive experiments on several subsets of the unconstrained Alex and Robert (AR) and Labeled Faces in the Wild (LFW) databases show that the MB-C-BSIF achieves superior and competitive results in unconstrained situations when compared to current state-of-the-art methods, especially when dealing with changes in facial expression, lighting, and occlusion. The average classification accuracies are 96.17% and 99% for the AR database with two specific protocols (i.e., Protocols I and II, respectively), and 38.01% for the challenging LFW database. These performances are clearly superior to those obtained by state-of-the-art methods. Furthermore, the proposed method uses algorithms based only on simple and elementary image processing operations that do not imply higher computational costs as in holistic, sparse or deep learning methods, making it ideal for real-time identification.

Continuity Detection Method in Binary Image Signal

This paper is focused on a description of a new method of detecting continuities in a binary image. The detecting method is called “Binary Large Object” (BLOB). A new algorithm with other elementary parameters, as processing speed and memory capability, are described here. The developed BLOB method with described algorithms is implemented in MATLAB. The simulation of the algorithms is tested in different conditions, with the time dependences determination. The research results of the computing time or the BLOB memory demand during computation are presented as well. The developed BLOB method is usable as one of the elementary image processing methods of the field detection dependencies in the images, like pattern recognition or OCR algorithm.

An image-based gangrene disease classification

Gangrene disease is one of the deadliest diseases on the globe which is caused by lack of blood supply to the body parts or any kind of infection. The gangrene disease often affects the human body parts such as fingers, limbs, toes but there are many cases of on muscles and organs. In this paper, the gangrene disease classification is being done from the given images of high resolution. The convolutional neural network (CNN) is used for feature extraction on disease images. The first layer of the convolutional neural network was used to capture the elementary image features such as dots, edges and blobs. The intermediate layers or the hidden layers of the convolutional neural network extracts detailed image features such as shapes, brightness, and contrast as well as color. Finally, the CNN extracted features are given to the Support Vector Machine to classify the gangrene disease. The experiment results show the approach adopted in this study performs better and acceptable.

Simple, contrast based, compartmentalization in 3D reconstruction from CT images

We are interested in different procedures and techniques to preprocess and postprocess the data in computedtomography reconstruction methods to achieve the better contrast, resolution or other types of the functionalanalysis of the data. We report the straightforward application of the imaging artifacts treating technique,that is the elimination of the overexposure artifact due to the presence of the metal object, for producingthe elementary image compartmentalization results with the artificial bone sample CT data. Thus, the highabsorption artifacts removal technique is given the complimentary function of the structural analysis andrevealing the metal pins scaffolding skeleton. The basic multilevel thresholding is used to reveal thetargeted structures. The results could be extremely useful for the noisy, unfiltered X-ray source, single rundata sets. The data have been supplied by the homemade CT scanner assembly and processed with thecustom reconstruction software developed specifically for this setup. OpenCV package tools for the C/C++libraries compiled with the MS Visual Studio IDE compiler were extensively used.

Visual Perception Entropy Measure

This paper is about a theoretical framework on perceptual reproduction quality assessment. Visual perception is the main theme for two approaches on image metrics: visual performance and image quality. Visual performance looks at the perception of standard elementary images, such as gratings, by different observers or by a single observer in different visual conditions; image quality looks at the perception of variations of the same complex scene by a standard observer. In the audiovisual field, all the configurations approved by the creative team must be reproduced in the final viewing. This cannot be measured by visual performance or image quality indexes. A third approach is needed, which may be called perceptual reproduction quality assessment. Visual perception assessment is not measuring defects, per se; it is built to evaluate the perceptual effectiveness of a given scene coding through a given reproduction technology. It is only dependent on the number of configurations being perceived through a given theater system relative to the various configurations reproduced and approved in the review room. As such, it is a statistical measure. It is, in fact, similar to the definition of relative entropy. One important fact is that this evaluation must include a model of visual perception. This paper illustrates how relative entropy and entropy loss can be estimated, while taking into account the coding transform, the projector performances, maximum light, and the characteristics of the human visual system. This scheme works easily for monochromatic content. Color, trichromacy, is quite challenging. The size of configurations goes from 2 12 to 2 36 , around 6 billion configurations. Calculation and memory requirements are huge. Nonlinearities are adding complexity, leading to calculation of an upper limit only of entropy loss. The presentation of results is also a challenge. One way to solve this is to separately process lightness and chromatic information. In summary, the main arguments in this paper are: 1) besides image quality and visual performance, there is a need for perceptual reproduction quality assessment; 2) entropy is the proper statistical estimator for evaluating the perceptual reproduction quality; and 3) color reproduction quality estimation requires processing lightness and chromatic attributes separately.

High-efficiency subwavelength bridge-shaped photo-absorbers based on transverse cavity modes.

We study a frequency-selective photon absorber able to efficiently confine light within very thin semiconducting layers. The active material is appropriately embedded in a cavity resonator where lateral resonances occur, based on nonplasmonic modes, thus minimizing undesired losses in the metallic parts. Typical examples and configurations in the infrared and visible ranges are illustrated. Theoretical quantum efficiencies around 90% may be reached, with a broad angular tolerance. Also, multispectral photon sorters may be designed by combining resonators of different sizes. These results especially find direct applications for elementary ultracompact detectors and imaging focal plane arrays.

Fast and robust phase-shift estimation in two-dimensional structured illumination microscopy

A method of determining unknown phase-shifts between elementary images in two-dimensional Structured Illumination Microscopy (2D-SIM) is presented. The proposed method is based on the comparison of the peak intensity of spectral components. These components correspond to the inherent structured illumination spectral content and the residual component that appears from wrongly estimated phase-shifts. The estimation of the phase-shifts is carried out by finding the absolute maximum of a function defined as the normalized peak intensity difference in the Fourier domain. This task is performed by an optimization method providing a fast estimation of the phase-shift. The algorithm stability and robustness are tested for various levels of noise and contrasts of the structured illumination pattern. Furthermore, the proposed approach reduces the number of computations compared to other existing techniques. The method is supported by the theoretical calculations and validated by means of simulated and experimental results.

天基合成孔径激光雷达成像理论初步

天基合成孔径激光雷达成像理论初步

The empirical characteristics of human pattern vision defy theoretically-driven expectations.

Contrast is the most fundamental property of images. Consequently, any comprehensive model of biological vision must incorporate this attribute and provide a veritable description of its impact on visual perception. Current theoretical and computational models predict that vision should modify its characteristics at low contrast: for example, it should become broader (more lowpass) to protect from noise, as often demonstrated by individual neurons. We find that the opposite is true for human discrimination of elementary image elements: vision becomes sharper, not broader, as contrast approaches threshold levels. Furthermore, it suffers from increased internal variability at low contrast and it transitions from a surprisingly linear regime at high contrast to a markedly nonlinear processing mode in the low-contrast range. These characteristics are hard-wired in that they happen on a single trial without memory or expectation. Overall, the empirical results urge caution when attempting to interpret human vision from the standpoint of optimality and related theoretical constructs. Direct measurements of this phenomenon indicate that the actual constraints derive from intrinsic architectural features, such as the co-existence of complex-cell-like and simple-cell-like components. Small circuits built around these elements can indeed account for the empirical results, but do not appear to operate in a manner that conforms to optimality even approximately. More generally, our results provide a compelling demonstration of how far we still are from securing an adequate computational account of the most basic operations carried out by human vision.

Optimized stereo matching algorithm for integral imaging microscopy and its potential use in precise 3-D optical manipulation

Abstract In this paper, we present an integral imaging microscopy, which can achieve continuous depth extraction. The multiple stereo matching algorithm is optimized with data point interpolation and calculation range correction, which can improve the computing efficiency and accuracy. A water suspension of silica-embedded magnetic particles in light-trap environment is used as a specimen. The elementary image array is obtained through sequentially imaging the specimen by the objective lens and microlens array, which contains continuous depth information that can give the optical tweezers a feedback for precise 3-D manipulation. Experimental and calculated results show the feasibility of our method.

Proof-of-concept robot platform for exploring automated harvesting of sugar snap peas

Currently, sugar snap peas are harvested manually. In high-cost countries like Norway, such a labour-intensive practise implies particularly large costs for the farmer. Hence, automated alternatives are highly sought after. This project explored a concept for robotic autonomous identification and tracking of sugar snap pea pods. The approach was based on a combination of visible–near infrared reflection measurements and image analysis, along with visual servoing. A proof-of-concept harvesting platform was implemented by mounting a robotic arm with hand-mounted sensors on a mobile unit. The platform was tested under plastic greenhouse conditions on potted plants of the sugar snap pea variety Cascadia using LED-lights and a partial shade. The results showed that it was feasible to differentiate the pods from the surrounding foliage using the light reflection at the spectral range around 970 nm combined with elementary image segmentation and shape modelling methods. The proof-of-concept harvesting platform was tested on 48 representative agricultural environments comprising dense canopy, varying pod sizes, partial occlusions and different working distances. A set of 104 images were analysed during the teleoperation experiment. The true positive detection rate was 93 and 87% for images acquired at long distances and at close distances, respectively. The robot arm achieved a success rate of 54% for autonomous visual servoing to a pre-grasp pose around targeted pods on 22 untouched scenarios. This study shows the potential of developing a prototype robot for semi-automated sugar snap pea harvesting.

Applying Machine Learning in Optical Music Recognition of Numbered Music Notation

Although research of optical music recognition (OMR) has existed for few decades, most of efforts were put in step of image processing to approach upmost accuracy and evaluations were not in common ground. And major music notations explored were the conventional western music notations with staff. On contrary, the authors explore the challenges of numbered music notation, which is popular in Asia and used in daily life for sight reading. The authors use different way to improve recognition accuracy by applying elementary image processing with rough tuning and supplementing with methods of machine learning. The major contributions of this work are the architecture of machine learning specified for this task, the dataset, and the evaluation metrics, which indicate the performance of OMR system, provide objective function for machine learning and highlight the challenges of the scores of music with the specified notation.

Specific computed virtual chromoendoscopy for detection of peritoneal carcinomatosis: an animal study.

Detection of an incipient Peritoneal Carcinomatosis (PC) is still challenging, and there is a crucial need for technological improvements in order to diagnose and to treat early this condition. The aim of this study was to create a murine model of incipient PC and to explore the PC with Fujinon Intelligent Chromo Endoscopy (FICE) in order to determine the wavelengths of the white light (WL) spectre that offer the highest contrast between PC nodules and surrounding peritoneum. Eighteen BALB/c mice had intraperitoneal injection of murine colonic cancer CT26 cells. Peritoneal exploration with FICE was performed at different times. For each PC nodule, 1 WL and 10 FICE images were recorded. Each image was then divided into its elementary red, green and blue band images. Depending on the FICE channel, each elementary image corresponds to a specific wavelength of the WL spectre. Through numerical analysis of these images, the value of the nodule and the background peritoneum were obtained, and the contrast value was calculated. Contrast values obtained with the different wavelengths were then compared. PC grew in all the mice. The number as well as the size of PC nodules was increasingly high depending on the day of exploration. Mean PCI was 1.6 ± 1.2 at day 5, 7.7 ± 2.6 at day 8 and 15.0 ± 7.3 at day 10. A total number of 1805 elementary images of PC nodules were analysed. The wavelength that offered the best contrast between PC nodules and background peritoneum was 460nm with a mean contrast value of 0.240 ± 0.151 (p < 0.0001). This murine model of incipient PC is effective, reliable and reproducible. A monochromatic light with a wavelength at 460nm offers the highest contrast between PC nodules and background peritoneum, allowing a better detection of PC.

Interpretation value of diffractions and sub-specular reflections – applications on the Zhao Dong field

We provide an overview of integrated pre-stack depth migration and diffraction imaging for the Zhao Dong field, Bohai Bay, China. This field is highly compartmentalized by complex faulting and further characterized by channel systems, fractures and volcanic features. The objective of the diffraction imaging is to better define these small-scale features. Tools to facilitate interpretation include displays with pre-stack depth migration and diffraction images overlain in different colour scales, as well as a weighted blending of them into a single volume. An important concept is that of the sub-specular reflection, which is obtained alongside the pure diffraction image by applying ultra-weak specularity tapers. Tuning properties of elementary diffractor images together with sub-specular reflectors provide a decisive uplift of diffraction imaging for the interpreter.

A proto-object based saliency model in three-dimensional space

Most models of visual saliency operate on two-dimensional images, using elementary image features such as intensity, color, or orientation. The human visual system, however, needs to function in complex three-dimensional environments, where depth information is often available and may be used to guide the bottom-up attentional selection process. In this report we extend a model of proto-object based saliency to include depth information and evaluate its performance on three separate three-dimensional eye tracking datasets. Our results show that the additional depth information provides a small, but statistically significant, improvement in the model’s ability to predict perceptual saliency (eye fixations) in natural scenes. The computational mechanisms of our model have direct neural correlates, and our results provide further evidence that proto-objects help to establish perceptual organization of the scene.

See-through integral imaging display with background occlusion capability.

Background occlusion capability is provided to a flat-panel-type integral imaging display that has a transparent screen and can superimpose three-dimensional (3D) images on real scenes. A symmetric integral imaging system that comprises two integral imaging systems connected by an additional lens array, is proposed. Elementary images are displayed on a flat-panel display on one integral imaging system to generate 3D images, and the occlusion mask patterns are displayed on a flat-panel display on the other integral imaging system to selectively block rays from background scenes. The proposed system was constructed and experimentally verified.