One-shot Image Research Articles

Three-dimensional surface reconstruction from reflectance direction fields with orthogonal multicolor filters

A three-dimensional (3D) freeform surface reconstruction method is proposed, specifically designed to handle complex surfaces, even those with steep inclinations. This method utilizes a one-shot color mapping imaging system to obtain reflectance direction fields using a stripe multicolor filter positioned in two orthogonal orientations. This imaging system can capture reflectance directions that are identifiable by corresponding colors. The multicolor filter consists of multiple stripe regions, each with a different transmission spectrum. Theoretically, an equation that describes the relationship between reflectance direction fields and the reflective 3D surface can be derived based on geometrical optics. By solving this equation using a deep neural network (DNN) as a gradient descent method without any training data, the 3D surface can be accurately reconstructed, even for surfaces with steep inclinations. The effectiveness of this method is validated through numerical demonstrations on several 3D surfaces, including complex ones with concave and convex areas, as well as steep inclinations.

Reinforcing Generated Images via Meta-Learning for One-Shot Fine-Grained Visual Recognition.

One-shot fine-grained visual recognition often suffers from the problem of having few training examples for new fine-grained classes. To alleviate this problem, off-the-shelf image generation techniques based on Generative Adversarial Networks (GANs) can potentially create additional training images. However, these GAN-generated images are often not helpful for actually improving the accuracy of one-shot fine-grained recognition. In this paper, we propose a meta-learning framework to combine generated images with original images, so that the resulting "hybrid" training images improve one-shot learning. Specifically, the generic image generator is updated by a few training instances of novel classes, and a Meta Image Reinforcing Network (MetaIRNet) is proposed to conduct one-shot fine-grained recognition as well as image reinforcement. Our experiments demonstrate consistent improvement over baselines on one-shot fine-grained image classification benchmarks. Furthermore, our analysis shows that the reinforced images have more diversity compared to the original and GAN-generated images.

Discrete aggregation hashing for image set classification

With the development of vision technology, image set classification (ISC) has flourished in the image processing field. Different from the one-shot image classification, ISC focuses on the set rather than a one-shot image. Hence, ISC can synthesize the abundant set information to alleviate various appearance variations. Despite the great success of the existing ISC methods, there are still some problems: (1) They usually face an expensive time complexity, which directly limits the practical application; (2) They largely ignore the intrinsic relationships between different sets. In light of this, we propose a novel Discrete Aggregation Hashing (DAH) for fast ISC. To be specific, to extract more semantic information from each set and each sample, we adopt the same projection standard to embed dual semantic labels (i.e., sample label and set label) into instance and set hash codes. Then we regard set hash codes as set-specific centers. A hashing aggregation strategy is proposed to learn compact discriminative instance hash codes via iteratively aggregating intrinsic neighborhood representations around each central node. Therefore, instance hash codes can obtain greater intra-set compactness and inter-set separability. Extensive experiments demonstrate that our DAH can obtain promising performance and outperform these state-of-the-art ISC methods on four image set datasets.

Imaging immune checkpoint networks in cancer tissues with supermultiplexed SERS nanoprobes

Combined immune checkpoint (ICP) inhibitors maximize immune response rates of patients compared to the single-drug treatment strategy in cancer immunotherapy, and prediction of such optimal combinations requires high-throughput imaging techniques and suitable data analysis. In this work, we report a rational strategy for predicting combined drugs of ICP inhibitors based on supermultiplexed surface-enhanced Raman scattering (SERS) imaging and correlation network analysis. To this end, we first built an ultrasensitive and supermultiplexed volume-active SERS (VASERS) nanoprobe platform, where Raman molecules are randomly arranged in 3D volumetric electromagnetic hotspots. By examining various bio-orthogonal Raman molecules with different electronic properties, we developed frequency modulation guidelines and achieved 32 resolvable colors in the Raman-silent region, the largest number of resolvable SERS colors demonstrated to date. We then demonstrated one-shot ten-color imaging of ICPs with high spectral resolution in clinical biopsies of breast cancer tissues, suggesting highly heterogeneous expression patterns of ICPs across tumor subtypes. Through correlation network analysis of these high-throughput Raman data, we investigated co-expression relationships among these ten-panel ICPs in cancer tissues and finally identified a variety of possible ICP combinations for synergistic immunotherapy of breast cancers, which may lead to novel therapeutical insights.

One-shot Ultra-high-Resolution Generative Adversarial Network That Synthesizes 16K Images On A Single GPU

We propose a one-shot ultra-high-resolution generative adversarial network (OUR-GAN) framework that generates non-repetitive 16K (16,384×8,640) images from a single training image and is trainable on a single consumer GPU. OUR-GAN generates an initial image that is visually plausible and varied in shape at low resolution, and then gradually increases the resolution by adding detail through super-resolution. Since OUR-GAN learns from a real ultra-high-resolution (UHR) image, it can synthesize large shapes with fine details and long-range coherence, which is difficult to achieve with conventional generative models that rely on the patch distribution learned from relatively small images. OUR-GAN can synthesize high-quality 16K images with 12.5 GB of GPU memory and 4K images with only 4.29 GB as it synthesizes a UHR image part by part through seamless subregion-wise super-resolution. Additionally, OUR-GAN improves visual coherence while maintaining diversity by applying vertical positional convolution. In experiments on the ST4K and RAISE datasets, OUR-GAN exhibited improved fidelity, visual coherency, and diversity compared with the baseline one-shot synthesis models. To the best of our knowledge, OUR-GAN is the first one-shot image synthesizer that generates non-repetitive UHR images on a single consumer GPU. The synthesized image samples are presented at https://our-gan.github.io.

Pol2Pol: self-supervised polarimetric image denoising.

In this Letter, we present a self-supervised method, polarization to polarization (Pol2Pol), for polarimetric image denoising with only one-shot noisy images. First, a polarization generator is proposed to generate training image pairs, which are synthesized from one-shot noisy images by exploiting polarization relationships. Second, the Pol2Pol method is extensible and compatible, and any network that performs well in supervised image denoising tasks can be deployed to Pol2Pol after proper modifications. Experimental results show Pol2Pol outperforms other self-supervised methods and achieves comparable performance to supervised methods.

MAGIC: Mask-Guided Image Synthesis by Inverting a Quasi-robust Classifier

We offer a method for one-shot mask-guided image synthesis that allows controlling manipulations of a single image by inverting a quasi-robust classifier equipped with strong regularizers. Our proposed method, entitled MAGIC, leverages structured gradients from a pre-trained quasi-robust classifier to better preserve the input semantics while preserving its classification accuracy, thereby guaranteeing credibility in the synthesis. Unlike current methods that use complex primitives to supervise the process or use attention maps as a weak supervisory signal, MAGIC aggregates gradients over the input, driven by a guide binary mask that enforces a strong, spatial prior. MAGIC implements a series of manipulations with a single framework achieving shape and location control, intense non-rigid shape deformations, and copy/move operations in the presence of repeating objects and gives users firm control over the synthesis by requiring to simply specify binary guide masks. Our study and findings are supported by various qualitative comparisons with the state-of-the-art on the same images sampled from ImageNet and quantitative analysis using machine perception along with a user survey of 100+ participants that endorse our synthesis quality.

One-shot reflectance direction field imaging for measuring the surface slope distribution of a capillary wave.

A method for measuring a surface slope distribution of a capillary wave is proposed. The method uses an optical imaging system that can capture a one-shot image of a light-reflectance direction field in a two-dimensional image plane. A dispersion relation between the wavelength and frequency of the capillary wave is shown to be obtainable by the imaging system, which agrees well with the theoretical prediction.

Vector sensor imaging

Several methods have been proposed for reconstructing the target image using a linear image sensor with detectors arranged in one dimension. However, they have problems with illumination time, reconstruction time, and complex synchronization between pattern illumination and measuring light intensity data. Here, we propose a one-shot imaging and classification method using a linear sensor and diffuser, called vector sensor imaging. We constructed two deep neural networks for this purpose and demonstrated their effectiveness through simulations and experiments.

Surface Normals and Shape From Water.

In this paper, we introduce a novel method for reconstructing surface normals and depth of dynamic objects in water. Past shape recovery methods have leveraged various visual cues for estimating shape (e.g., depth) or surface normals. Methods that estimate both compute one from the other. We show that these two geometric surface properties can be simultaneously recovered for each pixel when the object is observed underwater. Our key idea is to leverage multi-wavelength near-infrared light absorption along different underwater light paths in conjunction with surface shading. Our method can handle both Lambertian and non-Lambertian surfaces. We derive a principled theory for this surface normals and shape from water method and a practical calibration method for determining its imaging parameters values. By construction, the method can be implemented as a one-shot imaging system. We prototype both an off-line and a video-rate imaging system and demonstrate the effectiveness of the method on a number of real-world static and dynamic objects. The results show that the method can recover intricate surface features that are otherwise inaccessible.

A Relational Model for One-Shot Classification of Images and Pen Strokes

We show that a deep learning model with built-in relational inductive bias can bring benefits to sample-efficient learning, without relying on extensive data augmentation. Our study shows that excellent results can be achieved with a model in which the relational inductive bias is applied to images, while building an efficient one-shot classifier on top of raw strokes is more challenging. The proposed one-shot classification model performs relational matching of a pair of inputs in the form of local and pairwise attention. Our approach solves with almost perfect accuracy the one-shot image classification Omniglot challenge when combined with a Hungarian matching algorithm and attains competitive results on the same task on characters represented as rotation-augmented strokes.

Two-Branch Attention Network via Efficient Semantic Coupling for One-Shot Learning.

Over the past few years, Convolutional Neural Networks (CNNs) have achieved remarkable advancement for the tasks of one-shot image classification. However, the lack of effective attention modeling has limited its performance. In this paper, we propose a Two-branch (Content-aware and Position-aware) Attention (CPA) Network via an Efficient Semantic Coupling module for attention modeling. Specifically, we harness content-aware attention to model the characteristic features (e.g., color, shape, texture) as well as position-aware attention to model the spatial position weights. In addition, we exploit support images to improve the learning of attention for the query images. Similarly, we also use query images to enhance the attention model of the support set. Furthermore, we design a local-global optimizing framework that further improves the recognition accuracy. The extensive experiments on four common datasets (miniImageNet, tieredImageNet, CUB-200-2011, CIFAR-FS) with three popular networks (DPGN, RelationNet and IFSL) demonstrate that our devised CPA module equipped with local-global Two-stream framework (CPAT) can achieve state-of-the-art performance, with a significant improvement in accuracy of 3.16% on CUB-200-2011 in particular.

3D retinal imaging and measurement using light field technology.

.Significance: Light-field fundus photography has the potential to be a new milestone in ophthalmology. Up-to-date publications show only unsatisfactory image quality, preventing the use of depth measurements. We show that good image quality and, consequently, reliable depth measurements are possible, and we investigate the current challenges of this novel technology.Aim: We investigated whether light field (LF) imaging of the retina provides depth information, on which structures the depth is estimated, which illumination wavelength should be used, whether deeper layers are measurable, and what kinds of artifacts occur.Approach: The technical setup, a mydriatic fundus camera with an LF imager, and depth estimation were validated by an eye model and in vivo measurements of three healthy subjects and three subjects with suspected glaucoma. Comparisons between subjects and the corresponding optical coherence tomography (OCT) measurements were used for verification of the depth estimation.Results: This LF setup allowed for three-dimensional one-shot imaging and depth estimation of the optic disc with green light. In addition, a linear relationship was found between the depth estimates of the OCT and those of the setup developed here. This result is supported by the eye model study. Deeper layers were not measurable.Conclusions: If image artifacts can be handled, LF technology has the potential to help diagnose and monitor glaucoma risk at an early stage through a rapid, cost-effective one-shot technology.

One-shot three-dimensional imaging using a two-dimensional spectrometer with a fiber bundle

Using spectral interference from a chirped optical frequency comb, we developed a one-shot three-dimensional (3D) imaging method. We formed a two-dimensional spectrometer using fiber bundles made up of single-mode and multi-mode fibers to detect the spectral interference fringe images. The proposed method demonstrated one-shot 3D imaging of reflective targets with sub-µm level depth uncertainty and a wide dynamic range of 6 digits. Furthermore, by developing a method to remove the speckle noise caused by multi-mode fibers, we succeeded in capturing 3D profiles with sub-µm uncertainty in a one-shot with 299 times higher efficiency than single-mode fibers using a large numerical aperture.

Siamese network‐based computer vision approach to detect papaya seed adulteration in black peppercorns

Food adulteration is a growing peril for consumers, traders, and manufacturers worldwide. Food fraud costs the economy a fortune and creates mistrust among consumers and merchants. Black Pepper is a valuable and heavily adulterated spice. This study explores the ability of deep learning coupled with image processing to identify black pepper contaminated with its common adulterant papaya seeds. Prevalent methods work on a relatively small sample, requiring specialization and resources. Our research proposes a system to quickly and economically obtain a method to trace added impurities. It is a problematic venture for one-shot image classification systems to differentiate between identical entities. Therefore, we implement a Siamese network that is proficient in it. A data set of 2,000 images of each class is created. After experimentation, the training and validation accuracy of 0.96 and 0.92 is obtained, while human eye judgment of the same images gave a correctness score of 0.97. Novelty impact statement A novel non-destructive, agile, cost-effective, user-friendly technique to trace papaya seeds' adulteration in black peppercorns using Artificial Intelligence. The proposed system is better in accuracy than the majority of existing approaches.

Technical light-field setup for 3D imaging of the human nerve head validated with an eye model

Abstract With the new technology of 3D light field (LF) imaging, fundus photography can be expanded to provide depth information. This increases the diagnostic possibilities and additionally improves image quality by digitally refocusing. To provide depth information in the human optic nerve head such as in glaucoma diagnostics, a mydriatic fundus camera was upgraded with an LF imager. The aim of the study presented here was the validation of the technical setup and resulting depth estimations with an appropriate eye model. The technical setup consisted of a mydriatic fundus camera (FF450, Carl Zeiss Meditec AG, Jena, Germany) and an LF imager (R12, Raytrix GmbH, Kiel, Germany). The field of view was set to 30°. The eye model (24.65 mm total length) consisted of a two-lens optical system and interchangeable fundus models with papilla excavations from 0.2 to 1 mm in steps of 0.2 mm. They were coated with red acrylic lacquer and vessels were drawn with a thin brush. 15 images were taken for each papilla depth illuminated with green light (wavelength 520 nm ± 20 nm). Papilla depth was measured from the papilla ground to the surrounding flat region. All 15 measurements for each papilla depth were averaged and compared to the printed depth. It was possible to perform 3D fundus imaging in an eye model by means of a novel LF-based optical setup. All LF images could be digitally refocused subsequently. Depth estimation in the eye model was successfully performed over a 30° field of view. The measured virtual depth and the printed model papilla depth is linear correlated. The presented LF setup allowed high-quality 3D one-shot imaging and depth estimation of the optic nerve head in an eye model.

Lead Assays with Smartphone Detection Using a Monolithic Rod with 4-(2-Pyridylazo) Resorcinol.

A monolithic rod of polyurethane foam–[4-(2-pyridylazo) resorcinol] (PUF–PAR) as a simple chemical sensor for lead assays with smartphone detection and image processing was developed. With readily available simple apparatus such as a plastic cup and a stirrer rod, the monolithic PUF rod was synthesized in a glass tube. The monolithic PUF–PAR rod could be directly loaded by standard/sample solution without sample preparation. A one-shot image in G/B value from a profile plot in ImageJ for a sample with triplicate results via a single standard calibration approach was obtained. A linear single standard calibration was: [G/B value] = −0.038[µg Pb2+] + 2.827, R2 = 0.95 for 10–30 µg Pb2+ with a limit of quantitation (LOQ) of 33 µg L−1. The precision was lower than 15% RSD. The proposed method was tested by an assay for Pb2+ contents in drinking water samples from Bangkok. The results obtained by the proposed method agree with those of ICP-OES and with 100–120% recovery, demonstrating that the method is useful for screening on-site water monitoring.

One-shot BRDF imaging system to obtain surface properties

The bidirectional reflectance distribution function (BRDF) that describes an angle-resolved distribution of surface reflectance is available for characterizing surface properties of a material. A one-shot BRDF imaging system can capture an in-plane color mapping of light direction extracted from a surface BRDF distribution. A surface roughness identification method is then proposed here using the imaging system. A difference between surface properties of a matt paper and a glossy paper is experimentally shown to be detected using the method. A surface reconstruction method of an axisymmetric micro-object using the imaging system is also proposed here. The imaging system experimentally shows that it can reconstruct an axisymmetric aluminium cone surface with a height of 37 μm.

Siamese networks for large-scale author identification

Authorship attribution is the process of identifying the author of a text. Approaches to tackling it have been conventionally divided into classification-based ones, which work well for small numbers of candidate authors, and similarity-based methods, which are applicable for larger numbers of authors or for authors beyond the training set; these existing similarity-based methods have only embodied static notions of similarity. Deep learning methods, which blur the boundaries between classification-based and similarity-based approaches, are promising in terms of ability to learn a notion of similarity, but have previously only been used in a conventional small-closed-class classification setup.Siamese networks have been used to develop learned notions of similarity in one-shot image tasks, and also for tasks of mostly semantic relatedness in NLP. We examine their application to the stylistic task of authorship attribution on datasets with large numbers of authors, looking at multiple energy functions and neural network architectures, and show that they can substantially outperform previous approaches.

One-Shot Image Segmentation with U-Net

Image segmentation is an important task in the field of computer vision research. Many applications need accurate and efficient segmentation mechanisms. However, most existing methods need a large support set and have struggled with dealing with new classes in solving computer vision problems. In order to deal with the problem, this paper modified the traditional model to adapt it to the task of few-shot segmentation. Especially, the part of encoders was changed into a Siamese neural network for support branch and query branch. Then, the decoded query image is compared with different levels of features from the encoded support image. Experimental results show that this proposed model achieved the best performance compared with several baseline methods, and the improvement is more than 8%.