Compact Image Research Articles

Acceptability of the Improvised Compact Image Projector in Teaching Optics

Acceptability of the Improvised Compact Image Projector in Teaching Optics

S$^{2}$ Loop: A Lightweight Spectral-Spatio Loop Closure Detector for Resource-Constrained Platforms

Visual loop closure detection is an essential backend task for long-term vSLAM applications. However, prior works cannot simultaneously meet the requirements of high recall and low computing and memory overhead, which prohibits their applicability to resource-constrained platforms. In this work we propose S <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> Loop, a novel spectral-spatio loop detector that explores an efficient loop searching method based on spectral image analysis and direct image alignment. S <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> Loop digests an image by its Fourier spectrum which provides a compact image hash value for fast candidate selection. The spectra also enable a convenient method of image alignment based on Fourier phase correction. The aligned images are further matched by an optical-flow process at the pixel granularity, which precisely estimates the frame dissimilarity against the turbulence caused by the perspective warping of random 3D scenes. According to our benchmark experiments on various datasets, S <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> Loop shows two orders of magnitude of less computing and memory overhead with comparable precision and recall over the prior DBoW method on a desktop PC. It is also much faster than the prior Fern method that has low computing complexity, and exhibits a much higher recall. While querying loops in 1000 runtime keyframes, S <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> Loop deployed onto a low-end STM32F4 MCU achieves 64fps.

Improvement of measurement method for compact range image sensor using disparity and blur of multi-slit laser

Improvement of measurement method for compact range image sensor using disparity and blur of multi-slit laser

Fingerprints of the seismogram’s wavelet analysis results as a tool for creating a compact signal image for the purposes of neural network recognition

One of the modern directions in solving the problem of recognizing the type of a seismic event from its seismogram recorded by a single seismic receiver is the method of obtaining and comparing signal fingerprints. This paper provides a historical overview of the experience of using this technique initially for the analysis of audio recordings, and then for seismic ones. The existing method for earthquake fingerprinting, which includes the use of a two-dimensional discrete decomposition of the signal spectrogram using the Haar wavelet, is proposed to be improved based on a more accurate creation of the initial time-frequency image of the signal using a continuous wavelet transform. On the example of seismograms of known earthquakes and explosions recorded by the stations of the Kola branch of the GS RAS, the effectiveness of this approach in relation to fixing the key frequency and time features of the signal is shown. The fingerprints properties are indicated, which make them convenient for recognizing the type of seismic event using artificial neural networks.

Evaluation, Comparison and Monitoring of Multiparameter Systems by Unified Graphic Visualization of Activity Method on the Example of Learning Process

The article discusses the problem of visualization of complex multiparameter systems, defined by datasets on their structure, functional structure, and activity in the form of complex graphs and transition of traditional representation of the data acquired by graph mining to a compact image built by pictographic methods. In these situations, we propose using the Unified Graphic Visualization of Activity (UGVA) method for data concentration and structuring. The UGVA method allows coding in an anthropomorphic image of elements of graphs with data on structural and functional features of systems and overlaying these images with the data on the system’s activity using coloring and artifacts. The image can be composed in different ways: it can include the zone of integral evaluation parameters, segmented data axes of five types, and four types of symmetry. We describe the method of creating UGVA images, which consists of 13 stages: the parametric model is represented as a structural image that is converted to a basic image that is then detailed into the particular image by defining geometric parameters of the primitives and to the individualized image with the data about a particular object. We show how the individualized image can be overlaid with the operative data as color coding and artifacts and describe the principles of interpreting UGVA images. This allows solving tasks of evaluation, comparison, and monitoring of complex multiparameter systems by showing the decision-maker an anthropomorphic image instead of the graph. We describe a case study of using the UGVA method for visualization of data about an educational process: curricula and graduate students, including the data mined from the university’s learning management system at the Siberian Federal University for students majoring in “informatics and computing”. The case study demonstrates all stages of image synthesis and examples of their interpretation for situation assessment, monitoring, and comparison of students and curricula. It allowed for finding problematic moments in learning for individual students and their entire group by analyzing the development of their competence profiles and formulating recommendations for further learning. The effectiveness of the resulting images is compared to the other approaches: elastic maps and Chernoff faces. We discuss using graph mining to generate learning problems in order to lessen the workload of gathering raw data for the UGVA method and provide general recommendations for using the UGVA method based on our experience of supporting decision making.

Hetero‐Integrated InGaAs Photodiode and Oxide Memristor‐Based Artificial Optical Nerve for In‐Sensor NIR Image Processing

AbstractIn‐sensor computing is an emerging architectural paradigm that fuses data acquisition and processing within a sensory domain. The integration of multiple functions into a single domain reduces the system footprint while it minimizes the energy and time for data transfer between sensory and computing units. However, it is challenging for a simple and compact image sensor array to achieve both sensing and computing in each pixel. Here, this work demonstrates a focal plane array with a heterogeneously integrated one‐photodiode one‐resistor (1P‐1R)‐based artificial optical neuron that emulates the sensing, computing, and memorization of a biological retina system. This work employs an InGaAs photodiode featuring a high responsivity and a broad spectrum that covers near‐infrared (NIR) signals and employs an HfO2 memristor as the artificial synapse to achieve the computing/memorization in an analog domain. Using the fabricated focal plane array integrated with an artificial neural network, this work performs in‐sensor image identification of finger veins driven by NIR light illumination (≈84 % accuracy). The proposed in‐sensor image computing architecture that broadly covers the NIR spectrum offers widespread application of focal plane array for computer vision, neuromorphic computing, biomedical engineering, etc.

Compact mappings and s-mappings at subsets

Abstract Almost s s -mappings and almost compact mappings have been introduced and studied. In this article, we continue to research some questions related to the almost s-images (resp., almost compact images) of metric spaces. The following results are obtained. (1) A space X X is a quotient and almost compact image of a metric space if and only if X X is a sequential space having a c s ∗ c{s}^{\ast } -network which is point-regular at nonisolated points, which gives an affirmative answer to Question 4.9 in the article “S. Lin, X. W. Ling, and Y. Ge, Point-regular covers and sequence-covering compact mappings, Topology Appl. 271 (2020), 106987.” (2) There exists a bi-quotient and almost compact image of a metric space satisfying no base, which is point-countable at nonisolated points, which gives negative answers to Question 3.1 in the article “X. W. Ling and S. Lin, On open almost s-images of metric spaces, Adv. Math. (China) 48 (2019), no. 4, 489–496” and Question 3.7 in the article “X. W. Ling, S. Lin, and W. He, Point-countable covers and sequence-covering s-mappings at subsets, Topology Appl. 290 (2021), 107572.” (3) Some characterizations of countably bi-quotient and almost s s -images (resp., pseudo-open and almost compact images) of metric spaces.

Segment as Points for Efficient and Effective Online Multi-Object Tracking and Segmentation.

Current multi-object tracking and segmentation (MOTS) methods follow the tracking-by-detection paradigm and adopt 2D or 3D convolutions to extract instance embeddings for instance association. However, due to the large receptive field of deep convolutional neural networks, the foreground areas of the current instance and the surrounding areas containing the nearby instances or environments are usually mixed up in the learned instance embeddings, resulting in ambiguities in tracking. In this paper, we propose a highly effective method for learning instance embeddings based on segments by converting the compact image representation to un-ordered 2D point cloud representation. In this way, the non-overlapping nature of instance segments can be fully exploited by strictly separating the foreground point cloud and the background point cloud. Moreover, multiple informative data modalities are formulated as point-wise representations to enrich point-wise features. For each instance, the embedding is learned on the foreground 2D point cloud, the environment 2D point cloud, and the smallest circumscribed bounding box. Then, similarities between instance embeddings are measured for the inter-frame association. In addition, to enable the practical utility of MOTS, we modify the one-stage instance segmentation method SpatialEmbedding for instance segmentation. The resulting efficient and effective framework, named PointTrackV2, outperforms all the state-of-the-art methods including 3D tracking methods by large margins (4.8 percent higher sMOTSA for pedestrians over MOTSFusion) with the near real-time speed (20 FPS evaluated on a single 2080Ti). Extensive evaluations on three datasets demonstrate both the effectiveness and efficiency of our method. Furthermore, as crowded scenes for cars are insufficient in current MOTS datasets, we provide a more challenging dataset named APOLLO MOTS with a much higher instance density.

Image recommender system based on compact convolutional transformer image style recognition

Social network users face a large number of images; therefore, it is very important that social media finds and recommends images that satisfy the user’s preferences properly. This article investigates the effect of using image style on the performance of a social image recommender system. The effect of distinctive visual styles is evident in art, photography, film, and advertising; however, to the best of our knowledge, it has not been used in social image recommendation systems. We propose image style recognition as a component of user preferences. To do this, we first propose a method based on deep features and compact convolutional transformer to recognize the style of images; then, an image recommender system based on an image style recognition is presented. We consider 15 distinct styles from the Flickr Style dataset, consisting of five categories: atmosphere, mood, genre, composition styles, and color. The experimental results show that style has a positive effect and can significantly improve Recall@k and Precision@k by ∼5 % − 10 % (for the 100 top images recommended) for personalized image recommendation compared with not using the.

Meta-optic accelerators for object classifiers.

Rapid advances in deep learning have led to paradigm shifts in a number of fields, from medical image analysis to autonomous systems. These advances, however, have resulted in digital neural networks with large computational requirements, resulting in high energy consumption and limitations in real-time decision-making when computation resources are limited. Here, we demonstrate a meta-optic–based neural network accelerator that can off-load computationally expensive convolution operations into high-speed and low-power optics. In this architecture, metasurfaces enable both spatial multiplexing and additional information channels, such as polarization, in object classification. End-to-end design is used to co-optimize the optical and digital systems, resulting in a robust classifier that achieves 93.1% accurate classification of handwriting digits and 93.8% accuracy in classifying both the digit and its polarization state. This approach could enable compact, high-speed, and low-power image and information processing systems for a wide range of applications in machine vision and artificial intelligence.

Location recognition in laparoscopic surgery

Abstract Navigation systems play an increasingly important role in minimally invasive surgery (MIS) by mitigating the problems rising from the decoupling of hand-eye movement of the surgeon. Many of these systems suffer from a high dependency on external optical tracking systems that require a constant line-of-sight to the optical markers being tracked. Simultaneous localization and mapping (SLAM) algorithms allow tracking the endoscope in cases where optical tracking fails due to the cluttered environment in the operating room. To ensure a correct camera pose estimate and to correct for drift, a recognition of previously visited locations (loop closures) is essential. We propose a method for location recognition in a minimally invasive scenario that only requires a stereo endoscope and an inertial measurement unit (IMU).We use a hierarchical bag-of-visual-words (BoW) algorithm that saves compact image representations and enables querying for matching images. A two-staged consistency check using a random sample consensus (RANSAC) and the data measured by the IMU ensure a high matching precision.

Efficient waveguide-type four-color (red–green–blue–infrared) laser beam combiner for compact laser beam scanning image projectors

Efficient waveguide-type four-color (red–green–blue–infrared) laser beam combiner for compact laser beam scanning image projectors

Convolutional Neural Network Using for Multi-Sensor 3D Object Detection

The purpose of this article is to detect 3D objects inside the independent vehicle with great accuracy. The method proposed a Multi-View 3D System (MV3D) framework which encodes the sparse 3d-point cloud with a compact multi-view image, using LIDAR satellite image and RGB pictures as inputs, and predicts 3D boundary boxes. The network comprises two sub-networks: one for creating 3D artifacts and one for multi-visual fusion functionality. Propose an autonomous 3D object tracking approach to manipulate sparse and dense knowledge about romanticizing and geometry in stereo images. The Stereo R-CNN strategy applies Faster R-CNNs to stereo inputs such that objects are simultaneously identified and linked in conservative and liberal images. Such charts were then combined and fed into a 3D proposal generator to generate accurate 3D proposals for vehicles. In the second step, the refining network extended the features of the proposal regions further and carried through the classification, regression of a 3D package box, and guidance estimates, to predict vehicle location and heading in 3D area and add additional branches after the stereo region Proposal Network (RPN).

Highly-efficient waveguide-type red–green–blue laser beam combiners for compact projection-type displays

A waveguide-type RGB laser beam combiner was fabricated and its optical properties were measured and analyzed through comparison with a simulation. The combiner has three different waveguides for RGB light inputs, three directional couplers, and a single output waveguide. The fabricated combiner shows an excellent multiplexing efficiency of as high as 96.4%. The size of the multiplexing region of the fabricated combiner is as small as 2.3 mm. Furthermore, by optimizing the combiner structure, it was found that the multiplexing efficiency hardly depends on the gap width of the directional coupler in the combiner. The polarization dependence of the multiplexing efficiency was also measured. The difference in multiplexing efficiency between the TE and TM modes is within 4% when using the optimized combiner structure, demonstrating the small polarization dependence of the multiplexing efficiency. Thus, the fabricated combiner exhibit’s high performance, as well as excellent fabrication reproducibility, leading to the realization of compact laser scanning image projectors.

Investigation of REFINED CNN ensemble learning for anti-cancer drug sensitivity prediction.

MotivationAnti-cancer drug sensitivity prediction using deep learning models for individual cell line is a significant challenge in personalized medicine. Recently developed REFINED (REpresentation of Features as Images with NEighborhood Dependencies) CNN (Convolutional Neural Network)-based models have shown promising results in improving drug sensitivity prediction. The primary idea behind REFINED-CNN is representing high dimensional vectors as compact images with spatial correlations that can benefit from CNN architectures. However, the mapping from a high dimensional vector to a compact 2D image depends on the a priori choice of the distance metric and projection scheme with limited empirical procedures guiding these choices.ResultsIn this article, we consider an ensemble of REFINED-CNN built under different choices of distance metrics and/or projection schemes that can improve upon a single projection based REFINED-CNN model. Results, illustrated using NCI60 and NCI-ALMANAC databases, demonstrate that the ensemble approaches can provide significant improvement in prediction performance as compared to individual models. We also develop the theoretical framework for combining different distance metrics to arrive at a single 2D mapping. Results demonstrated that distance-averaged REFINED-CNN produced comparable performance as obtained from stacking REFINED-CNN ensemble but with significantly lower computational cost.Availability and implementationThe source code, scripts, and data used in the paper have been deposited in GitHub (https://github.com/omidbazgirTTU/IntegratedREFINED).Supplementary information Supplementary data are available at Bioinformatics online.

Image retrieval based on aggregated deep features weighted by regional significance and channel sensitivity

Deep convolutional neural networks (CNN) have demonstrated a very powerful approach for extracting discriminative local descriptors for image description. Many related works suggest that an effective aggregation representation for deep convolutional features is particularly important in forming robust and compact image representations. In this paper, a new robust global descriptor for image retrieval is proposed by creating an effective method for aggregating local deep convolutional features weighted by regional significance and channel sensitivity through sum-pooling on multiple regions. The proposed aggregation method effectively takes advantage of multiple scales, and considers both the varied significance of regional visual content and the sparsity and intensity of response values in the channel. This can improve the ability of deep features to be described and discerned. The experimental results on six benchmark datasets demonstrate that our method can achieve retrieval results comparable to those of some popular approaches for deep feature aggregation but without fine-tuning strategies and multiple image inputs.

Converting tabular data into images for deep learning with convolutional neural networks

Convolutional neural networks (CNNs) have been successfully used in many applications where important information about data is embedded in the order of features, such as speech and imaging. However, most tabular data do not assume a spatial relationship between features, and thus are unsuitable for modeling using CNNs. To meet this challenge, we develop a novel algorithm, image generator for tabular data (IGTD), to transform tabular data into images by assigning features to pixel positions so that similar features are close to each other in the image. The algorithm searches for an optimized assignment by minimizing the difference between the ranking of distances between features and the ranking of distances between their assigned pixels in the image. We apply IGTD to transform gene expression profiles of cancer cell lines (CCLs) and molecular descriptors of drugs into their respective image representations. Compared with existing transformation methods, IGTD generates compact image representations with better preservation of feature neighborhood structure. Evaluated on benchmark drug screening datasets, CNNs trained on IGTD image representations of CCLs and drugs exhibit a better performance of predicting anti-cancer drug response than both CNNs trained on alternative image representations and prediction models trained on the original tabular data.

Compound eye image scanner optimally arranged for image combining and suppressing axial chromatic aberration by wavefront coding

A compound eye optical system design allows us to construct a compact image scanner with a large field of view and a large depth of field, both of which are beneficial for use in a copier’s flatbed scanner. Our prototypes are constructed of 32 or 28 imaging units, which are ommatidia, and the representations from the imaging units are electrically combined. Since the arrangement of the imaging units is a key factor in combining the images correctly without any loss at the boundaries, we discuss two types of arrangements along with results of their prototypes: a one-line arrangement of varying magnification imaging units and a two-line staggered arrangement of telecentric imaging units. We also discuss the effectiveness of wavefront coding (WFC) in suppressing chromatic aberration, which is caused using cheap refractive imaging units, along with experimental results of applying both cubic- and quartic-phase-modulated WFC.

A scalable approach for the efficient segmentation of hyperspectral images

The number of applications of hyperspectral imaging (HSI) is steadily increasing, as technology evolves and cameras become more affordable. However, the volume of data in a hyperspectral image is large (order of Gigabytes) and standard off-the-shelf algorithms for multi-channel image analysis cannot be readily applied, due to the prohibitive computational time and large memory requirements. Therefore, new scalable approaches are required to perform hyperspectral image analysis. In this article we address an efficient methodology for conducting Unsupervised Image Segmentation – one of the basic and most fundamental image analysis operations. In the methodology proposed, unsupervised segmentation is conducted after transforming the spectral and spatial dimensions of the raw hyperspectral image into a more compact representation using multivariate and multiresolution techniques. The clusters identified in the compact image representation are then used to train a discriminative classifier. The classifier is then adapted and transferred for application to the raw image, where it will efficiently label all the original pixels. With the proposed methodology, the computational expensive operations (unsupervised clustering and classifier learning) are minimized, whereas the efficient implementation of the classifier guarantees the analysis at the native resolution. The effectiveness of the proposed methodology was tested on a real case study considering an industrial hyperspectral image capturing the reflectance spectrum for several objects made of different unknown materials. A significant reduction in the computational cost was achieved without compromising the quality of the unsupervised segmentation, demonstrating the potential of the proposed approach.

Deep multiscale divergence hashing for image retrieval

Image retrieval based on deep learning of hash has made great progress. The hash method increases retrieval speed greatly while saving storage space. However, some problems exist, such as the distinctiveness of image feature still needs to be improved and some image features are still redundant. We propose a new deep learning to hash method, namely, deep multiscale divergence hashing, which provides a high diversity and compact image feature for image retrieval. The discriminative features from deep neural networks are identified by the importance criterion in network pruning techniques and the feature redundancy is reduced. Then, the selected features across different layers are fused in a certain proportion to increase the diversity of features for image retrieval. We also present a hybrid loss function in hash space, which consists of the weighted pairwise cross-entropy loss function and the KL-divergence. It tends to minimize the hamming distance between similar images and maximize the hamming distance between different images, which helps improve the accuracy. Massive experimental results show that our method achieves better feature distinguishability and more advanced image retrieval accuracy, and surpasses HashNet by 11.46%, 7.58%, and 13.86% on MS COCO, NUS-WIDE, and CIFAR-10 datasets, respectively.