Compact Image Research Articles

О ДИФФЕРЕНЦИАЛЬНО-НЕАВТОНОМНОМ ПРЕДСТАВЛЕНИИ ИНТЕГРАТИВНОЙ АКТИВНОСТИ НЕЙРОПОПУЛЯЦИИ БИЛИНЕЙНОЙ МОДЕЛЬЮ ВТОРОГО ПОРЯДКА С ЗАПАЗДЫВАНИЕМ

For neuromorphic processes specified by the behavior of a local neuropopulation (for example, processes induced by a brain-machine interface platform of the Neuralink type), we study the solvability of the problem of the existence of a differential realization of these processes in the class of bilinear nonstationary ordinary differential equations of the second order (with delay) in separable Hilbert space. This formulation belongs to the type of inverse problems for an additive combination of nonstationary linear and bilinear operators of evolution equations in an infinite-dimensional Hilbert space. The metalanguage of the theory being developed is the constructions of tensor products of Hilbert spaces, lattice structures with orthocompletion, the functional apparatus of the nonlinear Rayleigh-Ritz operator, and the principle of maximum entropy. It is shown that the property of sublinearity of this operator, allows you to obtain conditions for the existence of such differential realizations; along the way, metric conditions for the continuity of the projectivization of this operator are substantiated with the calculation of the fundamental group of its compact image. This work was financially supported by the Russian Foundation for Basic Research (project no. 19-01-00301).

On spaces with point-star networks consisting of cs-finite cs⁎-coverings

In this paper, by a finite-type property P, some characterizations of point-star networks consisting of P and cs⁎-coverings are discussed in Cauchy sn-symmetric spaces. Then it is established that the relations between spaces with point-star networks consisting of certain cs⁎-coverings and the images of metric spaces under some mappings, which extends and unifies some results of An and Tuyen in 2018, and gives a partial answer to the following question posed by Tanaka and Ge in 2006: Is a symmetric space having a σ-point-finite cs⁎-network a quotient compact image of a metric space?

A Compact Full-color Laser Beam Scanning Image Projector Based on a Waveguide-type RGB Combiner

A Compact Full-color Laser Beam Scanning Image Projector Based on a Waveguide-type RGB Combiner

Automated multi-parameter high-dose-rate brachytherapy quality assurance via radioluminescence imaging

The purpose of this study is to leverage radioluminescence imaging for the development of an automated high-dose-rate (HDR) brachytherapy quality assurance (QA) system that enables simultaneous measurements of dwell position, dwell time, wire velocity, and relative source strength in a single test. The system consists of a radioluminescence phosphor sheet (a mixture of Gd2O2S:Tb and PDMS) positioned atop a HDR needle applicator, a complementary metal-oxide-semiconductor digital camera used to capture the emitted radioluminescence signals from the scintillator sheet, and an in-house graphical user interface for signal processing. The signal processing was used to extract source intensity, location, and elapsed time, yielding the final measurements on dwell position, dwell time, and wire velocity. The source strength relative to the well chamber calibration (in unit of Air-Kerma strength, Sk ) is measured by establishing a calibration curve that correlates Sk with the detector response. Validation experiments are performed using three customized treatment plans. With these plans, the dwell position and dwell time are verified for a range of 110.0 cm–117.5 cm and 2 s–16 s, respectively, and the linear correlation with Sk is demonstrated for the source strength varying between 28 348 U (cGy cm2 h−1) and 41 906 U. The wire velocity, i.e. the speed of the radioactive source averaged over the distance in between dwell positions, is calculated for various distances ranging from 5 mm to 50 mm. Results show that the mean deviations of the measured dwell position and dwell time are 0.1 mm (range from 0 to 0.2 mm) and 32.5 ms (range from 0 to 60.0 ms) with respect to the planned values, respectively, and the system response is highly linear with Sk ( R2 = 0.998). Moreover, the measured wire velocities are comparable to previously reported values. Benefitting from the compact hardware design and image processing algorithms, the system provides a practical, reliable, and comprehensive solution for HDR QA.

Season-Invariant and Viewpoint-Tolerant LiDAR Place Recognition in GPS-Denied Environments

Place recognition remains a challenging problem under various perceptual conditions, e.g., all weather, times of day, seasons, and viewpoint shifts. Different from most of the existing place recognition methods using pure vision, this article studies light detection and ranging (LiDAR) based approaches. Point clouds have some benefits for place recognition since they do not suffer from illumination changes. On the other hand, they are dramatically affected by structural changes from different viewpoints or across seasons. In this article, a novel LiDAR-based place recognition system is proposed to achieve long-term robust localization, even under severe seasonal changes and viewpoint shifts. To improve the efficiency, a compact cylindrical image model is designed to convert three-dimensional point clouds to two-dimensional images representing the prominent geometric relationships of scenes. The contexts (buildings, trees, road structures, etc.) of scenes are utilized for efficient place recognition. A sequence-based temporal consistency check is also introduced for postverification. Extensive real experiments on three datasets (Oxford RobotCar [1] , NCLT [2] , and DUT-AS) show that the proposed system outperforms both state-of-the-art visual and LiDAR-based methods, verifying its robust performance in challenging scenarios.

HM$^4$: Hidden Markov Model With Memory Management for Visual Place Recognition

Visual placerecognition needs to be robust against appearance variability due to natural and man-made causes. Training data collection should thus be an ongoing process to allow continuous appearance changes to be recorded. However, this creates an unboundedly-growing database that poses time and memory scalability challenges for place recognition methods. To tackle the scalability issue for visual place recognition in autonomous driving, we develop a Hidden Markov Model approach with a two-tiered memory management. Our algorithm, dubbed HM <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> , exploits temporal look-ahead to transfer promising candidate images between passive storage and active memory when needed. The inference process takes into account both promising images and a coarse representations of the full database. We show that this allows constant time and space inference for a fixed coverage area. The coarse representations can also be updated incrementally to absorb new data. To further reduce the memory requirements, we derive a compact image representation inspired by Locality Sensitive Hashing (LSH). Through experiments on real world data, we demonstrate the excellent scalability and accuracy of the approach under appearance changes and provide comparisons against state-of-the-art techniques.

A compact image encryption system based on Arnold transformation

A novel compact image encryption system is proposed based on Arnold transformation (AT). In general, AT is a pixel-scramble tool in image encryption, but its security is not high enough as an independent cryptosystem because of its periodicity. Hence, extra diffusion operations are needed to diffuse the pixel values to acquire the higher security. Here an encryption algorithm is proposed by AT. Firstly, the corresponding coordinates of each pixel in original image are calculated by AT. Secondly, the pixel values of the two corresponding coordinates are disturbed by another AT. These two processes can diffuse the pixel values of the original image. At last, the diffused image is pixel scrambled by AT with different parameters. This is the confusion process. In proposed scheme, the diffusing and confusing operations are both realized by AT. At the same time, the image is encrypted as a pixel distribution instead of a data stream. The key space, key sensitivity, the correlation, the cost time and the security of proposed cryptosystem are analyzed and the results confirm that the proposed image encryption algorithm demonstrates extraordinary performance.

Representation of features as images with neighborhood dependencies for compatibility with convolutional neural networks

Deep learning with Convolutional Neural Networks has shown great promise in image-based classification and enhancement but is often unsuitable for predictive modeling using features without spatial correlations. We present a feature representation approach termed REFINED (REpresentation of Features as Images with NEighborhood Dependencies) to arrange high-dimensional vectors in a compact image form conducible for CNN-based deep learning. We consider the similarities between features to generate a concise feature map in the form of a two-dimensional image by minimizing the pairwise distance values following a Bayesian Metric Multidimensional Scaling Approach. We hypothesize that this approach enables embedded feature extraction and, integrated with CNN-based deep learning, can boost the predictive accuracy. We illustrate the superior predictive capabilities of the proposed framework as compared to state-of-the-art methodologies in drug sensitivity prediction scenarios using synthetic datasets, drug chemical descriptors as predictors from NCI60, and both transcriptomic information and drug descriptors as predictors from GDSC.

Bounded Composition Operators on~Weighted Function Spaces in the Unit Disk

We introduce a general class of weighted spaces $\calH(\beta)$ of holomorphic functions in the unit disk $\mathbb{D}$, which contains several classical spaces, such as Hardy space, Bergman space, Dirichlet space. We~characterize boundedness of composition operators $C_{\varphi}$ induced by affine and monomial symbols $\varphi$ on these spaces $\calH(\beta)$. We also establish a sufficient condition under which the operator $C_{\varphi}$ induced by the symbol $\varphi$ with relatively compact image $\varphi(\mathbb{D})$ in $\mathbb{D}$ is bounded on $\calH(\beta)$. Note that in the setting of $\calH(\beta)$, the characterizations of boundedness of composition operators $C_{\varphi}$ depend closely not only on functional properties of the symbols $\varphi$ but also on the behavior of the weight sequence $\beta$.

Yield prediction and validation of onion (Allium cepa L.) using key variables in narrowband hyperspectral imagery and effective accumulated temperature

In this study, hyperspectral imagery was used to develop models for predicting onion yields in two years (2017 and 2018). First, the bands for the full width half maximum (FWHM) measurement of 5 nm in the canopied areas were merged as FWHM 10, 25, and 50 nm. This was based on a commercialized band-pass filter that considered the development of the compact multispectral image sensors. Then, band rationing was performed to correct the unstable reflectance through incomplete radiating normalization. Stepwise and variable importance in projection in partial least squares (PLS_VIP) approaches were applied to select the optimal FWHM by evaluating both models’ performance and the number of overlapped band ratios. The optimal FWHM measurement was 10 nm, with both high model performances and the highest number of overlapped band ratios. The overlapped ratios of 440/450, and 730/760 in variable importance in projection (VIP) 1 were fixed in the onion-yield prediction model. Conversely, the non-fixed, non-overlapped ratios of 420/430, 490/500, 500/510, 590/600, 620/630, 660/670, 670/680, 710/720, 810/820, and 870/880 were reduced one by one; this was dependent on their removal ranking in descending order of the mean ratio of reduction (MROR) based on the RMSE value, using the leave-one-out method. These combinations in both the fixed and non-fixed band ratios were used to develop prediction models with and without effective accumulated temperature (EAT) values. In all combinations, the models’ performance developed with EAT were increased by preventing slight or sharp decrease in performance, compared to those without EAT. The models, in each year, developed by seven band ratios (420/430, 440/450, 500/510, 590/600, 620/630, 670/680, and 730/760) among the combinations were maintained. The prediction models with EAT were cross-validated (by predicting the 2017 yields using the 2018 model and the 2018 yields using the 2017 model) to evaluate the reproducibility in other years. The reproducibility of the model developed by the seven band ratios was optimal with errors of RMSE = 172 g/m2, RE = 30.3% in 2017 using the 2018 model and RMSE = 215 g/m2 and RE = 33.2% in 2018 using the 2017 model. Consequently, the key band ratios for predicting the onion yields were identified as the seven band ratios with EAT at 420/430, 440/450, 500/510, 590/600, 620/630, 670/680, and 730/760. Ultimately, these findings will provide compact multispectral image sensors specialized in onion-yield prediction that can monitor wide agriculture fields mounted on various platforms.

CSIDNet: Compact single image dehazing network for outdoor scene enhancement

This paper proposes a novel deep learning-based single image dehazing network named as Compact Single Image Dehazing Network (CSIDNet) for outdoor scene enhancement. CSIDNet directly outputs a haze-free image from the given hazy input. The remarkable features of CSIDNet are that it has been designed only with three convolutional layers and it requires lesser number of images for training without diminishing the performance in comparison to the other commonly observed deep learning-based dehazing models. The performance of CSIDNet has been analyzed on natural hazy scene images and REalistic Single Image DEhazing (RESIDE) dataset. RESIDE dataset consists of Outdoor Training Set (OTS), Synthetic Objective Testing Set (SOTS), and real-world synthetic hazy images from Hybrid Subjective Testing Set (HSTS). The performance metrics used for comparison are Peak Signal to Noise Ratio (PSNR) and Structural SIMilarity (SSIM) index. The experimental results obtained using CSIDNet outperform several well known state-of-the-art dehazing methods in terms of PSNR and SSIM on images of SOTS and HSTS from RESIDE dataset. Additionally, the visual comparison shows that the dehazed images obtained using CSIDNet are more appealing with better edge preservation. Since the proposed network requires minimal resources and is faster to train along with lesser run-time, it is more practical and feasible for real-time applications.

Malware classification using compact image features and multiclass support vector machines

Malware and malicious code do not only incur considerable costs and losses but impact negatively the reputation of the targeted organisations. Malware developers, hackers, and information security specialists are continuously improving their strategies to defeat each other. Unfortunately, there is no one-size-fits-all solution to detect and eradicate any malware. This situation is aggravated more by the undetected vulnerabilities that usually impair computer software and internet tools. Such vulnerabilities will remain undetected until fully exploited by malware developers, which will eventually cause considerable financial and reputation losses. In this paper, we propose a novel scheme to detect and classify malware using only image representations of the malware binaries. Highly discriminative features of the malware category and structure are extracted in a compact subspace using principal component analysis. Then, an optimised support vector machine model classifies the extracted features into malware categories. Unlike existing classification models, our solution requires simple algebraic dot products to classify malware based on representative digital images. To assess its performance, publicly-available image datasets, Malimg, Ember and BIG 2015, are considered. Our performance analysis indicates that their classifier outperforms state-of-the-art models and attains classification accuracies of 0.998, 0.911, and 0.997 using Malimg, Ember and BIG 2015 malware datasets, respectively.

An embedded novel compact feature profile image in speech signal for teledermoscopy system.

Teledermoscopy is a promising telemedicine service for remote diagnosis and treatment of skin diseases using dermoscopy images. It requires high quality transmission services, efficient utilization of channel bandwidth, effective storage, and security. Thus, this work develops an improved teledermoscopy system that guarantees the efficient and secure transmission of the dermoscopy images. It proposed a novel feature-based secure diagnostic system that supports the automated classification of malignant melanoma and benign nevus at the receiver side (i.e. medical facility). To overcome the transmission of the original dermoscopy images having large size, a novel representation of the dermoscopy images is proposed, namely the compact feature profile (CFP). The proposed CFP represents the dermoscopy image only using its significant features. For security purpose, the CFP is embedded as a watermark in a speech signal using singular value decomposition (SVD) watermarking at the transmitter. Then, the de-embedding/reconstruction process is performed at the receiver end using a proposed modified SVD technique. Finally, the extracted CFP is fed into a classifier for diagnosis at the receiver. To evaluate the robustness of the proposed system, an additive white Gaussian noise (AWGN) attack was employed during the transmission process. To improve the immunity against the AWGN attack, a novel speech signal weight factor is proposed at the watermarking process. Moreover, a compensation factor is calculated at the training phase to compensate the effect of the channel AWGN attack at the receiver. In addition, the superior transform domain and embedding positions of the CFP in the speech signal were studied. The experimental results established that the proposed CFP diagnostic system achieved high classification accuracy, sensitivity, specificity, and F-measure for classifying the two skin cancer classes with the presence of signal-to-noise ratio (SNR) ranging from 10 to 25dB. This work established that the newly proposed CFP watermarked in speech signal using the DWT-based modified SVD followed by single-level decomposition Db1 with hard thresholding wavelet denoising achieved efficient diagnostic teledermoscopy system.

Fusing Transformed Deep and Shallow features (FTDS) for image-based facial expression recognition

In this paper, we propose combining between the transformed hand-crafted and deep features using PCA to recognize the six-basic facial expressions from static images. To evaluate our approach, we use three popular databases (CK+, CASIA and MMI). We introduce the use of the Pyramid Multi Level (PML) face representation for facial expression recognition. The hand-crafted features are obtained with such representations. Initially, we determine the optimal level of the PML features of three hand-crafted descriptors (HOG, LPQ and BSIF) using CK+, CASIA and MMI databases.After the optimal level of the PML is found for each descriptor, we combine them together with the transformed final VGG-face layers (FC6 and FC7) in order to get a compact image descriptor. In within-database experiments, our approach achieved higher accuracy than the state-of-art methods on both the CK+ and CASIA databases, and competitive result on the MMI database. Likewise, our approach outperformed the static methods in all six experiments of cross-databases.

Product Quantization Network for Fast Visual Search

Product quantization has been widely used in fast image retrieval due to its effectiveness of coding high-dimensional visual features. By constructing the approximation function, we extend the hard-assignment quantization to soft-assignment quantization. Thanks to the differentiable property of the soft-assignment quantization, the product quantization operation can be integrated as a layer in a convolutional neural network, constructing the proposed product quantization network (PQN). Meanwhile, by extending the triplet loss to the asymmetric triplet loss, we directly optimize the retrieval accuracy of the learned representation based on asymmetric similarity measurement. Utilizing PQN, we can learn a discriminative and compact image representation in an end-to-end manner, which further enables a fast and accurate image retrieval. By revisiting residual quantization, we further extend the proposed PQN to residual product quantization network (RPQN). Benefited from the residual learning triggered by residual quantization, RPQN achieves a higher accuracy than PQN using the same computation cost. Moreover, we extend PQN to temporal product quantization network (TPQN) by exploiting temporal consistency in videos to speed up the video retrieval. It integrates frame-wise feature learning, frame-wise features aggregation and video-level feature quantization in a single neural network. Comprehensive experiments conducted on multiple public benchmark datasets demonstrate the state-of-the-art performance of the proposed PQN, RPQN and TPQN in fast image and video retrieval.

Hardware-Compliant Compressive Image Sensor Architecture Based on Random Modulations and Permutations for Embedded Inference

This work presents a compact CMOS Image Sensor (CIS) architecture enabling embedded object recognition facilitated by a dedicated end-of-column Compressive Sensing (CS), reducing on-chip memory needs. Our sensing scheme is based on a combination of random modulations and permutations leading to an implementation with very limited hardware impacts. It is designed to meet both theoretical ( i.e. , stable embedding, measurements incoherence) and practical requirements ( i.e. , silicon footprint, power consumption). The only additional hardware compared to a standard CIS architecture using first order incremental Sigma-Delta ( $\Sigma \Delta $ ) Analog to Digital Converter (ADC) are a pseudo-random data mixing circuit, an in- $\Sigma \Delta ~\pm 1$ modulator and a small Digital Signal Processor (DSP). On the algorithmic side, three variants are presented to perform the inference on compressed measurements with a tunable complexity ( i.e. , one-vs.-all SVM, hierarchical SVM and small ANN with 1-D max-pooling). An object recognition accuracy of $\simeq 98.8$ % is reached on the COIL database (COIL, 100 classes) using our dedicated Neural Network classifier. We stress that the signal-independent dimensionality reduction performed by our dedicated CS scheme (1/480 in $480 \times 640$ VGA resolution case) allows to dramatically reduce memory requirements mainly related to the remotely learned coefficients used for the inference stage.

Predicting Citywide Road Traffic Flow Using Deep Spatiotemporal Neural Networks

Traffic flow forecasting has been a long-standing topic in intelligent transportation systems, and a renewed interest has been seen in recent years due to the development of artificial intelligence techniques. New deep neural networks have been developed to model traffic flow, but it is very challenging to predict citywide traffic flow at the road level in fine temporal scale owing to the influence of spatiotemporal dependencies and spatial sparsity. In this study, based on an in-depth analysis of traffic flow patterns, we propose a deep learning based spatiotemporal neural network model to predict citywide traffic flow for each road segment with high accuracy. Firstly, we examine the transformation of road network into its compact 2D image, where road segments correspond to pixels and their topological relationships are maintained in a large extent. Then, an end-to-end deep learning structure is designed to model traffic flow patterns. Specifically, recurrent convolutional network is employed to learn temporal dependencies and densely connected convolutional network is adopted to learn spatial dependencies and handle spatial sparsity. Our model attempts to aggregate the outputs of those hybrid networks using different weights, which is further enhanced by external information such as day of week. Experiments were conducted in Wuhan, China, where taxicab trajectory data were used to train and validate our model. When compared to current state of the art models, our model achieves higher accuracy in both single and multi-step traffic flow prediction tasks.

Delving Deeper in Drone-Based Person Re-Id by Employing Deep Decision Forest and Attributes Fusion

Deep learning has revolutionized the field of computer vision and image processing. Its ability to extract the compact image representation has taken the person re-identification (re-id) problem to a new level. However, in most cases, researchers are focused on developing new approaches to extract more fruitful image representation and use it in the re-id task. The extra information about images is rarely taken into account because the traditional person re-id datasets usually do not have it. Nevertheless, the research in multimodal machine learning has demonstrated that the utilization of the information from different sources leads to better performance. In this work, we demonstrate how a person re-id problem can benefit from the utilization of multimodal data. We have used the UAV drone to collect and label the new person re-id dataset, which is composed of pedestrian images and its attributes. We have manually annotated this dataset with attributes, and in contrast to the recent research, we do not use the deep network to classify them. Instead, we employ the continuous bag-of-words model to extract the word embeddings from text descriptions and fuse it with features extracted from images. Then the deep neural decision forest is used for pedestrians classification. The extensive experiments on the collected dataset demonstrate the effectiveness of the proposed model.

Compact Incoherent Image Differentiation with Nanophotonic Structures

There are significant recent interests in using nanophotonic structures to perform differentiation operation on images for edge detection purposes. All previous works using nanophotonic structures, however, can only operate with coherent light. Here we introduce a hybrid optoelectronic approach that enables one to use nanophotonic structures to perform differentiation operation with incoherent light. As a demonstration, we consider a photonic crystal slab structure and show that differentiation operation with incoherent light can be achieved by subtracting the optical transfer function of the structure at two different frequencies. Our method is robust to noise and directly integrable into existing imaging systems and, thus, points to a new avenue for improving image sensors using nanophotonic structures.

Focus on photonics, spectrometry, and spectroscopy

Focus on photonics, spectrometry, and spectroscopy