Color Information Of Image Research Articles

Study on estimation of light-receiving amount of plant leaves using color information of images

Study on estimation of light-receiving amount of plant leaves using color information of images

TABIMAE Mind Search System: Exploring Best Tourist Sightseeing Spots Match Mental Values before Travel

The availability of a variety of Internet-based services for tourists has drastically changed the way travel plans are made. This has indicated that instead of travel planning from traditional travel information (articles and guidebooks), it is starting from emotional information linked to photographs and other imagery information. In this study, in order to provide an "EmoTABI experience," a travel to feel EMOI, we have developed to analyze image of interest, extract a color emotion derived from the color information of image and object emotion related to objects in the image, respectively, and search for optimal sightseeing spots match the mental value of before travel, TABIMAE. Using this developed the TABIMAE mind search system, recommendations for tourist destinations were provided for Tokyo, Osaka, and Nasu Town, Tochigi Prefecture. Furthermore, we carried out a questionnaire survey to confirm the validity of the system. As a result, we confirmed that the results obtained from this system were valid. In this paper, the TABIMAE mind search system is overall and reported to discuss the obtained results through the prototype system.

Unsupervised Deep Exemplar Colorization via Pyramid Dual Non-local Attention.

Exemplar-based colorization is a challenging task, which attempts to add colors to the target grayscale image with the aid of a reference color image, so as to keep the target semantic content while with the reference color style. In order to achieve visually plausible chromatic results, it is important to sufficiently exploit the global color style and the semantic color information of the reference color image. However, existing methods are either clumsy in exploiting the semantic color information, or lack of the dedicated fusion mechanism to decorate the target grayscale image with the reference semantic color information. Besides, these methods usually use a single-stage encoder-decoder architecture, which results in the loss of spatial details. To remedy these problems, we propose an effective exemplar colorization strategy based on pyramid dual non-local attention network to exploit the long-range dependency as well as multi-scale correlation. Specifically, two symmetrical branches of pyramid non-local attention block are tailored to achieve alignments from the target feature to the reference feature and from the reference feature to the target feature respectively. The bidirectional non-local fusion strategy is further applied to get a sufficient fusion feature that achieves full semantic consistency between multi-modal information. To train the network, we propose an unsupervised learning manner, which employs the hybrid supervision including the pseudo paired supervision from the reference color images and unpaired supervision from both the target grayscale and reference color images. Extensive experimental results are provided to demonstrate that our method achieves better photo-realistic colorization performance than the state-of-the-art methods. The source code is available at https://github.com/wd1511/PDNLA-Net.

Guided Colorization Using Mono-Color Image Pairs.

Compared to color images captured by conventional RGB cameras, monochrome (mono) images usually have higher signal-to-noise ratios (SNR) and richer textures due to the lack of color filter arrays in mono cameras. Therefore, using a mono-color stereo dual-camera system, we can integrate the lightness information of target monochrome images with the color information of guidance RGB images to accomplish image enhancement in a colorization manner. In this work, based on two assumptions, we introduce a novel probabilistic-concept guided colorization framework. First, adjacent contents with similar luminance are likely to have similar colors. By lightness matching, we can utilize colors of the matched pixels to estimate the target color value. Second, by matching multiple pixels from the guidance image, if more of these matched pixels have similar luminance values to the target one, we can estimate colors with more confidence. Based on the statistical distribution of multiple matching results, we retain the reliable color estimates as initial dense scribbles and then propagate them to the rest of the mono image. However, for a target pixel, the color information provided by its matching results is quite redundant. Hence, we introduce a patch sampling strategy to accelerate the colorization process. Based on the analysis of the posteriori probability distribution of the sampling results, we can use much fewer matches for color estimation and reliability assessment. To alleviate incorrect color propagation in the sparsely scribbled regions, we generate extra color seeds according to the existed scribbles to guide the propagation process. Experimental results show that, our algorithm can efficiently and effectively restore color images with higher SNR and richer details from the mono-color image pairs, and achieves good performance in solving the color bleeding problem.

Development of 3D Scanner Application with Stereo Camera for 3D Object Reconstruction

Visual perception in RGBD (Red-Green-Blue and Depth) camera technology allows the camera to provide color image and depth structure information simultaneously from the surface of the scanned object. Based on this technology, data from an RGBD camera can be processed to reconstruct the scanned object in a three-dimensional (3D) model. However, devices that are prebuilt to perform this task with good accuracy are expensive and are only intended for 3D scanning tasks. On the other hand, some multi-purpose depth cameras can be used for 3D scanning but require some tweaks to ensure a good and accurate reconstructed model. A method has been proposed to optimize the camera and reconstruction system to compile RGBD information from recorded close-up objects and process them into a full-scale 3D digital model. This process involves developing the appropriate recording method along with settings and adjustments for scanning and model reconstruction of close-range objects. As a result, information from multi-purpose RGBD camera can be reconstructed in three-dimensional form that accurately represents real-life objects. The reconstructed model can be utilized for applications that rely on real-life objects as their main basis. The proposed method enables the use of inexpensive multi-purpose RGBD cameras for 3D scanning.

Key frame extraction based on quaternion Fourier transform with multiple features fusion

Key frame extraction, an important technique to improve video viewing efficiency and reduce video redundancy, has attracted considerable attention. Notwithstanding, existing key frame extraction for surveillance video not only fails to extract the local detail information of the target accurately but also preprocess the image with grayscale, so the integrity of the image information is also damaged. In order to solve the above-described problems, the paper presents a key frame extraction method based on the quaternion Fourier transform with the multiple features fusion. Initially, the method extracts dynamic and static features that can characterize the color image information separately. Then, the quaternion matrix is used to perform the quaternion Fourier transform on the different features to obtain the fused phase spectrum of the Fourier transform. Immediately after, the information containing the overall structure of the image is filtered by Gaussian filtering, and the quaternion Fourier inverse transform is performed to obtain the fused feature map. Finally, an adaptive key frame filtering criterion, which uses the fused feature map to extract the key frames is constructed in this paper for accurately extracting key frames. Notably, experimental results indicate that the proposed method ensures the integrity of the image information while accurately capturing the global and local motion state of the target.

VANet: a medical image fusion model based on attention mechanism to assist disease diagnosis

BackgroundToday’s biomedical imaging technology has been able to present the morphological structure or functional metabolic information of organisms at different scale levels, such as organ, tissue, cell, molecule and gene. However, different imaging modes have different application scope, advantages and disadvantages. In order to improve the role of medical image in disease diagnosis, the fusion of biomedical image information at different imaging modes and scales has become an important research direction in medical image. Traditional medical image fusion methods are all designed to measure the activity level and fusion rules. They are lack of mining the context features of different modes of image, which leads to the obstruction of improving the quality of fused images.MethodIn this paper, an attention-multiscale network medical image fusion model based on contextual features is proposed. The model selects five backbone modules in the VGG-16 network to build encoders to obtain the contextual features of medical images. It builds the attention mechanism branch to complete the fusion of global contextual features and designs the residual multiscale detail processing branch to complete the fusion of local contextual features. Finally, it completes the cascade reconstruction of features by the decoder to obtain the fused image.ResultsTen sets of images related to five diseases are selected from the AANLIB database to validate the VANet model. Structural images are derived from MR images with high resolution and functional images are derived from SPECT and PET images that are good at describing organ blood flow levels and tissue metabolism. Fusion experiments are performed on twelve fusion algorithms including the VANet model. The model selects eight metrics from different aspects to build a fusion quality evaluation system to complete the performance evaluation of the fused images. Friedman’s test and the post-hoc Nemenyi test are introduced to conduct professional statistical tests to demonstrate the superiority of VANet model.ConclusionsThe VANet model completely captures and fuses the texture details and color information of the source images. From the fusion results, the metabolism and structural information of the model are well expressed and there is no interference of color information on the structure and texture; in terms of the objective evaluation system, the metric value of the VANet model is generally higher than that of other methods.; in terms of efficiency, the time consumption of the model is acceptable; in terms of scalability, the model is not affected by the input order of source images and can be extended to tri-modal fusion.

Joint depth map super-resolution method via deep hybrid-cross guidance filter

Nowadays color-guided Depth map Super-Resolution (DSR) methods mainly have three thorny problems: (1) joint DSR methods have serious detail and structure loss at very high sampling rate; (2) existing DSR networks have high computational complexity; (3) color-depth inconsistency makes it hard to fuse dual-modality features. To resolve these problems, we propose a joint hybrid-cross guidance filter method to progressively recover the quality of degraded Low-Resolution (LR) depth maps by exploiting color-depth consistency from multiple perspectives. Specifically, the proposed method leverages pyramid structure to extract multi-scale features from High-Resolution (HR) color image. At each scale, hybrid side window filter block is proposed to achieve high-efficiency color feature extraction after each down-sampling for HR color image. This block is also used to extract depth features from the LR depth map. Meanwhile, we propose a multi-perspective cross-guided fusion filter block to progressively fuse high-quality multi-scale structure information of color image with corresponding enhanced depth features. In this filter block, two kinds of space-aware group-compensation modules are introduced to capture various spatial features from different perspectives. Meanwhile, color-depth cross-attention module is proposed to extract color-depth consistency features for impactful boundary preservation. Comprehensively qualitative and quantitative experimental results have demonstrated that our method can achieve superior performances against a lot of state-of-the-art depth SR approaches in terms of mean absolute deviation and root mean square error on Middlebury, NYU-v2 and RGB-D-D datasets.

MobileSal: Extremely Efficient RGB-D Salient Object Detection.

The high computational cost of neural networks has prevented recent successes in RGB-D salient object detection (SOD) from benefiting real-world applications. Hence, this article introduces a novel network, MobileSal, which focuses on efficient RGB-D SOD using mobile networks for deep feature extraction. However, mobile networks are less powerful in feature representation than cumbersome networks. To this end, we observe that the depth information of color images can strengthen the feature representation related to SOD if leveraged properly. Therefore, we propose an implicit depth restoration (IDR) technique to strengthen the mobile networks' feature representation capability for RGB-D SOD. IDR is only adopted in the training phase and is omitted during testing, so it is computationally free. Besides, we propose compact pyramid refinement (CPR) for efficient multi-level feature aggregation to derive salient objects with clear boundaries. With IDR and CPR incorporated, MobileSal performs favorably against state-of-the-art methods on six challenging RGB-D SOD datasets with much faster speed (450fps for the input size of 320×320) and fewer parameters (6.5M). The code is released at https://mmcheng.net/mobilesal.

Research on Retinex Algorithm Combining with Attention Mechanism for Image Enhancement

Considering the high noise and chromatic aberration in the Retinex-Net image enhancement results, this paper put forward a modified Retinex-Net algorithm for weak illumination image enhancement based on the Decom-Net and Enhance-Net structures of Retinex-Net. The improved structure proposed in this paper adds the attention mechanism ECA-Net into the Decom-Net and Enhance-Net convolution layer of the original Retinex-Net structure, which can effectively reduce the problem of irrelevant background and local brightness imbalance, activate sensitive features, and improve the image’s details and brightness processing ability. Additionally, deep connected attention networks are embedded between the introduced attention modules, so that all of the attention modules can be trained jointly to improve the learning ability. Furthermore, the improved method also introduces a noise reduction loss function and a color loss function to suppress noise and to reduce image color distortion. The test results of the proposed method indicate that the image’s overall brightness can be balanced, the local areas cannot be overexposed, and more image details and color information can be retained than with other enhancement algorithms.

Colorimetric analysis based on solid-phase extraction with sedimentable dispersed particulates: demonstration of concept and application for on-site environmental water analysis.

A novel simple and functional colorimetric methodology for on-site environmental water analysis was proposed. This method combines coloration of the analyte and extraction of the colored species on dispersed particulates during their sedimentation in the same container. The whole analysis can be performed within 15 min by comprising the addition of 1 mL of sample solution into a 1.5-mL microtube containing the powders of coloring reagents and the sedimentable fine particulates as an adsorbent. The analyte is determined by comparing the sediment color with the standard color by visual inspection or the color information of the photo image. The potential of this methodology was demonstrated through developing colorimetry for Fe2+ with o-phenanthroline, NO2- by azo-dye formation, HCHO by the MBTH method, and PO43- by the 4-aminoantipyrine method based on the enzyme reactions. The material, size, amount of the adsorbent particles, and other conditions were optimized for each analytes. The advantages of the methodology were as follows: high sensitivity, easy controllability of the sensitivity over the wide range by the amount, size, and material of the particulates, lower interference from the colored matrix components due to obtaining the color data from not the aqueous phase but the sedimented particulates, and acceleration of the color development rate by the particulates as seen in NO2- determination as consequence shorten the operation time. A simple device equipped with twin cells was proposed for on-site analysis which contains two successive different coloring operations. The developed methods were successfully applied to the environmental water samples with the good agreement of the results with those by the usual instrumental methods.

New Aggregation Approaches with HSV to Color Edge Detection

The majority of edge detection algorithms only deal with grayscale images, while their use with color images remains an open problem. This paper explores different approaches to aggregate color information of RGB and HSV images for edge extraction purposes through the usage of the Sobel operator and Canny algorithm. This paper makes use of Berkeley’s image data set, and to evaluate the performance of the different aggregations, the F-measure is computed. Higher potential of aggregations with HSV channels than with RGB channels is found. This article also shows that depending on the type of image used, RGB or HSV, some methods are more appropriate than others.

Evaluation of grinding surface roughness based on gradient similarity and color similarity

Existing machine vision detection methods for surface roughness based on design indices are mainly divided into two categories: grayscale information-based and color information-based. The former loses a large amount of image information when converting the image’s three-dimensional color space to one-dimensional grayscale space. The latter does not fully consider the color information and structural detail changes in images of different rough surfaces. To address the above problems, a visual measurement method of grinding surface roughness based on gradient similarity and color similarity is proposed in this study. This method purposefully uses gradient similarity and color similarity to evaluate the structural differences and color differences between different roughness images, respectively. The comparison experiments with CD, F2, and F5 indices show that the GC index we proposed has a strong correlation with the grinding surface roughness, and its regression fitting prediction model has a high prediction accuracy. In addition, we have discussed the effect of light intensity on the GC index. The experimental result indicates that the correlation between the GC index and roughness is relatively stable under different light intensities.

Fusion of Lidar and Aerial Imagery to Map Wetlands and Channels via Deep Convolutional Neural Network

Wetlands and channels provide significant ecological and societal services. Mapping their locations and types at high resolution remains a challenge for scientific communities and regulatory agencies. In the U.S.A., the National Wetland Inventory (NWI) provides the best nationwide wetland maps, but the NWI for some locations has not been updated for up to four decades and includes omission errors. To address these problems, we developed a deep learning framework for identifying wetlands and channels from lidar point clouds and 1 m aerial images. Both deep learning and terrain analysis were applied to classify wetlands and identify channels. The deep learning classifier was a convolutional neural network that utilized both image color information and lidar-derived canopy height. When tested on a 94 km2 Ohio watershed, the method achieved a classification accuracy of 85.6%. Compared with the NWI maps, the results included 76% more forested wetlands by area and 168% more channels by length. It was also found that only leaf-off images were useful for detecting forested wetlands and typical commission errors (i.e., false positives) were attributed to tree shadows. The study demonstrates the advantages of combining lidar structural and aerial spectral information over the use of each alone and exemplifies the utility of deep learning as an effective means to leverage open-source data for high resolution mapping of wetlands.

Field estimation of maize plant height at jointing stage using an RGB-D camera

Plant height can be used for assessing plant vigor and predicting biomass and yield. Manual measurement of plant height is time-consuming and labor-intensive. We describe a method for measuring maize plant height using an RGB-D camera that captures a color image and depth information of plants under field conditions. The color image was first processed to locate its central area using the S component in HSV color space and the Density-Based Spatial Clustering of Applications with Noise algorithm. Testing showed that the central areas of plants could be accurately located. The point cloud data were then clustered and the plant was extracted based on the located central area. The point cloud data were further processed to generate skeletons, whose end points were detected and used to extract the highest points of the central leaves. Finally, the height differences between the ground and the highest points of the central leaves were calculated to determine plant heights. The coefficients of determination for plant heights manually measured and estimated by the proposed approach were all greater than 0.95. The method can effectively extract the plant from overlapping leaves and estimate its plant height. The proposed method may facilitate maize height measurement and monitoring under field conditions.

Quaternion Capsule Neural Network With Region Attention for Facial Expression Recognition in Color Images

Facial expression recognition (FER) is an essential subject of computer vision and human-computer interaction. It has been reported that many factors are closely related to the FER performance such as the pose, facial muscle variations, and the ignored color information in facial images. In this study, we propose a quaternion capsule neural network (Q-CapsNet) with region attention mechanism for FER in color images. The proposed Q-CapsNet is an end-to-end deep learning framework, which adopts the concept of quaternion theory to the Capsule Neural Network and further uses an attention mechanism to focus on the facial region of interests (ROIs). The Q-CapsNet addresses the FER problems in the following aspects. First, the internal dependencies between color channels are captured by the quaternion technique, which is of great importance in color image processing. Second, the Q-CapsNet could disentangle latent geometry information from facial images by quaternion capsules and quaternion routing algorithm. Third, inspired by the fact that facial expressions are mainly determined by several vital facial regions, the region attention mechanism is introduced to extract emotional features from the ROIs of the face. The proposed Q-CapsNet is evaluated on four public color FER datasets including MMI, Oulu-CASIA, RAF-DB, and SFEW. The comprehensive experiments and visualization results demonstrate that the proposed network outperforms comparative models and the state-of-the-art FER methods.

A decision fusion method based on hyperspectral imaging and electronic nose techniques for moisture content prediction in frozen-thawed pork

This paper studies the feasibility of combining hyperspectral imaging (HSI) and electronic nose (E-nose) techniques to improve the prediction performance of moisture content (MC) in frozen-thawed pork. In this experiment, the data of 240 pork samples under different freeze-thaw cycles (0–5 times) were collected using HSI and E-nose technology. Afterward, the spectra and image (color and texture) information were extracted from the HSI sensor, while the odor information was extracted from the E-nose sensor. In this setting, an improved decision fusion method was proposed to fuse this information, thus detecting the MC in pork. Based on single information and its fusion at different fusion levels (pixel, feature, traditional decision fusion, and improved decision fusion), partial least squares regression (PLSR) prediction models were established and compared. The results show that the model of improved decision fusion based on HSI and E-nose fusion technology exhibits the best prediction capability with the determination coefficient of prediction set (R2p) of 0.9533 and root mean square error of prediction set (RMSEP) of 0.3869. This study demonstrates that the proposed method is an effective data fusion method, and the combination of HSI and E-nose technology improves the prediction performance of MC in frozen-thawed pork.

A Marker-Controlled Watershed Algorithm for the Intelligent Picking of Long Jujubes in Trees

Vision is the most important way for an unmanned picking or plant protection robot to navigate an external environment. To achieve intelligent picking or plant protection, it is essential to obtain target location information. A new marker-controlled watershed (MCW-D) algorithm is proposed for object segmentation. By analyzing the shortcomings of the watershed algorithm and the characteristics of objects, the proposed MCW-D method mainly solves three problems. First, it reduces the influence of shadow and other factors on image color information. Based on histogram specification, secondary mapping is used to reduce the effects of lighting. Second, marker images are selected. All points with markers need to be located in the target object. The hue feature of long jujubes and trees is used as the marker image. Third, a mask image is acquired, which requires a clear boundary between the target and the background. An adaptive angle rotation based on an energy-driven approach is designed to find large differences between the target and the background. In a natural environment, the proposed MCW-D method respectively achieves segmentation accuracies of 94.7% and 93.2% on a jujube dataset and a tree dataset, which exceed the accuracies of widely used machine learning methods. These results promote the development of the forest and fruit economies.

Surface topography data fusion of additive manufacturing based on confocal and focus variation microscopy.

In this paper, two innovative data fusion methods are proposed for reconstructing the surfaces produced by directed energy deposition (DED) additive manufacturing. The surface topographic data were obtained from confocal laser scanning microscopy (CLSM) and focus variation microscopy (FV). The first method (competitive data fusion) aims to improve the data quality by combining both the advantages of the CLSM and FV techniques, while the second method (cooperative data integration) is designed for generating a single representation that contains not only global information but also local details. The results show that both fusion methods achieved satisfactory results: in the competitive fusion, the fused data preserved the characteristics of FV data while its vertical resolution is also improved by integrating the short waves from the CLSM data; the cooperative data fusion achieved one pixel precision of the surface registration which adopted the feature-based registration method with the help of color image information. The computational complexity is reduced from O((m×n)2) to O(m×n + k). Both proposed data fusion methods provided innovative solutions for the microscopic surface reconstruction and surface representation in multiscales in the field of additive manufacturing.

Robust descriptor for key-point detection and matching in color images with radial distortion

Constructing descriptors for key points in image matching is a crucial task in computer vision, visual tracking, and pattern recognition. Highly rated descriptors such as scale-invariant feature transform (SIFT) and speeded up robust features (SURF) utilize the grayscale gradient information of the square regions around the key points. Nevertheless, these famous descriptors fail to take advantage of the image color information. Furthermore, their performances will be severely degraded for distorted images and the error due to rotation of their square windows. To address the problems, we proposed Circular Coordinate Combining Shape–Color Descriptor Under Distortion Based SURF (CSCD-SURF) that achieves competitive performance for colorful images with distortion contamination and numerical similar structures. The proposed approach can generate scale-space by adaptive filter to rectify distortion and integrate the SURF descriptor and shape–color information into our own custom concentric circular coordinate with flying colors. Experiments with Institut National de Recherche en Infomatique et Automatique dataset and spherical radial distortion-SIFT distorted image datasets prove that the ability of distortion rectification as well as matching performance against rotation, scaling, viewpoints, and blurring is more competitive than the state-of-the-art methods and its computation cost is acceptable.