Capture Color Images Research Articles

Deep demosaicking convolution neural network and quantum wavelet transform-based image denoising

ABSTRACT Demosaicking is a popular scientific area that is being explored by a vast number of scientists. Current digital imaging technologies capture colour images with a single monochrome sensor. In addition, the colour images were captured using a sensor coupled with a Colour Filter Array (CFA). Furthermore, the demosaicking procedure is required to obtain a full-colour image. Image denoising and image demosaicking are the two important image restoration techniques, which have increased popularity in recent years. Finding a suitable strategy for multiple image restoration is critical for researchers. Hence, a deep learning (DL) based image denoising and image demosaicking is developed in this research. Moreover, the Autoregressive Circle Wave Optimization (ACWO) based Demosaicking Convolutional Neural Network (DMCNN) is designed for image demosaicking. The Quantum Wavelet Transform (QWT) is used in the image denoising process. Similarly, Quantum Wavelet Transform (QWT) is used to analyse the abrupt changes in the input image with noise. The transformed image is then subjected to a thresholding technique, which determines an appropriate threshold range. Once the threshold range has been determined, soft thresholding is applied to the resulting wavelet coefficients. After that, the extraction and reconstruction of the original image is carried out using the Inverse Quantum Wavelet Transform (IQWT). Finally, the fused image is created by combining the results of both processes using a weighted average. The denoised and demosaicked images are combined using the weighted average technique. Furthermore, the proposed QWT+DMCNN-ACWO model provided the ideal values of Peak signal-to-noise ratio (PSNR), Second derivative like measure of enhancement (SDME), Structural Similarity Index (SSIM), Figure of Merit (FOM) of 0.890, and computational time of 49.549 dB, 59.53 dB, 0.963, 0.890, and 0.571, respectively.

Identification of exfoliated monolayer hexagonal boron nitride films with a digital color camera under white light illumination

Optical microscopy with white light illumination has been employed when obtaining exfoliated monolayer hexagonal boron nitride (1L hBN) films from a large number of randomly placed films on a substrate. However, real-time observation of 1L hBN using a color camera under white light illumination remains challenging since hBN is transparent in the visible wavelength range. The poor optical constant of 1L hBN films in microphotographs is significantly improved using a Si substrate coated with a SiN x thin-film (SiN x /Si). When observing hBN thin films on SiN x /Si using a color digital camera in an optical microscope under white light illumination, the clarity of the captured color images depends on the thickness of the SiN x film (d). For real-time direct observation, the d was optimized based on quantitative chromatic studies tailored to Bayer filters of a color image sensor. Through image simulation, it was determined that the color difference between 1L hBN and the bare substrate is maximized at d = 59 or 70 nm, which was experimentally verified. The SiN x /Si with optimized d values visualized 1L hBN films without requiring significant contrast enhancement via image processing under white light illumination in real-time. Furthermore, the captured color photographs facilitate the reliable determination of the number of layers in few-layer hBN films using the contrast of the green channel of the images.

A smartphone-enabled colorimetric sensing platform based on Au-decorated Fe3O4 nanoparticles as nanozyme for dual-mode visual monitoring of dopamine hydrochloride

The potential risks associated with the use of dopamine hydrochloride (DH) on the quality of meat and poultry products, as well as human health, are examined in this study. A novel approach for the synthesis of Au-decorated magnetic spherical nanoparticles (Fe3O4@Au) is proposed, aiming to develop colorimetric analysis sensors for mobile devices. Notably, this newly developed nanocomposite exhibits inherent peroxidase-like activity, enabling efficient catalysis of the oxidation reaction between the substrate 2,2′-azino-bis-(3-ethylbenzthiazoline-6-sulphonate) (ABTS) and H2O2. The results of the study led to the development of a colorimetric smartphone sensor, known as Color Image Capture, which demonstrated high sensitivity and visual capability in detecting DH. The methodology involved the utilization of a smartphone and hydrogen peroxide in conjunction with a Fe3O4@Au nanocomposite for the quantification of DH. The alteration in color exhibited by the platform was captured using a mobile camera and subsequently analyzed using a smartphone application. The smartphone colorimetric sensor exhibited a limit of detection (LOD) of 0.0722 μmol/L, enabling the quantitative detection of DH within the concentration range of 0.005–0.125 mmol/L. The colorimetric measurement technique demonstrated efficacy in quantifying DH levels in pork samples. The development of innovative catalysts and the implementation of novel methodologies have contributed novel insights into the detection of DH in intricate food and biological matrices.

Development of intelligent waste sorting system of low-value recyclable waste in Xiamen

The classification and recycling of waste are significant for environmental protection and resource conservation. In China, the fine sorting of low-value recyclable waste (LVRW) primarily uses manual sorting, which is inefficient and harmful to health. Automated sorting uses sensors to identify the type of waste on a conveyor belt, which is then automatically separated by a sorting organisation to improve the recycling process. A low-cost, high-performance sorting system was built in this paper. The system used a low-resolution industrial camera to capture colour images of waste and a pneumatic separator to sort the waste. The recognition accuracy of waste was improved by optimising the recognition model and using the copy–paste data augmentation method. Mark pneumatic valve control algorithm is proposed that achieves more than 99% waste recovery even under dense working conditions. The sorting system is suitable for sorting plants and has been applied to a case study of LVRW in Xiamen city. In the sorting experiments under complex working conditions, the purity and recycling rate of plastic and Tetra Pak wastes achieved over 95%. The experiment demonstrates the great potential of the system for waste recycling and will help solve the problem of municipal waste in developing countries.

RTDOD: A large-scale RGB-thermal domain-incremental object detection dataset for UAVs

Recently, visual understanding using unmanned aerial vehicles (UAVs) has gained significant attention due to its wide range of applications, including delivery, security investigation and surveillance. However, most existing UAV-based datasets only capture color images under ideal illumination and weather conditions, typically sunny days. This limitation fails to account for the complexity of real-world scenarios, such as cloudy or foggy weather, and nighttime conditions. Deep learning methods trained on color images with good lighting and weather conditions struggle to adapt to the complex visual scenes in these scenarios. Moreover, color images may not provide sufficient visual information under the complex visual scenes. To bridge this gap and meet the demands of real-world applications, we propose a large-scale RGB-Thermal Domain-incremental Object Detection (RTDOD) dataset in this paper. Our dataset includes RGB and thermal videos synchronously captured using calibrated color thermal cameras mounted on UAVs. It covers various weather conditions, from sunny to foggy to rainy, and spans from day to night. We sample and obtain approximately 16,200 pairs of images, and manually label dense annotations, including object bounding boxes and object categories. With the proposed dataset, we introduce a challenging domain-incremental object detection task. We also present a baseline approach that uses task-related gates to filter features for knowledge distillation to reduce forgetting. Experimental results on the RTDOD dataset demonstrate the effectiveness of our proposed method in domain-incremental object detection. To facilitate future research and development in domain-incremental object detection tasks on aerial images, the RTDOD dataset and our baseline model are made available at https://github.com/fenght96/RTDOD.ARTICLE INFO.

Digital Image Correlation with a Prism Camera and Its Application in Complex Deformation Measurement.

Given the low accuracy of the traditional digital image correlation (DIC) method in complex deformation measurement, a color DIC method is proposed using a prism camera. Compared to the Bayer camera, the Prism camera can capture color images with three channels of real information. In this paper, a prism camera is used to collect color images. Relying on the rich information of three channels, the classic gray image matching algorithm is improved based on the color speckle image. Considering the change of light intensity of three channels before and after deformation, the matching algorithm merging subsets on three channels of a color image is deduced, including integer-pixel matching, sub-pixel matching, and initial value estimation of light intensity. The advantage of this method in measuring nonlinear deformation is verified by numerical simulation. Finally, it is applied to the cylinder compression experiment. This method can also be combined with stereo vision to measure complex shapes by projecting color speckle patterns.

Bioinspired color-near infrared endoscopic imaging system for molecular guided cancer surgery.

Fluorescently guided minimally invasive surgery is improving patient outcomes and disease-free survival, but biomarker variability hinders complete tumor resection with single molecular probes. To overcome this, we developed a bioinspired endoscopic system that images multiple tumor-targeted probes, quantifies volumetric ratios in cancer models, and detects tumors in ex vivo samples. We present a new rigid endoscopic imaging system (EIS) that can capture color images while simultaneously resolving two near-infrared (NIR) probes. Our optimized EIS integrates a hexa-chromatic image sensor, a rigid endoscope optimized for NIR-color imaging, and a custom illumination fiber bundle. Our optimized EIS achieves a 60% improvement in NIR spatial resolution when compared to a leading FDA-approved endoscope. Ratio-metric imaging of two tumor-targeted probes is demonstrated in vials and animal models of breast cancer. Clinical data gathered from fluorescently tagged lung cancer samples on the operating room's back table demonstrate a high tumor-to-background ratio and consistency with the vial experiments. We investigate key engineering breakthroughs for the single-chip endoscopic system, which can capture and distinguish numerous tumor-targeting fluorophores. As the molecular imaging field shifts toward a multi-tumor targeted probe methodology, our imaging instrument can aid in assessing these concepts during surgical procedures.

Battery-free wireless imaging of underwater environments

Imaging underwater environments is of great importance to marine sciences, sustainability, climatology, defense, robotics, geology, space exploration, and food security. Despite advances in underwater imaging, most of the ocean and marine organisms remain unobserved and undiscovered. Existing methods for underwater imaging are unsuitable for scalable, long-term, in situ observations because they require tethering for power and communication. Here we describe underwater backscatter imaging, a method for scalable, real-time wireless imaging of underwater environments using fully-submerged battery-free cameras. The cameras power up from harvested acoustic energy, capture color images using ultra-low-power active illumination and a monochrome image sensor, and communicate wirelessly at net-zero-power via acoustic backscatter. We demonstrate wireless battery-free imaging of animals, plants, pollutants, and localization tags in enclosed and open-water environments. The method’s self-sustaining nature makes it desirable for massive, continuous, and long-term ocean deployments with many applications including marine life discovery, submarine surveillance, and underwater climate change monitoring.

Thinking Outside the Color Profiling Capture Box

Color-capture systems use color-correction processing operations to deliver expected results in the saved image files. For cultural heritage imaging projects, establishing and monitoring such operations are important when meeting imaging requirements and guidelines. To reduce unwanted variations, it is common to evaluate imaging performance, and adjust hardware and software settings. In most cases these include the use of ICC Color profiling software and supporting measurements. While advice on the subject by experts can be deftly persuasive, discussions of color goodness for capture are clouded by many imaging variables. This makes claims of a single, color-profiling approach or engine moot in the context of a greater workflow environment. We suggest looking outward and considering alternative profiling practices and evaluation methods that could improve color image capture accuracy and consistency.

Near-infrared fundus camera with a patterned interference filter for the retinal scattering detection

The fundus camera is a system that enables retinal observations for the diagnosis of eye diseases. A near-infrared (NIR) multispectral fundus imaging system can capture color images using NIR light instead of visible light, thus avoiding irritation of the eyes and allowing for easier observations of the fundus. In this study, we developed an NIR fundus camera with a patterned metal mask for retinal patterned illumination. The pattern illuminated fundus images from the experiment prove the feasibility of retinal scattering detection using retinal patterned illumination. The layout of the patterned mask was optimized to prevent the interaction of scattered light and the multi-layer film structures of a patterned interference filter were proposed. The simulated transmission spectra showed that the transmittance of the linear stripe pattern, visual target, and fundus illumination are sufficiently high for the retinal patterned illumination.

Demosaicing by Differentiable Deep Restoration

A mosaic of color filter arrays (CFAs) is commonly used in digital cameras as a spectrally selective filter to capture color images. The captured raw image is then processed by a demosaicing algorithm to recover the full-color image. In this paper, we formulate demosaicing as a restoration problem and solve it by minimizing the difference between the input raw image and the sampled full-color result. This under-constrained minimization is then solved with a novel convolutional neural network that estimates a linear subspace for the result at local image patches. In this way, the result in an image patch is determined by a few combination coefficients of the subspace bases, which makes the minimization problem tractable. This approach further allows joint learning of the CFA and demosaicing network. We demonstrate the superior performance of the proposed method by comparing it with state-of-the-art methods in both settings of noise-free and noisy data.

High dynamic range real-time 3D measurement based on Fourier transform profilometry

The high dynamic range (HDR) fringe projection technique is widely used to measure the three-dimensional (3D) shapes of objects with shiny surfaces; however, it tends to compromise when measuring the dynamic objects. To reduce the number of images and improve the real-time performance, a Fourier transform profilometry (FTP)-based method that can increase the dynamic range for real-time 3D measurement is proposed. Firstly, single-color patterns are projected onto the measured object, and one captured color image can be separated into three monochrome images with different intensities by taking advantage of the different responses of R, G, and B channels. Second, the HDR fringe pattern is generated by compositing three monochrome images, and then processed by the background-normalized algorithm to obtain the final fringe pattern. Subsequently, FTP is employed to retrieve the phase map and obtain the 3D shape. The experimental results demonstrated that the proposed method can measure the dynamic objects with shiny surfaces in real-time 3D measurement system.

Image-Guided Human Reconstruction via Multi-Scale Graph Transformation Networks.

3D human reconstruction from a single image is a challenging problem. Existing methods have difficulties to infer 3D clothed human models with consistent topologies for various poses. In this paper, we propose an efficient and effective method using a hierarchical graph transformation network. To deal with large deformations and avoid distorted geometries, rather than using Euclidean coordinates directly, 3D human shapes are represented by a vertex-based deformation representation that effectively encodes the deformation and copes well with large deformations. To infer a 3D human mesh consistent with the input real image, we also use a perspective projection layer to incorporate perceptual image features into the deformation representation. Our model is easy to train and fast to converge with short test time. Besides, we present the D2Human (Dynamic Detailed Human) dataset, including variously posed 3D human meshes with consistent topologies and rich geometry details, together with the captured color images and SMPL models, which is useful for training and evaluation of deep frameworks, particularly for graph neural networks. Experimental results demonstrate that our method achieves more plausible and complete 3D human reconstruction from a single image, compared with several state-of-the-art methods. The code and dataset are available for research purposes at http://cic.tju.edu.cn/faculty/likun/projects/MGTnet.

Kinect Validation of Ergonomics in Human Pick and Place Activities Through Lateral Automatic Posture Detection

In this paper we evaluate a system based on the Microsoft Kinect™ sensor, aimed at the automatic detection of risk postures during human work activities. We first introduce a pick and place task, where three different lateral standing subjects move light cardboard boxes from the various levels of a bookcase to its top, and then putting them back to their original places. They repeat the task over several work cycles and we capture all their natural movements in a continuous way using Kinect, storing the joint positions and the color images. Secondly, from the joint positions, our system detects specific risk postures following the definitions of the Rapid Upper Limb Assessment (RULA) method. Finally, we compare the posture detections by our system with the baseline detections made by a panel of five experts who used the captured color images. In our study we find that the experts have problems to distinguish among some RULA postures during a work cycle because of the narrow detection margin and the difficulty to perceive if a limb reached a certain position; which is particularly true for the cases of wrist and neck. This leads to a larger false positive rate and to a lower general accuracy, with our system detecting postures that experts do not. After applying a ±1° of relaxation to our system, which in negligible for human perception, we are able to reach an accuracy of 0.93 in the comparison with the baseline. Our results show the suitability of Kinect for lateral risk posture detection in pick and place activities.

A comparison between a white LED confocal imaging system and a conventional flash fundus camera using chromaticity analysis

BackgroundConventional flash fundus cameras capture color images that are oversaturated in the red channel and washed out in the green and blue channels, resulting in a retinal picture that often looks flat and reddish. A white LED confocal device was recently introduced to provide a high-quality retinal image with enhanced color fidelity. In this study, we aimed to evaluate the color rendering properties of the white LED confocal system and compare them to those of a conventional flash fundus camera through chromaticity analysis.MethodsA white LED confocal device (Eidon, Centervue, Padova, Italy) and a traditional flash fundus camera (TRC-NW8, Topcon Corporation, Tokyo, Japan) were used to capture fundus images. Color images were evaluated with respect to chromaticity. Analysis was performed according to the image color signature. The color signature of an image was defined as the distribution of its pixels in the rgb chromaticity space. The descriptors used for the analysis are the average and variability of the barycenter positions, the average of the variability and the number of unique colors (NUC) of all signatures.ResultsTwo hundred thirty-three color photographs were acquired with each retinal camera. The images acquired by the confocal white LED device demonstrated an average barycenter position (rgb = [0.448, 0.328, 0.224]) closer to the center of the chromaticity space, while the conventional fundus camera provides images with a clear shift toward red at the expense of the blue and green channels (rgb = [0.574, 0.278, 0.148] (p < 0.001). The variability of the barycenter positions was higher in the white LED confocal system than in the conventional fundus camera. The average variability of the distributions was higher (0.003 ± 0.007, p < 0.001) in the Eidon images compared to the Topcon camera, indicating a greater richness of color. The NUC percentage was higher for the white LED confocal device than for the conventional flash fundus camera (0.071% versus 0.025%, p < 0.001).ConclusionsEidon provides more-balanced color images, with a wider richness of color content, compared to a conventional flash fundus camera. The overall higher chromaticity of Eidon may provide benefits in terms of discriminative power and diagnostic accuracy.

Research on Robot Control Technology Based on Kinect

With the development of robot control technology, the interaction between humans and robots has become more and more common. Using human movements and gestures to control the robot can replace complex and cumbersome program operations, easily and conveniently manipulate the robot, issue commands to the robot, and interact with the robot. Human vision and gesture recognition based on machine vision is an indispensable key technology for implementing a new generation of human-computer interaction systems. As a revolutionary product, Kinect somatosensory devices capture color images, depth images, and human bone images, and provide a new way of human-computer interaction. It captures and tracks human movements, gestures and sounds. In this paper, the Kinect somatosensory device is used for static gesture recognition and dynamic gesture recognition. Based on this, the robot action is used to realize the contactless interactive interaction of the robot through gesture control. This paper introduces a Kinect device-based robot control method, which is related to the field of robot control, and is applied to the two-wheeled self-balancing robot motion control and human-computer interaction system. This method utilizes Kinect for the motion control system of two-wheeled self-balancing robot. It is controlled with the DTW gesture recognition algorithm.

Colour imaging in the monitoring and documentation of choroidal naevi. Are Optomap colour images adequate for this purpose?

An audit project to evaluate and compare three different imaging systems used to photograph choroidal naevi and to determine whether the Optos Optomap® can be used as the only colour image capture system for monitoring and documenting choroidal naevi. A further aim was to assess whether existing protocols could be improved to accurately document position and appearance of choroidal naevi. Twenty patients with choroidal naevi were photographed on three different colour image capture systems. Colour images were taken on the Optomap® wide field P200MA camera; the Zeiss FF450plus® mydriatic camera and the Topcon TRC-NW6S®. All images were reviewed retrospectively by a medical retina consultant (SD) who completed a questionnaire to determine the most effective photographic system(s) in demonstrating the location of the naevi and the features of the condition. The Optomap® was the most effective in pinpointing the location of the naevus and the Zeiss FF450plus mydriatic camera best captured the features of the naevus. The non-mydriatic camera was rated the least satisfactory for both tasks. The location of the naevus on the retina should determine the choice of modality. If it is possible to photograph the lesion and include the optic disc or central macula, then the mydriatic camera is considered the best modality for recording both the position and features of the pathology. However, if it is not possible, because of the location, to include both the disc or central macula with the lesion in the same frame, then the Optomap® should be used to photograph the naevus to record its position and ideally a colour image on the mydriatic camera should also be taken to record the appearance of the lesion.

Calculation of Tree Height and Canopy Crown from Drone Images Using Segmentation

Drone imaging, which is more cost-effective and controllable compared to airborne LiDAR, requires a lowcost camera and is used for capturing color images. From the overlapped color images, we produced two highresolution digital surface models over different test areas. After segmentation, we performed tree identification according to the method proposed by La et al. (2015), and computed the tree height and the canopy crown size. Compared with the field measurements, the computed results for the tree height in test area 1 (coniferous trees) were found to be accurate, while the results in test area 2 (deciduous coniferous trees) were found to be underestimated. The RMSE of the tree height was 0.84 m, and the width of the canopy crown was 1.51 m in test area 1. Further, the RMSE of the tree height was 2.45 m, and the width of the canopy crown was 1.53 m in test area 2. The experiment results validated the use of drone images for the extraction of a tree structure.

Visually lossless compression for Bayer color filter array using optimized Vector Quantization

Devices using single sensors to capture colour images are cheaper due to high cost of Charge Couple Device (CCD) sensors or Complementary Metal-Oxide Semiconductor (CMOS) sensors. Single sensor devices use Colour Filter Array (CFA) to sample one colour band at every pixel location. Demosaicking process is applied to interpolate the two missing colours from the surrounding. Typically compression is done on the demosaicked images which may not be efficient due to the individual compression of the different colour space. This work investigated compression of raw data before demosaicking and performs demosaicking to reconstruct the R, G, B bands later. A novel Vector Quantization (VQ) technique for encoding the wavelet decomposed image using Modified Artificial Bee Colony (ABC) optimization algorithm is proposed. The proposed technique is compared with Genetic Algorithm based VQ and ABC based quantization and with standard LBG and Lloyd algorithm. Results show higher Peak Signal-to-Noise Ratio (PSNR) indicating better reconstruction.

Multi-directional Hole Filling Method for Virtual View Synthesis

In 3D video applications, view synthesis is widely used for generating a virtual viewpoint images. However, the quality of synthesized image is dissimilar to captured color image. Furthermore, noise occurs at object boundary areas in synthesized images. In this paper, we propose a new hole filling method to improve the synthesized image quality. In order to synthesize an image without the noise, we remove the boundary noise by expanding it as a hole region. Based on the expanded hole region, we use proposed hole scanning and directional hole filling method in simultaneous time. Additionally, we compute a cost value among the candidate pixels to find out proper hole filling pixel. We conducted experiments on various test sequences for evaluation of the proposed method. The proposed method produces better objective results than other conventional hole filling methods.