Image Convolution Research Articles

Advanced Filtering and Enhancement Techniques for Diabetic Retinopathy Image Analysis

Diabetic retinopathy is a leading cause of visual impairment and blindness in diabetes sufferers. Early detection is crucial to prevent severe outcomes. This study presents an image processing method for retinal images to aid early detection. The method involves four steps: image enlargement, preprocessing, enhancement, and convolution. First, an algorithm enlarges the retinal image to increase resolution and reveal finer details. Preprocessing uses a min-max filtering algorithm to reduce noise and improve image quality. Next, specific pixel range enhancement techniques further refine the image and highlight relevant features. Finally, convolution with customized kernels detects and emphasizes areas indicating diabetic retinopathy, such as aneurysms and hemorrhages. Experimental results show improvement in image clarity and detail, enabling more accurate detection of diabetic retinopathy features. The correlation results are as follows: Filtering (0.35275, 0.20157, 0.4345), Enhancement (0.3214, 0.15823 0.34674), and Convolution (0.33542, 0.15758, 0.36826). The proposed algorithm enhances early detection and diagnosis by improving retinal image quality. Future work can optimize the algorithm and validate results with larger datasets, aiming to refine the determination of areas or pixel values relevant to diabetic retinopathy.

Pyrolysis front detection in carbon phenolic composites using x-ray computed tomography

Carbon phenolic composites are used as thermal protection systems (TPS) materials on space capsules to protect them from the hot aerothermal environment. The phenolic resin in the composite material decomposes (pyrolyzes) at low temperatures resulting in a pyrolysis front within the material. The detection of the pyrolysis front after exposure to heat has historically been achieved by physically sectioning cross-sections of the material. We combine the phase contrast retrieval method to reconstruct x-ray computed tomography scans along with image convolution to identify the pyrolysis front in carbon phenolic composites. Unlike the standard filtered back projection method that captures only the carbon phase, the phase contrast retrieval method uses both the attenuation coefficients and refractive indices to illuminate all three phases (carbon, resin, and voids) of carbon phenolic composites. Image convolution is applied on scans reconstructed using the phase contrast retrieval method to develop a density map of the composite to locate the pyrolysis front. The analysis is performed on a sample of phenolic impregnated carbon ablator that was tested in an arc-jet facility. For the sample analyzed, the depth of the pyrolysis front from the surface of the sample is calculated to be 2.150 ± 0.148 mm. Although the proposed approach is applied to detect the pyrolysis front, the tools can be used to illuminate the structure of any carbon phenolic composite, and we propose the use of the phase contrast retrieval method as a methodological standard to analyze carbon phenolic composites used on space capsules.

محاكاة مخططات التصفية المطبقة لتحسين الصور الفوتوغرافية

In signal processing, filters generally play a pivotal role to remove unwanted parts of the signal, such as random noise, or to extract useful parts of the signal, such as the components lying within a certain frequency range to enhance the performance and denoising scenario. This paper presents the implementation of linear and non-linear filtering schemes for noise reduction and images enhancement. This is achieved by performing convolution of a grayscale image with a mask filter of multiple sizes using MATLAB software. The average and median filters are implemented on the same image of an injected salt and pepper noise. In accordance with the findings, the nonlinear filters show more promising performance with a better peak-signal-noise ratio (PSNR) compared to the linear filters.

Nano-particles size measurement based on semantic segmentation via convolution neural network

Automatic recognition and size measurement of nanoparticles in SEM images have been an evolving field of study within the material informatics domain. These methods have been typically assessed using traditional image processing techniques. This paper presents a robust and fully automated pipeline for all researchers, particularly those in chemical engineering, based on image processing techniques and convolution neural networks. We evaluated our method on a dataset of 1,701 2D particle images, employing a three-fold split for training, validation, and testing. The proposed approach integrates semantic segmentation models (U-Net, LinkNet) with size measurement algorithms, achieving high accuracy even in the presence of noise or blurriness. Our results demonstrate the method effectiveness in precisely estimating particle sizes, with a Pearson correlation coefficient of 0.9633 and mean absolute error of 1.287e-06. The method’s robustness against poor-quality images and its fully automated procedures make it a valuable tool for researches in SEM images field.

A real-time and energy-efficient SRAM with mixed-signal in-memory computing near CMOS sensors

In-memory computing (IMC) represents a promising approach to reducing latency and enhancing the energy efficiency of operations required for calculating convolution products of images. This study proposes a fully differential current-mode architecture for computing image convolutions across all four quadrants, intended for deep learning applications within CMOS imagers utilizing IMC near the CMOS sensor. This architecture processes analog signals provided by a CMOS sensor without the need for analog-to-digital conversion. Furthermore, it eliminates the necessity for data transfer between memory and analog operators as convolutions are computed within modified SRAM memory. The paper suggests modifying the structure of a CMOS SRAM cell by incorporating transistors capable of performing multiplications between binary (−1 or +1) weights and analog signals. Modified SRAM cells can be interconnected to sum the multiplication results obtained from individual cells. This approach facilitates connecting current inputs to different SRAM cells, offering highly scalable and parallelized calculations. For this study, a configurable module comprising nine modified SRAM cells with peripheral circuitry has been designed to calculate the convolution product on each pixel of an image using a 3×3\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$3 \ imes 3$$\\end{document} mask with binary values (−1 or 1). Subsequently, an IMC module has been designed to perform 16 convolution operations in parallel, with input currents shared among the 16 modules. This configuration enables the computation of 16 convolutions simultaneously, processing a column per cycle. A digital control circuit manages both the readout or memorization of digital weights, as well as the multiply and add operations in real-time. The architecture underwent testing by performing convolutions between binary masks of 3 × 3 values and images of 32 × 32 pixels to assess accuracy and scalability when two IMC modules are vertically integrated. Convolution weights are stored locally as 1-bit digital values. The circuit was synthesized in 180 nm CMOS technology, and simulation results indicate its capability to perform a complete convolution in 3.2 ms, achieving an efficiency of 11,522 1-b TOPS/W (1-b tera-operations per second per watt) with a similarity to ideal processing of 96%.

Research on deep learning restoration algorithm of X-ray backscatter imaging based on virtual training dataset

The X-ray lobster eye lens, an innovative technique for focusing high-energy radiation, enables wide-field X-ray imaging. However, its inherent cross point spread function introduces noise and degradation into the resultant images. Conventional image restoration methods are inadequate for suppressing such noise. This paper introduces a backscatter image restoration technique utilizing a virtual training dataset. By convolving the point spread function (PSF) with an object to simulate the image degradation process, the method generates a multitude of convolved images for deep learning training, eliminating the need for manual annotation. Given the high structural similarity between the synthetic convolved images and actual backscatter images, the trained model effectively restores real backscatter images. The restoration process yields a structural similarity index (SSIM) of 0.86 and a mean intersection over union (MIoU) of 0.83 when compared to the reference images. This approach mitigates the limitations of sparse real backscatter datasets, substantially reducing image acquisition time, decreasing radiation flux, and enhancing system safety.

Design Efficient Vedic-Multiplier for Floating-Point MAC Module

Multiplication-accumulation (MAC) operation plays a crucial role in digital signal processing (DSP) applications, such as image convolution and filters, especially when performed on floating-point numbers to achieve high-level of accuracy. The performance of MAC module highly relies upon the performance of the multiplier utilized. This work offers a distinctive and efficient floating-point Vedic multiplier (VM) called adjusted-VM (AVM) to be utilized in MAC module to meet modern DSP demands. The proposed AVM is based on Urdhva-Tiryakbhyam-Sutra (UT-Sutra) approach and utilizes an enhanced design for the Brent-Kung carry-select adder (EBK-CSLA) to generate the final product. A (6*6)-bit AVM is designed first, then, it is extended to design (12*12)-bit AVM which in turns, utilized to design (24*24)-bit AVM. Moreover, the pipelining concept is used to optimize the speed of the offered (24*24)-bit AVM design. The proposed (24*24)-bit AVM can be used to achieve efficient multiplication for mantissa part in binary single-precision (BSP) floating-point MAC module. The proposed AVM architectures are modeled in VHDL, simulated, and synthesized by Xilinx-ISE14.7 tool using several FPGA families. The implementation results demonstrated a noticeable reduction in delay and area occupation by 33.16% and 42.42%, respectively compared with the most recent existing unpipelined design, and a reduction in delay of 44.78% compared with the existing pipelined design.

Research on Anti-breakage Target Detection Method of Transmission Line based on Improved YOLOv5

Abstract Illegal construction operations and sudden outbursts in transmission corridor protection areas pose a threat to the safe and stable operation of transmission lines. It is very important to identify the transmission line breakage risk to guarantee power grid security. The existing target detection model has many parameters and a large capacity, which makes it difficult to implement the deployment of edge terminals and cannot detect the risk of transmission line outbreaks in real time. This paper proposed a transmission line anti-breach detection method based on improved YOLOv5. Firstly, the CSPdarknet53 module is replaced with a lightweight MobilenetV3 module to reduce the model parameter number. The SPP module is replaced by SimSPPF module to realize the fast conversion of multi-scale image convolution features. Finally, the ECA attention mechanism is introduced to improve the ability of the model to focus on key features, thereby improving the overall performance of the model. Experiment results show that the proposed method has a smaller parameter number and a mAP value of 97.77%, which is the best overall performance, and provides support for the realization of outbreak risk warning of transmission lines based on edge intelligence.

Composite Neighbor-Aware Convolutional Metric Networks for Hyperspectral Image Classification.

Supervised classification of hyperspectral image (HSI) is generally required to obtain better performance in spectral-spatial feature learning by fully using complex pixel- and superpixel-level interdependencies with small labeled samples. Limited by the local regular convolutions, convolutional neural networks (CNNs) can only exploit information from the short-range Euclidean neighbors of a target, hindering the effectiveness of feature representation. In contrast, graph convolutional networks (GCNs) can learn long-range dependencies between non-Euclidean neighbors but usually require the input of a full graph constructed from a whole HSI, making GCNs must be trained in a full-batch manner with tremendous computational consumption. In this work, we propose a composite neighbor-aware convolutional metric network (CNCMN), aiming to learn each target's representation from its composite neighbors (i.e., both Euclidean and non-Euclidean neighbors) in a batchwise manner. Specifically, for each target in an HSI, its Euclidean neighbors are the pixels in the local square region centered on itself, and its non-Euclidean neighbors are several related nodes selected from the constructed full graph. Correspondingly, a composite convolution (CoConv) is proposed by coupling an image convolution and a graph convolution, which can perform flexible convolutions on those composite neighbors and extract adaptively fused features from them. Besides, to further boost classification, we also propose a mini-batch metric classifier to dynamically optimize interclass and intraclass distances of samples batch by batch, which is then combined with the CoConv to form the mini-batch CNCMN. Extensive experiments on three real-world HSIs demonstrate the advantages of the proposed method over mini-batch deep learning algorithms and have obtained the state-of-the-art performance in these fields. The code is available at: https://github.com/qichaoliu/HSI-CNCMN.

RedLionfish - fast Richardson-Lucy Deconvolution package for efficient point spread function suppression in volumetric data.

The experimental limitations with optics observed in many microscopy and astronomy instruments result in detrimental effects for the imaging of objects. This can be generally described mathematically as a convolution of the real object image with the point spread function that characterizes the optical system. The popular Richardson-Lucy (RL) deconvolution algorithm is widely used for the inverse process of restoring the data without these optical aberrations, often a critical step in data processing of experimental data. Here we present the versatile RedLionfish python package, that was written to make the RL deconvolution of volumetric (3D) data easier to run, very fast (by exploiting GPU computing capabilities) and with automatic handling of hardware limitations for large datasets. It can be used programmatically in Python/numpy using conda or PyPi package managers, or with a graphical user interface as a napari plugin.

Violence detection in crowd videos using nuanced facial expression analysis

Video analysis for violence detection is crucial, especially when dealing with crowd data, where the potential for severe mob attacks in sensitive areas is high. This paper proposes a solution utilizing Convolutional Restricted Boltzmann Machine (CRBM) for video analysis, integrating the strengths of Convolutional Neural Network (CNN) and Restricted Boltzmann Machine (RBM). By focusing on image patches rather than entire frames, the method addresses the challenge of object detection in crowded scenes. The CRBM combines deep-level image analysis from CNN with unsupervised feature extraction in RBM, facilitated by image convolution using Gabor filters in the hidden layer. Dropout regularization mitigates overfitting, enhancing model generality. Extracted features are inputted into an SVM classifier for face detection and a custom VGG16 model for emotion identification. Event probability is then determined through logistic regression based on facial expressions. Despite existing approaches for smart crowd behaviour identification, there remains a tradeoff between accuracy and processing time. Our proposed solution addresses this by employing proper frame preprocessing techniques for feature extraction. Validation using quantitative and qualitative metrics confirms the effectiveness of the approach.

Can processed images be used to determine the modulation transfer function and detective quantum efficiency?

The modulation transfer function (MTF) and detective quantum efficiency (DQE) of x-ray detectors are key Fourier metrics of performance, valid only for linear and shift-invariant (LSI) systems and generally measured following IEC guidelines requiring the use of raw (unprocessed) image data. However, many detectors incorporate processing in the imaging chain that is difficult or impossible to disable, raising questions about the practical relevance of MTF and DQE testing. We investigate the impact of convolution-based embedded processing on MTF and DQE measurements. We use an impulse-sampled notation, consistent with a cascaded-systems analysis in spatial and spatial-frequency domains to determine the impact of discrete convolution (DC) on measured MTF and DQE following IEC guidelines. We show that digital systems remain LSI if we acknowledge both image pixel values and convolution kernels represent scaled Dirac -functions with an implied sinc convolution of image data. This enables use of the Fourier transform (FT) to determine impact on presampling MTF and DQE measurements. It is concluded that: (i)the MTF of DC is always an unbounded cosine series; (ii)the slanted-edge method yields the true presampling MTF, even when using processed images, with processing appearing as an analytic filter with cosine-series MTF applied to raw presampling image data; (iii)the DQE is unaffected by discrete-convolution-based processing with a possible exception near zero-points in the presampling MTF; and (iv)the FT of the impulse-sampled notation is equivalent to the transform of image data.

Advancement in Soil Analysis for Tailored Tomato Cultivation Using Deep Learning & IoT

Abstract: In the agriculture sector, one of the major problems in the plants is its seed diseases. The seed diseases can be caused by various factors such as viruses, bacteria, fungus etc. Most of the farmers are unaware of such diseases. That's why the detection of various diseases of seeds is very essential to prevent the damages that it can make to the plants itself as well as to the farmers and the whole agriculture ecosystem. Regarding this practical issues, this research aimed to classify and detect the plant's diseases automatically especially for the tomato plant. As per the hardware requirement, PC Sis the major computing unit. Image processing is the key process of the project which includes image acquisition, adjusting image ROI, feature extraction and convolution neural network (CNN) based classification. Here, Python programming language, OPENCV library is used to manipulate raw input image. To train on CNN architecture and creating a machine learning model that can predict the type of diseases, image data is collected from the authenticated online source. On providing the soil nutrient values obtained using NPK sensor, the recommendation of the suitable crop is displayed. The proposed system takes nitrogen, phosphorus, potassium values from the soil and recommends crop that is best suitable for the soil.

IoT based technological support for moisture stress experimental study on plants with stress prediction using deep learning

BACKGROUND: The increased depletion of ground water resources poses the risk of higher moisture stress environment for agriculture crops. The rapid increase in the moisture stress situation imposes the need of efficient agricultural research on determining the impact of moisture stress on variety of crops. OBJECTIVE: The prime objective of the proposed work is building an IoT based Plant Phenotyping Device for moisture stress experimental study on variety of crops with deep learning model for stress response detection. METHODS: In this work, IoT technology is used for building a proposed system for conducting the moisture stress experiments on plants and adopting the image processing and convolution neural network based model for stress prediction. RESULTS: The accuracy of the proposed system was experimentally evaluated and empirical results were satisfactory in maintaining the desired level of moisture stress. Performance analysis of LeNet, AlexNet, customized AlexNet and GoogLeNet CNN models were carried out with hyper-parameters variations on the leaf images. GoogLeNet achieved a better validation accuracy of 96% among other models. The trained GoogLeNet model is used for predicting the moisture stress response and predicted results were matched with manual observation of stress response. SIGNIFICANCE: The affirmative results of proposed system would increases its adoption for in-house precision agriculture and also for conducting various moisture stress experiments on variety of crops. The confirmative detection of moisture stress tolerance level of plant provides knowledge on minimum level of water requirement for plant growth, which in-turn save the water by avoiding excess watering to plants.

Image convolution techniques integrated with YOLOv3 algorithm in motion object data filtering and detection.

In order to address the challenges of identifying, detecting, and tracking moving objects in video surveillance, this paper emphasizes image-based dynamic entity detection. It delves into the complexities of numerous moving objects, dense targets, and intricate backgrounds. Leveraging the You Only Look Once (YOLOv3) algorithm framework, this paper proposes improvements in image segmentation and data filtering to address these challenges. These enhancements form a novel multi-object detection algorithm based on an improved YOLOv3 framework, specifically designed for video applications. Experimental validation demonstrates the feasibility of this algorithm, with success rates exceeding 60% for videos such as "jogging", "subway", "video 1", and "video 2". Notably, the detection success rates for "jogging" and "video 1" consistently surpass 80%, indicating outstanding detection performance. Although the accuracy slightly decreases for "Bolt" and "Walking2", success rates still hover around 70%. Comparative analysis with other algorithms reveals that this method's tracking accuracy surpasses that of particle filters, Discriminative Scale Space Tracker (DSST), and Scale Adaptive Multiple Features (SAMF) algorithms, with an accuracy of 0.822. This indicates superior overall performance in target tracking. Therefore, the improved YOLOv3-based multi-object detection and tracking algorithm demonstrates robust filtering and detection capabilities in noise-resistant experiments, making it highly suitable for various detection tasks in practical applications. It can address inherent limitations such as missed detections, false positives, and imprecise localization. These improvements significantly enhance the efficiency and accuracy of target detection, providing valuable insights for researchers in the field of object detection, tracking, and recognition in video surveillance.

DETECTING THE SECURITY LEVEL OF VARIOUS CRYPTOSYSTEMS USING MACHINE LEARNING MODELS

In contemporary data management, cloud storage is widely adopted, yet it remains susceptible to security vulnerabilities. Utilizing cryptography techniques is essential for enhancing data security in such environments. One promising approach is hybrid cryptography, which combines multiple algorithms to bolster protection. Our proposed solution integrates RDH with DES algorithms, leveraging their respective strengths to fortify data security before it's stored in the cloud. Extensive testing has validated the efficacy of this approach. Specifically, we introduce an RDH with DES block-based transformation algorithm tailored for image content protection. Notably, our framework enables direct image retrieval and convolution on the content-protected images, enhancing usability alongside security. Keywords: deployment models, Infrastructure as a service, cryptosystems, machine learning

Multilevel Ferroelectric Domain Wall Memory for Neuromorphic Computing

AbstractLow‐power and parallel processing Neuromorphic computing that imitates on the human brain's extraordinary data processing and learning capabilities is promising in non‐von Neumann application platforms in the post‐Moore era. Ferroelectric domain walls appearing as nanoscale topological defects in solids exhibit coexisting ordering parameters besides the spontaneous polarization, leading to the emergence of rich physics and electronic effects in the development of neuromorphic electronics that go beyond the conventional CMOS. In this study, the four‐state domain wall memory is developed on LiNbO3 ferroelectric thin films integrated with Si substrates, where each state can be stably manipulated at a specific low voltage, showcasing outstanding fatigue resistance and retention performance. The constructed crossbar array that functions as a kernel for image convolution can be used to implement processing operations like image filtering, sharpness enhancement, and edge detection. Additional simulation from a convolutional neural network model shows 96.98% accuracy on the Modified National Institute of Standards and Technology handwritten digits dataset. The stable multi‐state domain wall memory provides a new approach for computing and promotes research in domain wall electronics to meet future enormous demands of big data.

VCSEL-based photonic spiking neural networks for ultrafast detection and tracking

Inspired by efficient biological spike-based neural networks, we demonstrate for the first time the detection and tracking of target patterns in image and video inputs at high-speed rates with networks of multiple artificial spiking optical neurons. Using photonic systems of in-parallel spiking vertical cavity surface emitting lasers (VCSELs), we demonstrate the implementation of multiple convolutional kernel operators which, in combination with optical spike signalling, enable the detection and tracking of target features in images/video feeds at an ultrafast photonic operation speed of 1 ns per pixel. Alongside a single layer optical spiking neural network (SNN) demonstration, a multi-layer network of photonic (GHz-rate) spike-firing neurons is reported where the photonic system successfully tracks a large complex feature (Handwritten Digit 3). The consecutive photonic layers perform spike-enabled image reduction and convolution operations, and interact with a software-implemented SNN, that learns the feature patterns that best identify the target to provide a high detection efficiency even in the presence of a distractor feature. This work therefore highlights the effectiveness of combining neuromorphic photonic hardware and software SNNs, for efficient learning and ultrafast operation, thanks to the use of spiking light signals, towards tackling complex AI and computer vision problems.

STGIC: A graph and image convolution-based method for spatial transcriptomic clustering.

Spatial transcriptomic (ST) clustering employs spatial and transcription information to group spots spatially coherent and transcriptionally similar together into the same spatial domain. Graph convolution network (GCN) and graph attention network (GAT), fed with spatial coordinates derived adjacency and transcription profile derived feature matrix are often used to solve the problem. Our proposed method STGIC (spatial transcriptomic clustering with graph and image convolution) is designed for techniques with regular lattices on chips. It utilizes an adaptive graph convolution (AGC) to get high quality pseudo-labels and then resorts to dilated convolution framework (DCF) for virtual image converted from gene expression information and spatial coordinates of spots. The dilation rates and kernel sizes are set appropriately and updating of weight values in the kernels is made to be subject to the spatial distance from the position of corresponding elements to kernel centers so that feature extraction of each spot is better guided by spatial distance to neighbor spots. Self-supervision realized by Kullback-Leibler (KL) divergence, spatial continuity loss and cross entropy calculated among spots with high confidence pseudo-labels make up the training objective of DCF. STGIC attains state-of-the-art (SOTA) clustering performance on the benchmark dataset of 10x Visium human dorsolateral prefrontal cortex (DLPFC). Besides, it's capable of depicting fine structures of other tissues from other species as well as guiding the identification of marker genes. Also, STGIC is expandable to Stereo-seq data with high spatial resolution.

Compensation Method for Correcting the Topography Convolution of the 3D AFM Profile Image of a Diffraction Grating

Any 3D AFM image is a convolution of the geometry of the AFM tip and the profile of the scanned sample, especially when the dimensions of the scanned sample are comparable to those of the AFM tip shape. The precise profile of the scanned sample can be extracted from the 3D AFM image if the geometry of the AFM tip is known. Therefore, in order to separate the geometry of the AFM probe tip from the 3D AFM image of a diffraction grating with a rectangular profile and to correct for the topographic convolutions induced by the AFM probe tip, a method is used to quantitatively evaluate the geometry of the AFM probe tip, including the tip radius and the included angle. A model for reconstructing the measured AFM image is proposed to correct topography convolutions caused by the AFM tip shape when scanning a diffraction grating with rectangular profiles. A series of experiments were performed to verify the effectiveness of the proposed AFM tip geometry evaluation method, and comparison experiments were conducted to demonstrate the feasibility and reliability of the proposed reconstruction model.