Image Processing Operations Research Articles

Efficient New Conjugate Gradient Methods for Removing Impulse Noise Images

In most applications, denoising image is fundamental to subsequent image processing operations. In this research, we derivation a new formula of conjugate gradient methods based on the quadratic model. The fact that the search direction created at each iteration of the proposed approach is descending and independent of the line search makes it interesting. The use of Wolfeconditions also determines the global convergence of the suggested approach. To prove the viability of the suggested approach, comparison tests on impulse noise reduction are given.

Traffic Light Detection and Recognition Method Based on YOLOv5s and AlexNet

Traffic light detection and recognition technology are of great importance for the development of driverless systems and vehicle-assisted driving systems. Since the target detection algorithm has the problems of lower detection accuracy and fewer detection types, this paper adopts the idea of first detection and then classification and proposes a method based on YOLOv5s target detection and AlexNet image classification to detect and identify traffic lights. The method first detects the traffic light area using YOLOv5s, then extracts the area and performs image processing operations, and finally feeds the processed image to AlexNet for recognition judgment. With this method, the shortcomings of the single-target detection algorithm in terms of low recognition rate for small-target detection can be avoided. Since the homemade dataset contains more low-light images, the dataset is optimized using the ZeroDCE low-light enhancement algorithm, and the performance of the network model trained after optimization of the dataset can reach 99.46% AP (average precision), which is 0.07% higher than that before optimization, and the average accuracy on the traffic light recognition dataset can reach 87.75%. The experimental results show that the method has a high accuracy rate and can realize the recognition of many types of traffic lights, which can meet the requirements of traffic light detection on actual roads.

Mathematical insights into the original Retinex algorithm for image enhancement.

The Retinex theory, originally developed by Land and McCann as a computation model of the human color sensation, has become, with time, a pillar of digital image enhancement. In this area, the Retinex algorithm is widely used to improve the quality of any input image by increasing the visibility of its content and details, enhancing its colorfulness, and weakening, or even removing, some undesired effects of the illumination. The algorithm was originally described by its creators in terms of a sequence of image processing operations and was not fully formalized mathematically. Later, works focusing on aspects of the original formulation and adopting some of its principles tried to frame the algorithm within a mathematical formalism: this yielded every time a partial rendering of the model and resulted in several interesting model variants. The purpose of the present work is to fill a gap in the Retinex-related literature by providing a complete mathematical formalization of the original Retinex algorithm. The overarching goals of this work are to provide mathematical insights into the Retinex theory, promote awareness of the use of the model within image enhancement, and enable better appreciation of differences and similarities with later models based on Retinex principles. For this purpose, we compare our model with others proposed in the literature, paying particular attention to the work published in 2005 by Provenzi and others.

Beyond PRNU: Learning Robust Device-Specific Fingerprint for Source Camera Identification.

Source-camera identification tools assist image forensics investigators to associate an image with a camera. The Photo Response Non-Uniformity (PRNU) noise pattern caused by sensor imperfections has been proven to be an effective way to identify the source camera. However, the PRNU is susceptible to camera settings, scene details, image processing operations (e.g., simple low-pass filtering or JPEG compression), and counter-forensic attacks. A forensic investigator unaware of malicious counter-forensic attacks or incidental image manipulation is at risk of being misled. The spatial synchronization requirement during the matching of two PRNUs also represents a major limitation of the PRNU. To address the PRNU’s fragility issue, in recent years, deep learning-based data-driven approaches have been developed to identify source-camera models. However, the source information learned by existing deep learning models is not able to distinguish individual cameras of the same model. In light of the vulnerabilities of the PRNU fingerprint and data-driven techniques, in this paper, we bring to light the existence of a new robust data-driven device-specific fingerprint in digital images that is capable of identifying individual cameras of the same model in practical forensic scenarios. We discover that the new device fingerprint is location-independent, stochastic, and globally available, which resolves the spatial synchronization issue. Unlike the PRNU, which resides in the high-frequency band, the new device fingerprint is extracted from the low- and mid-frequency bands, which resolves the fragility issue that the PRNU is unable to contend with. Our experiments on various datasets also demonstrate that the new fingerprint is highly resilient to image manipulations such as rotation, gamma correction, and aggressive JPEG compression.

Automating implant reconstruction in interstitial brachytherapy of the breast: A hybrid approach combining electromagnetic tracking and image segmentation

Background and purposeTo develop a method for automatic reconstruction of catheter implants in interstitial brachytherapy (iBT) of the breast by means of electromagnetic tracking (EMT) with the goal of making treatment planning as time-effective and accurate as possible. Materials and methodsThe implant geometry of 64 patients was recorded using an afterloader prototype with EMT functionality immediately after the planning CT. EMT data were transferred to the CT image space by rigidly registering the catheter fixation buttons as landmarks. To further improve reconstruction accuracy, the EMT reconstruction points were used as starting points to define small regions of interest (ROI) in the CT image. Within these ROIs, the catheter track was segmented in the CT using image processing operations such as thresholding and blob detection, thus refining the reconstruction. The perpendicular distance between the refined EMT implant reconstruction points and the manually reconstructed catheters by an experienced treatment planner was calculated as a measure of their geometric agreement. ResultsGeometrically, the refined EMT based implant reconstruction shows excellent agreement with the manual reconstruction. The median distance across all patients is 0.25 mm and the 95th percentile is 1 mm. Refinement takes approximately 0.5 s per reconstruction point and typically does not exceed 3 min per implant at no user interaction. ConclusionThe refined EMT based implant reconstruction proved to be extremely accurate and fast compared to manual reconstruction. The presented procedure can in principle be easily transferred to clinical routine and therefore has enormous potential to provide significant time savings in iBT treatment planning whilst improving reconstruction accuracy.

A Systematic Analysis for Cardiovascular Disease Classification Using Deep Learning

The processing of medical data has profited from automation and contemporary computing breakthroughs, which have given rise to several novel instructional approaches. In both traditional and cutting-edge disciplines, deep learning has emerged as a cutting-edge machine learning paradigm. Deep learning algorithms have developed into supervised, semi-supervised, and unsupervised modes for a variety of real-time applications. The technology has shown to be broadly applicable in image processing, computer vision, medical diagnostics, robotics, and control operation. Deep learning is essential in the field of medical science for recognizing various diseases and solving health issues. The deep learning revolution will significantly alter cardiovascular imaging within the next decade. To ensure that deep learning may meaningfully impact clinical practice, it is crucial for medical practitioners to stay up with these breakthroughs. This evaluation is intended to be a preliminary step in that process. In this study cardiovascular disease diagnosis and classification have been examined using different state-of-the-art deep learning approaches. Deep Neural Networks can be used to improve the classification of heart disease as a whole in a crucial field called heart illness. The classification process can be carried out using a variety of methods, such as DL, KNN, SVM, ANN Nave Bayes, Random Forest, SSD, DNN, and TDNN where DL exhibited maximum accuracy. Although deep learning-based automated cardiovascular classification algorithms have shown highly accurate results, they have not yet been widely used by healthcare professionals.

Using the spectral approach in image and arbitrary data processing

Introduction: The transfer of image processing operations to the spectral region is considered. Due to the duality of information representation, coding, filtering, compression and other transformations are possible both in the space of an object and in the region of spatial frequencies, in relation to its spatial spectrum. Purpose: To evaluate the efficiency of performing image processing operations and arbitrary digital information in the spectral domain. Results: A method for spectral holographic coding is proposed, which provides a gain in noise immunity by 7–8 dB and has less complexity of coding and decoding when changing redundancy over a wide range. During the coding process, the source data block is replaced by a code word, which is a linear one-dimensional hologram of a virtual point source. For transmission over a communication channel, we synthesize a signal with a given spectrum, the shape of which is a one-dimensional hologram, with its values rounded to one bit and the hologram itself being a sequence of zeros and ones – one means the presence of the corresponding harmonic in the spectrum, zero means the absence. To create a signal with such a line spectrum, it suffices to add a set of harmonics of equal amplitude with numbers corresponding to the position numbers of the units in the hologram. This operation is one of the types of orthogonal frequency division multiplexing, characterized in that the frequencies of the orthogonal subcarriers are in a multiple ratio, and amplitude shift keying is used as digital modulation. We propose a spectral method for image compression based on a detailed analysis of the spatial spectrum of an image and the removal of a large number of insignificant areas from it. Practical relevance: The reduction in the amount of information and, accordingly, the size of the image can be 4–8 times or more with a slight decrease in image quality.

Head pose healthiness prediction using a novel image quality based stacked autoencoder

This paper introduces an approach aiming to determine head pose healthiness of computer users. The main contributions of this paper are: 1) Image Quality Assessment (IQA) based Stacked Autoencoder (referred to as IQASAE) which adjusts the value of learning rate based on the quality of images; 2) Head Pose Healthiness Prediction (HPHP) framework which leverages the proposed IQASAE algorithm in combination with image processing operations; 3) A set of features suitable for face analysis applications; 4) Ontology-driven semantic framework which enables further exploiting pose estimation results within applications in synergy with healthcare expert domain knowledge about pose healthiness. Our framework was evaluated on both offline (BIWI and AFLW) and online (our own, collected using Arduino) datasets. Furthermore, it was compared to several state-of-art methods, including Multi-Layer Perceptron (MLP), CART, Random Forest, Convolutional Neural Networks (CNN), Temporal Deep Learning Model (TDLM), hybrid CNN with Support Vector Machine (SVM), Quatnet and Trinet. According to the achieved experimental results, it reaches accuracy up to 79.63% outperforming all of them, except Quatnet and Trinet. However, the main advantages of IQASAE compared to state-of-art methods are: 1) it does not require selection of features, so the processing time is reduced, 2) utilizing angle between chin and mouth reduces training time for SAE, 3) leveraging vector-based feature set to create training data resulted in a significant improvement, especially in offline facial images.

Continually reactivating iterative-projection method for instantiating microstructure from two-point statistics

Processing-structure (PS) linkages play a significant role in materials development. Nevertheless, currently there is a fascinating but of bottleneck task in terms of instantiating microstructure to understand and validate PS linkages. Its challenge lies on the lack of a robust method in computation efficiency and accuracy. Inspired from several advanced techniques such as hybrid input-output (HIO) algorithm, image processing operations and different-phase neighbors-based pixel swapping rule, we proposed a continually reactivating iterative-projection process (CRIP) method to address the challenge above. The output at each iteration in CRIP is continuously improved by eliminating and moving isolated or noise pixels towards an error-reduction direction to activate the iterative-projection process of the next iteration. The performance of the method was examined on two microstructure examples. It shows at least 99.86% improvement in convergence time compared with the classical simulated annealing algorithm and around 6.14 ×10−4 reduction in error compared with the traditional HIO algorithm. More importantly, CRIP is suitable for the task of instantiating microstructure from periodic two-point statistics predicted by PS linkage, which has been demonstrated by applying it to two experimental datasets of Ni-based superalloys and dual-phase steels. Our proposed method has the advantages of high efficiency, easy operation, reliable accuracy and remarkable generalization ability.

Centroid based person detection using pixelwise prediction of the position

Implementations of person detection in tracking and counting systems tend towards processing of orthogonally captured images on edge computing devices. The ellipse-like shape of heads in orthogonally captured images inspired us to predict head centroids to determine positions of persons in images. We predict the centroids using a fully convolutional network (FCN). We combine the FCN with simple image processing operations to ensure fast inference of the detector. We experiment with the size of the FCN output to further decrease the inference time. We compare the proposed centroid-based detector with bounding box-based detectors on head detection task in terms of the inference time and the detection performance. We propose a performance measure which allows quantitative comparison of the two detection approaches. For the training and evaluation of the detectors, we form original datasets of 8000 annotated images, which are characterized by high variability in terms of lighting conditions, background, image quality, and elevation profile of scenes. We propose an approach which allows simultaneous annotation of the images for both bounding box-based and centroid-based detection. The centroid-based detector shows the best detection performance while keeping edge computing standards.

CFS-crafter: An open-source tool for creating and analyzing images for continuous flash suppression experiments.

Continuous flash suppression (CFS) is a popular masking technique used to manipulate visual awareness. By presenting a rapidly changing stimulus to one eye (the 'mask'), a static image viewed by the other (the 'target') may remain invisible for many seconds. This effectiveness affords a means to assess unconscious visual processing, leading to the widespread use of CFS in several basic and clinical sciences. However, the lack of principled stimulus selection has impeded generalization of conclusions across studies, as the strength of interocular suppression is dependent on the spatiotemporal properties of the CFS mask and target. To address this, we created CFS-crafter, a point-and-click, open-source tool for creating carefully controlled CFS stimuli. The CFS-crafter provides a streamlined workflow to create, modify, and analyze mask and target stimuli, requiring only a rudimentary understanding of image processing that is well supported by help files in the application. Users can create CFS masks ranging from classic Mondrian patterns to those comprising objects or faces, or they can create, upload, and analyze their own images. Mask and target images can be custom-designed using image-processing operations performed in the frequency domain, including phase-scrambling and spatial/temporal/orientation filtering. By providing the means for the customization and analysis of CFS stimuli, the CFS-crafter offers controlled creation, analysis, and cross-study comparison. Thus, the CFS-crafter-with its easy-to-use image processing functionality-should facilitate the creation of visual conditions that allow a principled assessment of hypotheses about visual processing outside of awareness.

The incremental online k-means clustering algorithm and its application to color quantization

Color quantization is a common image processing operation with various applications in computer graphics, image processing, and computer vision. Color quantization is essentially a large-scale combinatorial optimization problem. Many clustering algorithms, both of hierarchical and partitional types, have been applied to this problem since the 1980s. In general, hierarchical color quantization algorithms are faster, whereas partitional ones produce better results provided that they are initialized properly. In this paper, we propose a novel partitional color quantization algorithm based on a binary splitting formulation of MacQueen’s online k-means algorithm. Unlike MacQueen’s original algorithm, the proposed algorithm is both deterministic and free of initialization. Experiments on a diverse set of public test images demonstrate that the proposed algorithm is significantly faster than two popular batch k-means algorithms while yielding nearly identical results. In other words, unlike previously proposed k-means variants, our algorithm addresses both the initialization and acceleration issues of k-means without sacrificing the simplicity of the algorithm. The presented algorithm may be of independent interest as a general-purpose clustering algorithm.

Secure sparse watermarking on DWT-SVD for digital images

This paper suggests a sparse watermark image embedding scheme on the singular values of the pre-selected wavelet sub-band coefficients of the digital images. Dictionary learning (DL) method is used to make the watermark image sparse. Then a watermark decoder is designed based on the theory of compressed sensing (CS) in the framework of alternating direction method of multiplier (ADMM). The integration of DL and CS offers two benefits, firstly DL combined with singular value decomposition (SVD), enables protection against false positive detection (FPD) where the sparse watermark enhances embedding capacity at high imperceptibility of the hidden data. Simulation results show high visual quality for the watermarked images (PSNR = 60.45 dB, SSIM = 0.9997 and FSIM = 0.9979) and robustness (NCC value for the decoded watermark is ∼ 0.9) against various image processing operations including an unauthorized decoding - a typical value of 85% CS measurements lead to a watermarked image results in a FPD value of 0.42 that shows a visually indistinguishable watermark.

An Efficient Approach of Finger Knuckle Print Based Recognition Using Googlenet Model

Abstract: The need of personal recognition and identification became necessary in order to keep intruders away and to allow only the legitimate persons in order to protect privacy and to secure data from being altered. But, with the technological evolution and increasing risk of data theft, it has been a mandatory demand. A persons’s identity is not limited to his/her name, password or PIN but today, their biometrics are being used as an integral part of their identification. The emergence of Deep learning, a subset of Artificial Intelligence, brought a revolution in the field of computer vision and biometric identification. The Convolutional Neural Network (CNN) models of Deep learning possess a resemblance to the human brain and it brought a revolution in the field of computer vision and biometric identification. I adopted a transfer learning approach by using a pretrained CNN model, GoogleNetfor conducting my experiment. The GoogleNet model automatically does all the image processing operations as well as the extraction of features too. I chose a very unique human biometric trait for this experiment, which is a person’s Finger Knuckleprint (FKP). It bears a complex pattern and unique structures. I requested the knuckleprint samples from The Hong Kong Polytechnic University and I used its Contactless Finger Knuckle Images Database (Version 1.0). An excellent result obtained by conducting this experiment. Keyword: Finger Knuckle Print, Convolution, GoogleNet, Feature Extraction, Classifier.

Restorable Image Operators with Quasi-Invertible Networks

Image operators have been extensively applied to create visually attractive photos for users to share processed images on social media. However, most image operators often smooth out details or generate textures after the processing, which removes the original content and raises challenges for restoring the original image. To resolve this issue, we propose a quasi-invertible model that learns common image processing operators in a restorable fashion: the learned image operators can generate visually pleasing results with the original content embedded. Our model is trained on input-output pairs that represent an image processing operator's behavior and uses a network that consists of an invertible branch and a non-invertible branch to increase our model's approximation capability. We evaluate the proposed model on ten image operators, including detail enhancement, abstraction, blur, photographic style, and non-photorealistic style. Extensive experiments show that our approach outperforms relevant baselines in the restoration quality, and the learned restorable operator is fast in inference and robust to compression. Furthermore, we demonstrate that the invertible operator can be easily applied to practical applications such as restorable human face retouching and highlight preserved exposure adjustment.

Development of automatic detection model for stem-end rots of ‘Hass’ avocado fruit using X-ray imaging and image processing

Postharvest qualitative avocado losses attributable to stem-end rot are serious problems in various main production areas. Postharvest inspection of avocadoes is still commonly performed by hand; this process depends upon only slight differences in fruit appearance or firmness, and it can lead to misclassification. The development of a non-destructive stem-end rot detection technique would help improve objective inspection and reduce misclassification. Therefore, this study proposes an X-ray machine vision approach to automatically detect stem-end rot in avocados. Four-hundred-and-twenty-two X-ray images of fully ripened ‘Hass’ avocadoes were obtained to develop and validate an automatic classification model. The stem-end rot features visible in the X-ray images were extracted using two image processing operations: image segmentation and fruit-outline evaluation. Then, an extremely randomized trees model classified rot development into two classes (< 20% and ≥ 20%) using two features extracted from the X-ray image. The overall classification accuracy was 0.89 with an area under the curve of 0.93 and a recall of 0.89, which suggests the possibility of performing stem-end rot inspections via X-ray imaging and image processing.

Automatic Detection of Coseismic Landslides Using a New Transformer Method

Earthquake-triggered landslides frequently occur in active mountain areas, which poses great threats to the safety of human lives and public infrastructures. Fast and accurate mapping of coseismic landslides is important for earthquake disaster emergency rescue and landslide risk analysis. Machine learning methods provide automatic solutions for landslide detection, which are more efficient than manual landslide mapping. Deep learning technologies are attracting increasing interest in automatic landslide detection. CNN is one of the most widely used deep learning frameworks for landslide detection. However, in practice, the performance of the existing CNN-based landslide detection models is still far from practical application. Recently, Transformer has achieved better performance in many computer vision tasks, which provides a great opportunity for improving the accuracy of landslide detection. To fill this gap, we explore whether Transformer can outperform CNNs in the landslide detection task. Specifically, we build a new dataset for identifying coseismic landslides. The Transformer-based semantic segmentation model SegFormer is employed to identify coseismic landslides. SegFormer leverages Transformer to obtain a large receptive field, which is much larger than CNN. SegFormer introduces overlapped patch embedding to capture the interaction of adjacent image patches. SegFormer also introduces a simple MLP decoder and sequence reduction to improve its efficiency. The semantic segmentation results of SegFormer are further improved by leveraging image processing operations to distinguish different landslide instances and remove invalid holes. Extensive experiments have been conducted to compare Transformer-based model SegFormer with other popular CNN-based models, including HRNet, DeepLabV3, Attention-UNet, U2Net and FastSCNN. SegFormer improves the accuracy, mIoU, IoU and F1 score of landslide detectuin by 2.2%, 5% and 3%, respectively. SegFormer also reduces the pixel-wise classification error rate by 14%. Both quantitative evaluation and visualization results show that Transformer is capable of outperforming CNNs in landslide detection.

Detection of image manipulation with convolutional neural network and local feature descriptors

In recent times, numerous digital image manipulation detection approaches have been proposed to detect which processing operations were applied to manipulate digital images. Most of these approaches consider the situation in which an image is manipulated by only one manipulation operation. However, practical image manipulation often involves multiple manipulation operations. It is important to detect multiple image manipulation operations and the order in which they were applied to establish the origin and genuineness of a given image as well as the processing history it has gone through. In this article, we proposed a new method to determine multiple image processing operation and operation chains based on convolutional neural network (CNN) and local optimal oriented pattern (LOOP). The proposed method is based on CNN and LOOP in which CNN extracts and learns image manipulation traces from the LOOP maps of the input images that are classified using softmax, extra-tree, and extreme gradient boosting (XGBOOST) classifiers. Detailed experiments show that the proposed model can attain overall detection accuracies of 99.81% and 99.15% in identifying different image manipulations and manipulation operation chains, respectively

Cotton Yield Estimation From Aerial Imagery Using Machine Learning Approaches.

Estimation of cotton yield before harvest offers many benefits to breeding programs, researchers and producers. Remote sensing enables efficient and consistent estimation of cotton yields, as opposed to traditional field measurements and surveys. The overall goal of this study was to develop a data processing pipeline to perform fast and accurate pre-harvest yield predictions of cotton breeding fields from aerial imagery using machine learning techniques. By using only a single plot image extracted from an orthomosaic map, a Support Vector Machine (SVM) classifier with four selected features was trained to identify the cotton pixels present in each plot image. The SVM classifier achieved an accuracy of 89%, a precision of 86%, a recall of 75%, and an F1-score of 80% at recognizing cotton pixels. After performing morphological image processing operations and applying a connected components algorithm, the classified cotton pixels were clustered to predict the number of cotton bolls at the plot level. Our model fitted the ground truth counts with an R2 value of 0.93, a normalized root mean squared error of 0.07, and a mean absolute percentage error of 13.7%. This study demonstrates that aerial imagery with machine learning techniques can be a reliable, efficient, and effective tool for pre-harvest cotton yield prediction.

A novel approach of ceramic tile crack detection using morphological operations

In ceramic tiles manufacturing industry, tiles are manufactured at large scale which makes it more challenging to ensure the quality of each tile according to the set standards. Mostly, Statistical Process Control (SPC) is used by tile manufacturers at each step to monitor various processing parameters. SPC procedures are implemented manually that requires sufficient number of experienced human resource to identify defected tiles from a batch of tiles. The manual inspection also gives low accuracy of defect detection due to human errors and hard environment. Considering these drawbacks, in this paper an automated defect detection method is proposed which is based on image processing and morphological operation to ensure the quality and standard of tiles. The proposed method resizes and converts the input RGB image into grayscale image and removes any possible noisy artifacts. An edge detection algorithm is applied on grayscale image to enhance the edges representing the cracks. Afterwards, morphological erosion and dilation operations are applied, one at a time, to get two intermediate images. Finally, edges are detected by subtracting eroded intermediate image from dilated intermediate image. For detection, the proposed algorithm does not require any separate reference image. The algorithm is tested on an image set of sixty different defected tile images and attained 92% average detection accuracy.