Area Of Image Processing Research Articles

FaceSwap based DeepFakes Detection

The progression of Machine Learning (ML) has introduced new trends in the area of image processing. Moreover, ML presents lightweight applications capable of running with minimum computational resources like Deepfakes, which generates widely manipulated multimedia data. Deepfakes introduce a serious danger to the confidentiality of humans and bring extensive religion, sect, and political anxiety. The FaceSwapp-based deepfakes are problematic to be identified by people due to their realism. Hence, the researchers are facing serious issues to detect visual manipulations. In the presented approach, we have proposed a novel technique for recognizing FaceSwap-based deepfakes. Initially, landmarks are computed from the input videos by employing Dlib-library. In the next step, the computed landmarks are used for training two classifiers namely Support Vector Machine (SVM) and Artificial Neural Network (ANN). The reported results demonstrate that SVM works well than ANN in classifying the manipulated samples due to its power to deal with over-fitted training data.

A nonlinear fractional partial differential equation for image inpainting

Image inpainting is an important research area in image processing. Its main purpose is to supplement missing or damaged domains of images using information from surrounding areas. This step can be performed by using nonlinear diffusive filters requiring a resolution of partial differential evolution equations. In this paper, we propose a filter defined by a partial differential nonlinear evolution equation with spatial fractional derivatives. Due to this, we were able to improve the performance obtained by known inpainting models based on partial differential equations and extend certain existing results in image processing. The discretization of the fractional partial differential equation of the proposed model is carried out using the shifted Grünwald–Letnikov formula, which allows us to build stable numerical schemes. The comparative analysis shows that the proposed model produces an improved image quality better or comparable to that obtained by various other efficient models known from the literature.

Image Segmentation Based on Block Level and Hybrid Directional Local Extrema

In the recent decade, the digitalization of various tasks has added great flexibility to human lifestyle and has changed daily routine activities of communities. Image segmentation is a key step in digitalization. Segmentation plays a key role in almost all areas of image processing, and various approaches have been proposed for image segmentation. In this paper, a novel approach is proposed for image segmentation using a nonuniform adaptive strategy. Region-based image segmentation along with a directional binary pattern generated a better segmented image. An adaptive mask of 8 × 8 was circulated over the pixels whose bit value was 1 in the generated directional binary pattern. Segmentation was performed in three phases: first, an image was divided into sub-images or image chunks; next, the image patches were taken as input, and an adaptive threshold was generated; and finally the image chunks were processed separately by convolving the adaptive mask on the image chunks. Gradient and Laplacian of Gaussian algorithms along with directional extrema patterns provided a double check for boundary pixels. The proposed approach was tested on chunks of varying sizes, and after multiple iterations, it was found that a block size of 8 × 8 performs better than other chunks or block sizes. The accuracy of the segmentation technique was measured in terms of the count of ill regions, which were extracted after the segmentation process.

Image Compression Using Stochastic-AFD Based Multisignal Sparse Representation.

Adaptive Fourier decomposition (AFD) is a newly developed signal processing tool that can adaptively decompose any single signal using a Szegö kernel dictionary. To process multiple signals, a novel stochastic-AFD (SAFD) theory was recently proposed. The innovation of this study is twofold. First, a SAFD-based general multi-signal sparse representation learning algorithm is designed and implemented for the first time in the literature, which can be used in many signal and image processing areas. Second, a novel SAFD based image compression framework is proposed. The algorithm design and implementation of the SAFD theory and image compression methods are presented in detail. The proposed compression methods are compared with 13 other state-of-the-art compression methods, including JPEG, JPEG2000, BPG, and other popular deep learning-based methods. The experimental results show that our methods achieve the best balanced performance. The proposed methods are based on single image adaptive sparse representation learning, and they require no pre-training. In addition, the decompression quality or compression efficiency can be easily adjusted by a single parameter, that is, the decomposition level. Our method is supported by a solid mathematical foundation, which has the potential to become a new core technology in image compression.

A Particle Swarm Optimization Based Deep Learning Model for Vehicle Classification

Image classification is a core field in the research area of image processing and computer vision in which vehicle classification is a critical domain. The purpose of vehicle categorization is to formulate a compact system to assist in real-world problems and applications such as security, traffic analysis, and self-driving and autonomous vehicles. The recent revolution in the field of machine learning and artificial intelligence has provided an immense amount of support for image processing related problems and has overtaken the conventional, and handcrafted means of solving image analysis problems. In this paper, a combination of pre-trained CNN GoogleNet and a nature-inspired problem optimization scheme, particle swarm optimization (PSO), was employed for autonomous vehicle classification. The model was trained on a vehicle image dataset obtained from Kaggle that has been suitably augmented. The trained model was classified using several classifiers; however, the Cubic SVM (CSVM) classifier was found to outperform the others in both time consumption and accuracy (94.8%). The results obtained from empirical evaluations and statistical tests reveal that the model itself has shown to outperform the other related models not only in terms of accuracy (94.8%) but also in terms of training time (82.7 s) and speed prediction (380 obs/sec).

Wavelet thresholding techniques implementation in retinal images for suppressing noises

In medical image processing, noise plays a major role of reducing the details of the images and blurs the features which are important for the diagnosis of the disease. Retinal images are high resolution images. Gaussian noise, Salt & Pepper noise affects certain important features of them which are needed for proper diagnosis of the retinal diseases. So removing noises in these images is a kind of difficulty that all researcher experiences. The main objective is to reduce the effect of Gaussian noise and Salt & Pepper noise in retinal images. Recently, wavelet transform is being used in all areas of image processing and especially in medical image denoising, it is highly used because it has multi resolution property and sparsity. So here, wavelet thresholding techniques such as Visu shrink, SURE Shrink and Bayes shrink are implemented to remove Gaussian noise and Salt & Pepper noise. The performance of all the three methods are evaluated using PSNR (Peak Signal to Noise Ratio), MSE (Mean Square Error), NK (Normalized cross correlation) and Time elapsed. Also, the performances of all the thresholding techniques were compared using above mentioned metrics.

A multi-scale unsupervised learning for deformable image registration.

Image registration is a fundamental task in the area of image processing, and it is critical to many clinical applications, e.g., computer-assisted surgery. In this work, we attempt to design an effective framework that gains higher accuracy at a minimal cost of the invertibility of registration field. A hierarchically aggregated transformation (HAT) module is proposed. Within each HAT module, we connect multiple convolutions in a hierarchical manner to capture the multi-scale context, enabling small and large displacements between a pair of images to be taken into account simultaneously during the registration process. Besides, an adaptive feature scaling (AFS) mechanism is presented to refine the multi-scale feature maps derived from the HAT module by rescaling channel-wise features in the global receptive field. Based on the HAT module and AFS mechanism, we establish an efficacious and efficient unsupervised deformable registration framework. The devised framework is validated on the dataset of SCARED and MICCAI Instrument Segmentation and Tracking Challenge 2015, and the experimental results demonstrate that our method achieves better registration accuracy with fewer number of folding pixels than three widely used baseline approaches of SyN, NiftyReg and VoxelMorph. We develop a novel method for unsupervised deformable image registration by incorporating the HAT module and AFS mechanism into the framework, which provides a new way to obtain a desirable registration field between a pair of images.

Real-world single image super-resolution: A brief review

Single image super-resolution (SISR), which aims to reconstruct a high-resolution (HR) image from a low-resolution (LR) observation, has been an active research topic in the area of image processing in recent decades. Particularly, deep learning-based super-resolution (SR) approaches have drawn much attention and have greatly improved the reconstruction performance on synthetic data. However, recent studies show that simulation results on synthetic data usually overestimate the capacity to super-resolve real-world images. In this context, more and more researchers devote themselves to develop SR approaches for realistic images. This article aims to make a comprehensive review on real-world single image super-resolution (RSISR). More specifically, this review covers the critical publicly available datasets and assessment metrics for RSISR, and four major categories of RSISR methods, namely the degradation modeling-based RSISR, image pairs-based RSISR, domain translation-based RSISR, and self-learning-based RSISR. Comparisons are also made among representative RSISR methods on benchmark datasets, in terms of both reconstruction quality and computational efficiency. Besides, we discuss challenges and promising research topics on RSISR.

CNN Variants for Computer Vision: History, Architecture, Application, Challenges and Future Scope

Computer vision is becoming an increasingly trendy word in the area of image processing. With the emergence of computer vision applications, there is a significant demand to recognize objects automatically. Deep CNN (convolution neural network) has benefited the computer vision community by producing excellent results in video processing, object recognition, picture classification and segmentation, natural language processing, speech recognition, and many other fields. Furthermore, the introduction of large amounts of data and readily available hardware has opened new avenues for CNN study. Several inspirational concepts for the progress of CNN have been investigated, including alternative activation functions, regularization, parameter optimization, and architectural advances. Furthermore, achieving innovations in architecture results in a tremendous enhancement in the capacity of the deep CNN. Significant emphasis has been given to leveraging channel and spatial information, with a depth of architecture and information processing via multi-path. This survey paper focuses mainly on the primary taxonomy and newly released deep CNN architectures, and it divides numerous recent developments in CNN architectures into eight groups. Spatial exploitation, multi-path, depth, breadth, dimension, channel boosting, feature-map exploitation, and attention-based CNN are the eight categories. The main contribution of this manuscript is in comparing various architectural evolutions in CNN by its architectural change, strengths, and weaknesses. Besides, it also includes an explanation of the CNN’s components, the strengths and weaknesses of various CNN variants, research gap or open challenges, CNN applications, and the future research direction.

Target Recognition and Trajectory Planning of Apple Harvesting Robot considering Color Multimedia Image Segmentation Algorithm

For the purpose of significantly reducing the processing time of the apple harvesting robot during the harvesting process, it is highly necessary to carry out the corresponding studies on the methods for rapid recognition and trajectory planning. Through the comprehensive application of information relevance, the image processing area can be reduced. For image recognition and trajectory planning, the related template matching algorithm for removing the mean value and normalization product can be adopted, and segmentation methods based on different threshold values can be used for the realization of the effect. Subsequently, the comparative experiments are properly carried out to verify the effectiveness of the method used.

Development of data driven adaptive edge detectors for image processing

In most vision processing activities, the early stage involves identifying the features in an image that provide cues to structure and properties of the object in the scene. Most common features in an image or in a scene arel edges. Edges arel significant local changes in intensity within anlimage. Most important goal of edge detection is to produce a line drawing from anlimage representing the scene. The significant features of an image such as line, curve and corners can be extracted from edges. During the stage of discovering and exploring the information from an image of that scene, edge detection is the most important and early-stage activity and as such it is prominent active area in image processing. Most popular edge detection algorithm such as Robert, Sobel, Canny, Prewitt and Laplacian of Gaussian (LoG), etc. are currently in use. This paper emphasis on an experimental study of limitations of conventional edge detectors and to devise a novel approach to resolve the conflicting issues i.e., limitations of these edge detectors in adaptive space utilizing novel methods such as Bi-dimensional Empirical Mode Decomposition (BEMD), Image Empirical Mode Decomposition (IEMD), Complete Ensemble Empirical Mode Decomposition (CEEMD) and Multivariate Decomposition techniques. Further, to study the performance of these modified edge detectors on the images of complex scenes which are of societal and agricultural importance.

No-reference stereoscopic image quality assessment using quaternion wavelet transform and heterogeneous ensemble learning

As the demand for high-quality stereo images has grown in recent years, stereoscopic image quality assessment (SIQA) has become an important research area in modern image processing technology.In this paper, we propose a no-reference stereoscopic image quality assessment (NR-SIQA) model using heterogeneous ensemble learning ‘quality-aware’ features from luminance image, chrominance image, disparity and cyclopean images via quaternion wavelet transform (QWT). Firstly, luminance image and chrominance image are generated by CIELAB color space as monocular perception, and the novel disparity and cyclopean images are utilized to complement with monocular information. Then, a number of ‘quality-aware’ features in the quaternion wavelet domain are discovered, including entropy, texture features, energy features, energy differences features and MSCN coefficients of high frequency sub-band. Finally, a heterogeneous ensemble model via support vector regression (SVR) & extreme learning machine (ELM) & random forest (RF) is proposed to predict quality score, and bootstrap sampling and rotated feature space are used to increase the diversity of data distribution. Comparing with the state-of-the-art NR-SIQA models, experimental results on four public databases prove the accuracy and robustness of the proposed model.

Speckle filtering of ultrasonic images using weighted nuclear norm minimization in wavelet domain

Medical images such as ultrasound images suffer from multiplicative speckle noise which reduces the contrast of ultrasound images and adversely affects the diagnosis process. Discrete Wavelet transform (DWT) is widely accepted in the area of image processing due to its time localization, multi-resolution and sparseness properties. DWT decomposes the noisy image into approximation coefficients and detail coefficients. The detail wavelet coefficients of clean image have sparsity property, whereas in the case of noisy image, sparsity of detail coefficients is reduced due to the presence of speckle noise. Thus, in the proposed method, low rank based Weighted Nuclear Norm Minimization (WNNM) is applied on detail coefficients to reveal the sparsity property of DWT. WNNM is applied on the group matrix of non-local similar patches of detail subbands of DWT to approximate the low-rank denoised version of the subbands. Moreover, less amount of edge and structure information is present in the approximation coefficients of DWT. Thus, in the proposed method, Non-local Means (NLM) filter with Square-Chord distance is also used to denoise speckle noise from approximation coefficients of DWT. Exhaustive experiments are conducted on various images such as real US images, simulated kidney image and synthetic image. It is observed that the mean improvement in terms of PSNR and CNR values of the proposed hybrid method are 1.50% and 3.81% respectively over WNNM based Despeckling using Low-Rank Approximation (DLRA) method. Qualitative and quantitative analyses show that the proposed hybrid method performed better than existing speckle reduction methods in terms of both speckle reduction and edge preservation.

PathCNN: interpretable convolutional neural networks for survival prediction and pathway analysis applied to glioblastoma

MotivationConvolutional neural networks (CNNs) have achieved great success in the areas of image processing and computer vision, handling grid-structured inputs and efficiently capturing local dependencies through multiple levels of abstraction. However, a lack of interpretability remains a key barrier to the adoption of deep neural networks, particularly in predictive modeling of disease outcomes. Moreover, because biological array data are generally represented in a non-grid structured format, CNNs cannot be applied directly.ResultsTo address these issues, we propose a novel method, called PathCNN, that constructs an interpretable CNN model on integrated multi-omics data using a newly defined pathway image. PathCNN showed promising predictive performance in differentiating between long-term survival (LTS) and non-LTS when applied to glioblastoma multiforme (GBM). The adoption of a visualization tool coupled with statistical analysis enabled the identification of plausible pathways associated with survival in GBM. In summary, PathCNN demonstrates that CNNs can be effectively applied to multi-omics data in an interpretable manner, resulting in promising predictive power while identifying key biological correlates of disease.Availability and implementationThe source code is freely available at: https://github.com/mskspi/PathCNN.

Scope and Key Areas of Medical Image Processing

Over the past century, scientific advances in diagnostic devices have offered new potential for noninvasive diagnoses and entrenched computed tomography as a critical component of today's health services. The multidisciplinary field of health image analysis is one of the key areas of innovation that represents these achievements. This area of rapid growth deals with a wide range of operations that support the whole data flow in current health monitoring systems (from raw data capture through digital image transfer). These technologies now have better spatial and luminance resolutions, as well as quicker collection periods, resulting in a large volume of high critical image files that must be appropriately processed and evaluated in order to provide reliable diagnostics findings. This article examines the core kinds of clinical image analysis, as well as the background of various imaging technologies and the major difficulties and developments in the field.

An effective fuzzy logic and particle swarm optimization based thermal and visible-light image fusion framework using curve-let transform

Thermal and visible-light image fusion is an active and one of the most significant research area in image processing. The result produced through this fusion process can be very much useful in surveillance and military applications especially for improving visibility and situation awareness. This paper proposes an effective fuzzy logic and particle swarm optimization based thermal and visible-light image fusion framework using curve-let transform. A novel fuzzy logic based contrast enhancement method is introduced in this framework for improving the visual appearance of both thermal and visible-light source images. In this framework curve-let transform is used for extracting more multi-scale features from source images. Upon applying curve-let transform on enhanced source images will produce low frequency (approximation) and high frequency (detailed) coefficients. Particle swarm optimization based fusion strategy has been used for producing more featured fused approximation coefficients. Correspondingly Max fusion strategy has been used for generating more textures based fused detailed coefficients. Finally upon applying inverse curve-let transform generates final fused image. The obtained simulation results clearly demonstrate that our framework has shown superior results compared to other fusion methods both in terms of subjective and objective assessments.

A Portable Smartphone-Based Laryngoscope System for High-Speed Vocal Cord Imaging of Patients With Throat Disorders: Instrument Validation Study.

BackgroundCurrently, high-speed digital imaging (HSDI), especially endoscopic HSDI, is routinely used for the diagnosis of vocal cord disorders. However, endoscopic HSDI devices are usually large and costly, which limits access to patients in underdeveloped countries and in regions with inadequate medical infrastructure. Modern smartphones have sufficient functionality to process the complex calculations that are required for processing high-resolution images and videos with a high frame rate. Recently, several attempts have been made to integrate medical endoscopes with smartphones to make them more accessible to people in underdeveloped countries.ObjectiveThis study aims to develop a smartphone adaptor for endoscopes, which enables smartphone-based vocal cord imaging, to demonstrate the feasibility of performing high-speed vocal cord imaging via the high-speed imaging functions of a high-performance smartphone camera, and to determine the acceptability of the smartphone-based high-speed vocal cord imaging system for clinical applications in developing countries.MethodsA customized smartphone adaptor optical relay was designed for clinical endoscopy using selective laser melting–based 3D printing. A standard laryngoscope was attached to the smartphone adaptor to acquire high-speed vocal cord endoscopic images. Only existing basic functions of the smartphone camera were used for HSDI of the vocal cords. Extracted still frames were observed for qualitative glottal volume and shape. For image processing, segmented glottal and vocal cord areas were calculated from whole HSDI frames to characterize the amplitude of the vibrations on each side of the glottis, including the frequency, edge length, glottal areas, base cord, and lateral phase differences over the acquisition time. The device was incorporated into a preclinical videokymography diagnosis routine to compare functionality.ResultsSmartphone-based HSDI with the smartphone-endoscope adaptor could achieve 940 frames per second and a resolution of 1280 by 720 frames, which corresponds to the detection of 3 to 8 frames per vocal cycle at double the spatial resolution of existing devices. The device was used to image the vocal cords of 4 volunteers: 1 healthy individual and 3 patients with vocal cord paralysis, chronic laryngitis, or vocal cord polyps. The resultant image stacks were sufficient for most diagnostic purposes. The cost of the device including the smartphone was lower than that of existing HSDI devices. The image processing and analytics demonstrated the successful calculation of relevant diagnostic variables from the acquired images. Patients with vocal pathologies were easily differentiable in the quantitative data.ConclusionsA smartphone-based HSDI endoscope system can function as a point-of-care clinical diagnostic device. The resulting analysis is of higher quality than that accessible by videostroboscopy and promises comparable quality and greater accessibility than HSDI. In particular, this system is suitable for use as an accessible diagnostic tool in underdeveloped areas with inadequate medical service infrastructure.

A Nonlinear Anisotropic Diffusion Equation for Image Restoration with Forward-backward Diffusivities

Background: In the image processing area, deblurring and denoising are the most challenging hurdles. The deblurring image by a spatially invariant kernel is a frequent problem in the field of image processing. Methods: For deblurring and denoising, the total variation (TV norm) and nonlinear anisotropic diffusion models are powerful tools. In this paper, nonlinear anisotropic diffusion models for image denoising and deblurring are proposed. The models are developed in the following manner: first multiplying the magnitude of the gradient in the anisotropic diffusion model, and then apply priori smoothness on the solution image by Gaussian smoothing kernel. Results: The finite difference method is used to discretize anisotropic diffusion models with forward- backward diffusivities. Conclusion: The results of the proposed model are given in terms of the improvement.

Handwritten Character Recognition using Deep Learning and Neural Network

Handwritten character recognition is among the most challenging research areas in pattern recognition and image processing. With everything going digital, applications of handwritten character recognition are emerging in different offices, educational institutes, healthcare units, commercial units and banks etc., where the documents that are handwritten are dealt more frequently. Many researchers have worked with recognition of characters of different languages but there is comparatively less work carried for Devanagari Script. In past few years, however the work carried out in this direction is increasing to a great extent. Handwritten Devanagari Character Recognition is more challenging in comparison to the recognition of the Roman characters. The complexity is mostly due to the presence of a header line known as shirorekha that connects the Devanagari characters to form a word. The presence of this header line makes the segmentation process of characters more difficult. There is uniqueness to the handwriting styles of every individual which adds to the complexity. In this paper, a recognition system based on neural network has been proposed for Devanagari (Marathi) alphabets. Each of the characters that are extracted through query image is resized and is then passed to the neural networks for the process of recognition.

A Study of Novel Optical Character Recognition Algorithms

Optical Character Recognition has been an inseparable part of human life during everyday transactions. The OCR has extended its application areas in almost all fields viz. healthcare, finance, banking, entertainment, trading system, digital storage, and so on. In the recent past, handwriting recognition is one of the hardest study areas in the area of image processing. In this paper, the various techniques for converting textual content from number plates, printed, handwritten paper documents into machine code have been discussed. The transforming method used in all these techniques is known as OCR. The English OCR system is necessary for the conversion of various published books and other documents in English into human editable computer text files. The latest researches in this area have included methodologies that identify different fonts and styles of English handwritten scripts. As of date, even though a number of algorithms are available, it has its own pros and cons. Since the recognition of different styles and fonts in machine-printed and handwritten English script is the biggest challenge, this field is open for researchers to implement new algorithms that would overcome the deficiencies of its predecessors.