Simple Image Processing Research Articles

High-resolution outdoor videography of insects using Fast Lock-On tracking.

Insects have important roles globally in ecology, economy, and health, yet our understanding of their behavior remains limited. Bees, for example, use vision and a tiny brain to find flowers and return home, but understanding how they perform these impressive tasks has been hampered by limitations in recording technology. Here, we present Fast Lock-On (FLO) tracking. This method moves an image sensor to remain focused on a retroreflective marker affixed to an insect. Using paraxial infrared illumination, simple image processing can localize the sensor location of the insect in a few milliseconds. When coupled with a feedback system to steer a high-magnification optical system to remain focused on the insect, a high-spatiotemporal resolution trajectory can be gathered over a large region. As the basis for several robotic systems, we show that FLO is a versatile idea that can be used in combination with other components. We demonstrate that the optical path can be split and used for recording high-speed video. Furthermore, by flying an FLO system on a quadcopter drone, we track a flying honey bee and anticipate tracking insects in the wild over kilometer scales. Such systems have the capability to provide higher-resolution information about insects behaving in natural environments and as such will be helpful in revealing the biomechanical and neuroethological mechanisms used by insects in natural settings.

Vertical back movement of cows during locomotion: detecting lameness with a simple image processing technique.

This research paper proposes a simple image processing technique for automatic lameness detection in dairy cows under farm conditions. Seventy-five cows were selected from a dairy farm and visually assessed for a reference/real lameness score (RLS) as they left the milking parlor, while simultaneously being video-captured. The method employed a designated walking path and video recordings processed through image analysis to derive a new computerized automatic lameness score (ALDS) based on calculated factors from back arch posture. The proposed automatic lameness detection system was calibrated using 12 cows, and the remaining 63 were used to evaluate the diagnostic characteristics of the ALDS. The agreement and correlation between ALDS and RLS were investigated. ALDS demonstrated high diagnostic accuracy with 100% sensitivity and specificity and was found to be 100% accurate with a perfect agreement (ρc = 1) and strong correlation (r = 1, P < 0.001) for lameness detection in binary scores (lame/non-lame). Moreover, the ALDS had a strong agreement (ρc = 0.885) and was highly correlated (r = 0.840; 0.796-1.000 95% confidence interval, P < 0.001) with RLS in ordinal scores (lameness severity; LS1 to LS5). Our findings suggest that the proposed method has the potential to compete with vision-based lameness detection methods in dairy cows in farm conditions.

Instance segmentation of cells and nuclei from multi-organ cross-protocol microscopic images.

Light microscopy is a widely used technique in cell biology due to its satisfactory resolution for cellular structure analysis, prevalent availability of fluorescent probes for staining, and compatibility for the dynamic analysis of live cells. However, the segmentation of cells and nuclei from microscopic images is not a straightforward process because it has several challenges such as high variation in morphology and shape, the presence of noise and diverse contrast in backgrounds, clustering or overlapping nature of cells. Dealing with these challenges and facilitating more reliable analysis necessitates the implementation of computer-aided methods that leverage image processing techniques and deep learning algorithms. The major goal of this study is to propose a model, for instance segmentation of cells and nuclei, applying the most cutting-edge deep learning techniques. A fine-tuned You Only Look at Once version 9 extended (YOLOv9-E) model is initially applied as a prompt generator to generate bounding box prompts. Using the generated prompts, a pre-trained segment anything model (SAM) is subsequently applied through zero-short inferencing to produce raw segmentation masks. These segmentation masks are then refined using non-max suppression and simple image processing methods such as image addition and morphological processing. The proposed method is developed and evaluated using an open-sourced dataset called Expert Visual Cell Annotation (EVICAN), which is relatively large and contains 4,738 microscopy images extracted from cross organs using different protocols. Based on the evaluation results on three different levels of EVICAN test sets, the proposed method demonstrates noticeable performances showing average mAP50 [mean average precision at intersection over union (IoU) =0.50] scores of 96.25, 95.05, and 94.18 for cell segmentation, and 68.04, 54.66, and 38.29 for nucleus segmentation on easy, medium, and difficult test sets, respectively. Our proposed method for instance segmentation of cells and nuclei provided favorable performance compared to the existing methods in literature, indicating its potential utility as an assistive tool for cell culture experts, facilitating prompt and reliable analysis.

Color Face Image Generation with Improved Generative Adversarial Networks

This paper focuses on the development of an improved Generative Adversarial Network (GAN) specifically designed for generating color portraits from sketches. The construction of the system involves using a GPU (Graphics Processing Unit) computing host as the primary unit for model training. The tasks that require high-performance calculations are handed over to the GPU host, while the user host only needs to perform simple image processing and use the model trained by the GPU host to generate images. This arrangement reduces the computer specification requirements for the user. This paper will conduct a comparative analysis of various types of generative networks which will serve as a reference point for the development of the proposed Generative Adversarial Network. The application part of the paper focuses on the practical implementation and utilization of the developed Generative Adversarial Network for the generation of multi-skin tone portraits. By constructing a face dataset specifically designed to incorporate information about ethnicity and skin color, this approach can overcome a limitation associated with traditional generation networks, which typically generate only a single skin color.

Minimum intensity projection of embossed quadrant-detection images for improved photoreceptor mosaic visualisation.

Non-confocal split-detection imaging reveals the cone photoreceptor inner segment mosaic in a plethora of retinal conditions, with the potential of providing insight to ageing, disease, and response to treatment processes, in vivo, and allows the screening of candidates for cell rescue therapies. This imaging modality complements confocal reflectance adaptive optics scanning light ophthalmoscopy, which relies on the waveguiding properties of cones, as well as their orientation toward the pupil. Split-detection contrast, however, is directional, with each cone inner segment appearing as opposite dark and bright semicircles, presenting a challenge for either manual or automated cell identification. Quadrant-detection imaging, an evolution of split detection, could be used to generate images without directional dependence. Here, we demonstrate how the embossed-filtered quadrant-detection images, originally proposed by Migacz etal. for visualising hyalocytes, can also be used to generate photoreceptor mosaic images with better and non-directional contrast for improved visualisation. As a surrogate of visualisation improvement between legacy split-detection images and the images resulting from the method described herein, we provide preliminary results of simple image processing routines that may enable the automated identification of generic image features, as opposed to complex algorithms developed specifically for photoreceptor identification, in pathological retinas.

Dual imaging technique for a real-time inspection system of foreign object detection in fresh-cut vegetables

Fresh-cut vegetables are a food product susceptible to contamination by foreign materials (FMs). To detect a range of potential FMs in fresh-cut vegetables, a dual imaging technique (fluorescence and color imaging) with a simple and effective image processing algorithm in a user-friendly software interface was developed for a real-time inspection system. The inspection system consisted of feeding and sensing units, including two cameras positioned in parallel, illuminations (white LED and UV light), and a conveyor unit. A camera equipped with a long-pass filter was used to collect fluorescence images. Another camera collected color images of fresh-cut vegetables and FMs. The feeding unit fed FMs mixed with fresh-cut vegetables onto a conveyor belt. Two cameras synchronized programmatically in the software interface simultaneously collected fluorescence and color image samples based on the region of interest as they moved through the conveyor belt. Using simple image processing algorithms, FMs could be detected and depicted in two different image windows. The results demonstrated that the dual imaging technique can effectively detect potential FMs in two types of fresh-cut vegetables (cabbage and green onion), as indicated by the combined fluorescence and color imaging accuracy. The test results showed that the real-time inspection system could detect FMs measuring 0.5 mm in fresh-cut vegetables. The results showed that the combined detection accuracy of FMs in the cabbage (95.77%) sample was superior to that of green onion samples (87.89%). Therefore, the inspection system was more effective at detecting FMs in cabbage samples than in green onion samples.

Development of simple image processing for in-situ TEM toward live processing

Development of simple image processing for in-situ TEM toward live processing

Combining Color and Spatial Image Features for Unsupervised Image Segmentation with Mixture Modelling and Spectral Clustering

The demand for accurate and reliable unsupervised image segmentation methods is high. Regardless of whether we are faced with a problem for which we do not have a usable training dataset, or whether it is not possible to obtain one, we still need to be able to extract the desired information from images. In such cases, we are usually gently pushed towards the best possible clustering method, as it is often more robust than simple traditional image processing methods. We investigate the usefulness of combining two clustering methods for unsupervised image segmentation. We use the mixture models to extract the color and spatial image features based on the obtained output segments. Then we construct a similarity matrix (adjacency matrix) based on these features to perform spectral clustering. In between, we propose a label noise correction using Markov random fields. We investigate the usefulness of our method on many hand-crafted images of different objects with different shapes, colorization, and noise. Compared to other clustering methods, our proposal performs better, with 10% higher accuracy. Compared to state-of-the-art supervised image segmentation methods based on deep convolutional neural networks, our proposal proves to be competitive.

High-Performance Thin-Layer Chromatography hyphenated with image processing and chemometrics as a tool for forensic discrimination of Cannabis sativa L. chemotypes

Cannabis sativa L. is the most widely cultivated, trafficked, and abused illicit drug. The reliable distinction between drug type and fiber type C. sativa remains a topic of interest for forensic chemists. For the first time, we applied a combination of simple, cost-effective, and reliable High-Performance Thin-Layer Chromatography (HPTLC) with image analysis as well as sophisticated multivariate tools for differentiating C. sativa drug, fiber and intermediate chemotypes. After extraction, major cannabinoids from 43 seized C. sativa were separated using the HPTLC method. Peak areas of Δ9-THC, CBN, and CBD were calculated by simple image processing, and Xfactor = [THC + CBN]/CBD was determined as criteria used to discriminate three C. sativa chemotypes. The obtained results were compared with Gas Chromatography with Flame Ionization Detection (GC-FID) as the reference method for the determination of Xfactor.Principal Component Analysis and Hierarchic Cluster Analysis were applied to classify three C. sativa chemotypes according to their chemical pattern. The proposed approach clearly distinguishes 26 drugs, 13 fiber, and 4 intermediate chemotypes in seized C. sativa, aligning with GC-FID analysis. The results showed that the HPTLC technique in combination with multivariate methods is an accurate and reliable tool for high-throughput and forensic screening of three C. sativa chemotypes. Compounds such as Δ9-THC and CBD were marked as the most important cannabinoids responsible for the classification of the seized C. sativa chemotypes.

The Methods of Determining Temporal Direction Based on Asymmetric Information of the Optic Disc for Optimal Fovea Detection

Accurate localization of the fovea in fundus images is essential for diagnosing retinal diseases. Existing methods often require extensive data and complex processes to achieve high accuracy, posing challenges for practical implementation. In this paper, we propose an effective and efficient approach for fovea detection using simple image processing operations and a geometric approach based on the optic disc’s position. A key contribution of this study is the successful determination of the temporal direction by leveraging readable asymmetries related to the optic disc and its surroundings. We discuss three methods based on asymmetry conditions, including blood vessel distribution, cup disc inclination, and optic disc location ratio, for detecting the temporal direction. This enables precise determination of the optimal foveal region of interest. Through this optimized fovea region, fovea detection is achieved using straightforward morphological and image processing operations. Extensive testing on popular datasets (DRIVE, DiaretDB1, and Messidor) demonstrates outstanding accuracy of 99.04% and a rapid execution time of 0.251 s per image. The utilization of asymmetrical conditions for temporal direction detection provides a significant advantage, offering high accuracy and efficiency while competing with existing methods.

Inside out: transforming images of lab-grown plants for machine learning applications in agriculture.

Machine learning tasks often require a significant amount of training data for the resultant network to perform suitably for a given problem in any domain. In agriculture, dataset sizes are further limited by phenotypical differences between two plants of the same genotype, often as a result of different growing conditions. Synthetically-augmented datasets have shown promise in improving existing models when real data is not available. In this paper, we employ a contrastive unpaired translation (CUT) generative adversarial network (GAN) and simple image processing techniques to translate indoor plant images to appear as field images. While we train our network to translate an image containing only a single plant, we show that our method is easily extendable to produce multiple-plant field images. Furthermore, we use our synthetic multi-plant images to train several YoloV5 nano object detection models to perform the task of plant detection and measure the accuracy of the model on real field data images. The inclusion of training data generated by the CUT-GAN leads to better plant detection performance compared to a network trained solely on real data.

Face, Fingerprint, and Signature based Multimodal Biometric System using Score Level and Decision Level Fusion Approaches

Principal Component Analysis (PCA) is the best face recognition method. This research suggests PCA for fingerprint and signature recognition. Simple image processing transforms like DCT, 2D-DCT, DWT, SWT, 2D-SWT, SVD (Singular Vector Decomposition), Entropy, and Rank can be used for feature extraction. These transforms and measures are utilized with PCA as a feature extraction module to construct uni-modal and multimodal biometric systems using face, fingerprint, and signature modalities. Most PCA biometrics systems compare stored template to claimed identification using Euclidean distance. This paper proposes matching modules using similarity and dissimilarity measures viz. Absolute Pearson's Correlation Coefficient (APCC), Absolute Uncentered Pearson's Correlation Coefficient (AUPCC), Bray Curtis Distance (BC), Canberra distance (CB), Chebyshev Distance (CBS), Chessboard Distance (CSB), City block or Manhattan distance (CTB), Cross Correlation (CC), Dot product (DP), Euclidean distance (EUC), Extended Jaccard Distance (EJ), Hamming Distance (HM), Harmonically Summed Euclidean distance (HSEUC), Kendall Correlation Coefficient (KCC), Mahalanobis Distance (MH), Minimum Coordinate Difference (MCD), Minkowiski distance (MNK), Multivariate Kurtosis Coefficient (MVK), Multivariate Skew (MVS), Normalized City Block or Manhattan distance (NCTB), Normalized Cross-correlation (NCC), Normalized Euclidean distance (NEUC), Pearson’s Cosine Distance (PCOS), Pearson's Correlation Coefficient (PCC), Pearson's Absolute Value Dissimilarity (PAVD), Pearson's Linear Dissimilarity (PLDISS), Spearman Correlation Coefficient (SCC), Standardized Euclidean Distance (SEUC), Uncentered Pearson's Correlation Coefficient (UPCC), Wave-Hedges Distance (WVH). This study again discusses score level fusion of face, fingerprint, and signature using sum and max rules, z-score normalization, and decision level fusion using AND rule.

Automated extraction of seed morphological traits from images

Abstract The description of biological objects, such as seeds, mainly relies on manual measurements of few characteristics, and on visual classification of structures, both of which can be subjective, error prone and time‐consuming. Image analysis tools offer means to address these shortcomings, but we currently lack a method capable of automatically handling seeds from different taxa with varying morphological attributes and obtaining interpretable results. Here, we provide a simple image acquisition and processing protocol and introduce Traitor, an open‐source software available as a command‐line interface (CLI), which automates the extraction of seed morphological traits from images. The workflow for trait extraction consists of scanning seeds against a high‐contrast background, correcting image colours, and analysing images with the software. Traitor is capable of processing hundreds of images of varied taxa simultaneously with just three commands, and without a need for training, manual fine‐tuning or thresholding. The software automatically detects each object in the image and extracts size measurements, traditional morphometric descriptors widely used by scientists and practitioners, standardised shape coordinates, and colorimetric measurements. The method was tested on a dataset comprising of 91,667 images of seeds from 1228 taxa. Traitor's extracted average length and width values closely matched the average manual measurements obtained from the same collection (concordance correlation coefficient of 0.98). Further, we used a large image dataset to demonstrate how Traitor's output can be used to obtain representative seed colours for taxa, determine the phylogenetic signal of seed colour, and build objective classification categories for shape with high levels of visual interpretability. Our approach increases productivity and allows for large‐scale analyses that would otherwise be unfeasible. Traitor enables the acquisition of data that are readily comparable across different taxa, opening new avenues to explore functional relevance of morphological traits and to advance on new tools for seed identification.

Machine learning and handcrafted image processing methods for classifying common weeds in corn field

Weed management practices strive to reduce weeds, which compete with crops for nutrients, sunlight, and water and are thus important for maintaining yield. In most weed management practices, the first step is to identify or classify weeds. However, efficient identification and classification of weeds are challenging using conventional manual methods such as field visits. Therefore, in this study, we propose to classify four common weeds in the corn field of North Dakota (common lambsquarters, common purslane, horseweed, and redroot pigweed), also applicable to other regions, using computer vision methods. Weeds were grown in plastic trays under natural conditions from the field soil, and images were collected using an RGB camera. For each weed species, 21 shape features were extracted through a developed ImageJ plugin. The handcrafted simple image processing approach was highly successful in distinguishing common lambsquarters and redroot pigweed from horseweed, while did not perform well with other species combinations and classifying them collectively owing to the similarity of the shapes. However, the three advanced non-parametric machine learning (ML) models, namely, k-nearest neighbor (kNN), random forest (RF), and support vector machine (SVM), resulted in high accuracies with RF outperforming the others. It is recommended that the handcrafted simple image processing algorithm should be tried first, due to its simplicity, for the identification and classification of weeds before resorting to advanced and complex versatile ML modeling approaches. Such tools and methodologies using open-source platforms will be easily accessible and helpful to farmers, producers, and other stakeholders related to crop production.

HemispheR: an R package for fisheye canopy image analysis

Hemispherical photography is a relevant tool to estimate canopy attributes such as leaf area index (LAI). Advancements in digital photography and image processing tools have supported long-lasting use of digital hemispherical photography (DHP). While some open-source tools exists for DHP, very few solutions have been made available in R programming packages, and none of these allows a full processing workflow to retrieve LAI and other canopy attributes from fisheye images.To fill this gap, we developed an R package (hemispheR) to support the whole processing of DHP images in an automated, fast, and reproducible way. The package functions, which are designed for step-by-step single-image analysis, can be performed sequentially in a pipeline, while allowing inspecting the quality of each image processing step. The package allows to analyze both circular and fullframe fisheye images, collected either with upward facing (forest canopies) or downward facing (short canopies and crops) camera orientation. In addition, the package allows to implement two consolidated LAI methods (LAI-2000/2200 and 57° method).A case study is presented to demonstrate the reliability of canopy attributes derived from hemispheR in temperate deciduous forests with variable canopy density and structure. Canopy attributes were validated against either results obtained from a reference proprietary software, either by benchmarking plot-level LAI with measurements obtained from littertraps. Results indicated hemispheR provide reliable openness and leaf area index in forest canopies as compared with reference values. We also found that combining hemispheR with other R packages further advance analysis of hemispherical canopy images, by reducing the sensitivity of results to camera exposure in both raw and non-raw canopy imagery.By providing a simple, transparent, and flexible image processing procedure, hemispheR supported the use of DHP for routine measurements and monitoring of forest canopy attributes. Hosting the package in a Git repository can further support development of the package, through either collaborative coding or forking projects.

3D Whole-body skin imaging for automated melanoma detection.

Existing artificial intelligence for melanoma detection has relied on analysing images of lesions of clinical interest, which may lead to missed melanomas. Tools analysing the entire skin surface are lacking. To determine if melanoma can be distinguished from other skin lesions using data from automated analysis of 3D-images. Single-centre, retrospective, observational convenience sample of patients diagnosed with melanoma at a tertiary care cancer hospital. Eligible participants were those with a whole-body 3D-image captured within 90 days prior to the diagnostic skin biopsy. 3D-images were obtained as standard of care using VECTRA WB360 Whole Body 3-dimensional Imaging System (Canfield Scientific). Automated data from image processing (i.e. lesion size, colour, border) for all eligible participants were exported from VECTRA DermaGraphix research software for analysis. The main outcome was the area under the receiver operating characteristic curve (AUC). A total of 35 patients contributed 23,538 automatically identified skin lesions >2 mm in largest diameter (102-3021 lesions per participant). All were White patients and 23 (66%) were males. The median (range) age was 64 years (26-89). There were 49 lesions of melanoma and 22,489 lesions that were not melanoma. The AUC for the prediction model was 0.94 (95% CI: 0.92-0.96). Considering all lesions in a patient-level analysis, 14 (28%) melanoma lesions had the highest predicted score or were in the 99th percentile among all lesions for an individual patient. In this proof-of-concept pilot study, we demonstrated that automated analysis of whole-body 3D-images using simple image processing techniques can discriminate melanoma from other skin lesions with high accuracy. Further studies with larger, higher quality, and more representative 3D-imaging datasets would be needed to improve and validate these results.

ThermoSecure: Investigating the Effectiveness of AI-Driven Thermal Attacks on Commonly Used Computer Keyboards

Thermal cameras can reveal heat traces on user interfaces, such as keyboards. This can be exploited maliciously to infer sensitive input, such as passwords. While previous work considered thermal attacks that rely on visual inspection of simple image processing techniques, we show that attackers can perform more effective artificial intelligence (AI)–driven attacks. We demonstrate this by presenting the development of ThermoSecure and its evaluation in two user studies (N = 21, N = 16), which reveal novel insights about thermal attacks. We detail the implementation of ThermoSecure and make a dataset of 1,500 thermal images of keyboards with heat traces resulting from input publicly available. Our first study shows that ThermoSecure successfully attacks 6-symbol, 8-symbol, 12-symbol, and 16-symbol passwords with an average accuracy of 92%, 80%, 71%, and 55% respectively, and even higher accuracy when thermal images are taken within 30 seconds. We found that typing behavior significantly impacts vulnerability to thermal attacks: hunt-and-peck typists are more vulnerable than fast typists (92% vs. 83% thermal attack success. respectively, if performed within 30 seconds). The second study showed that keycap material has a statistically significant effect on the effectiveness of thermal attacks: ABS keycaps retain the thermal trace of user presses for a longer period of time, making them more vulnerable to thermal attacks, with a 52% average attack accuracy compared with 14% for keyboards with PBT keycaps. Finally, we discuss how systems can leverage our results to protect from thermal attacks and present 7 mitigation approaches that are based on our results and previous work.

An automatic image processing based on Hough transform algorithm for pavement crack detection and classification

PurposeIncipient detection of pavement deterioration (such as crack identification) is critical to optimizing road maintenance because it enables preventative steps to be implemented to mitigate damage and possible failure. Traditional visual inspection has been largely superseded by semi-automatic/automatic procedures given significant advancements in image processing. Therefore, there is a need to develop automated tools to detect and classify cracks.Design/methodology/approachThe literature review is employed to evaluate existing attempts to use Hough transform algorithm and highlight issues that should be improved. Then, developing a simple low-cost crack detection method based on the Hough transform algorithm for pavement crack detection and classification.FindingsAnalysis results reveal that model accuracy reaches 92.14% for vertical cracks, 93.03% for diagonal cracks and 95.61% for horizontal cracks. The time lapse for detecting the crack type for one image is circa 0.98 s for vertical cracks, 0.79 s for horizontal cracks and 0.83 s for diagonal cracks. Ensuing discourse serves to illustrate the inherent potential of a simple low-cost image processing method in automated pavement crack detection. Moreover, this method provides direct guidance for long-term pavement optimal maintenance decisions.Research limitations/implicationsThe outcome of this research can help highway agencies to detect and classify cracks accurately for a very long highway without a need for manual inspection, which can significantly minimize cost.Originality/valueHough transform algorithm was tested in terms of detect and classify a large dataset of highway images, and the accuracy reaches 92.14%, which can be considered as a very accurate percentage regarding automated cracks and distresses classification.

A simple image processing pipeline to sharpen topology maps in multi-wavelength interference microscopy.

Multi-wavelength standing wave (SW) microscopy and interference reflection microscopy (IRM) are powerful techniques that use optical interference to study topographical structure. However, the use of more than two wavelengths to image the complex cell surface results in complicated topographical maps, and it can be difficult to resolve the three-dimensional contours. We present a simple image processing method to reduce the thickness and spacing of antinodal fringes in multi-wavelength interference microscopy by up to a factor of two to produce clearer and more precise topographical maps of cellular structures. We first demonstrate this improvement using model non-biological specimens, and we subsequently demonstrate the benefit of our method for reducing the ambiguity of surface topography and revealing obscured features in live and fixed-cell specimens.

Study on Automatic Electric Vehicle Charging Socket Detection Using ZED 2i Depth Sensor

This article introduces the utilization of the ZED 2i depth sensor in a robot-based automatic electric vehicle charging application. The employment of a stereo depth sensor is a significant aspect in robotic applications, since it is both the initial and the fundamental step in a series of robotic operations, where the intent is to detect and extract the charging socket on the vehicle’s body surface. The ZED 2i depth sensor was utilized for scene recording with artificial illumination. Later, the socket detection and extraction were accomplished using both simple image processing and morphological operations in an object extraction algorithm with tilt angles and centroid coordinates determination of the charging socket itself. The aim was to use well-known, simple, and proven image processing techniques in the proposed method to ensure both reliable and smooth functioning of the robot’s vision system in an industrial environment. The experiments demonstrated that the deployed algorithm both extracts the charging socket and determines the slope angles and socket coordinates successfully under various depth assessment conditions, with a detection rate of 94%.