Traditional Image Processing Research Articles

Weed–Crop Segmentation in Drone Images with a Novel Encoder–Decoder Framework Enhanced via Attention Modules

The rapid expansion of the world’s population has resulted in an increased demand for agricultural products which necessitates the need to improve crop yields. To enhance crop yields, it is imperative to control weeds. Traditionally, weed control predominantly relied on the use of herbicides; however, the indiscriminate application of herbicides presents potential hazards to both crop health and productivity. Fortunately, the advent of cutting-edge technologies such as unmanned vehicle technology (UAVs) and computer vision has provided automated and efficient solutions for weed control. These approaches leverage drone images to detect and identify weeds with a certain level of accuracy. Nevertheless, the identification of weeds in drone images poses significant challenges attributed to factors like occlusion, variations in color and texture, and disparities in scale. The utilization of traditional image processing techniques and deep learning approaches, which are commonly employed in existing methods, presents difficulties in extracting features and addressing scale variations. In order to address these challenges, an innovative deep learning framework is introduced which is designed to classify every pixel in a drone image into categories such as weed, crop, and others. In general, our proposed network adopts an encoder–decoder structure. The encoder component of the network effectively combines the Dense-inception network with the Atrous spatial pyramid pooling module, enabling the extraction of multi-scale features and capturing local and global contextual information seamlessly. The decoder component of the network incorporates deconvolution layers and attention units, namely, channel and spatial attention units (CnSAUs), which contribute to the restoration of spatial information and enhance the precise localization of weeds and crops in the images. The performance of the proposed framework is assessed using a publicly available benchmark dataset known for its complexity. The effectiveness of the proposed framework is demonstrated via comprehensive experiments, showcasing its superiority by achieving a 0.81 mean Intersection over Union (mIoU) on the challenging dataset.

Methods of video quality-improving

Video content has become integral to our daily lives, but poor video quality can significantly reduce viewers' experience and engagement. Various super-resolution methods are used to correct this, thereby reconstructing high-resolution videos from low-resolution ones. Two main categories of super-resolution methods exist traditional image processing and deep learning-based techniques. Deep learning-based techniques, such as Convolutional Neural Networks (CNNs), Generative Adversarial Networks (GANs), and Recurrent Neural Networks (RNNs), have shown great promise in enhancing video quality. The article discusses multiple adaptations of contemporary deep learning models to enhance video resolution. It also briefly explains the framework's design and implementation aspects. Lastly, the paper presents an overview and comparative analysis of the VSR techniques' efficiency on various benchmark datasets. At the same time, the paper describes potential challenges when choosing training sets; performance metrics, which can be used to compare different algorithms quantitatively. This work does not describe absolutely all existing VSR methods, but it is expected to contribute to the development of recent research in this field and potentially deepen our understanding of deep learning-based VSR methods, as well as stimulate further research in this area. In this work, new solutions for improving the performance of the methods are proposed, in particular, new quality metrics and datasets for model training. Overall, AI-based methods for VSR are becoming increasingly crucial with the rising demand for high-quality video content

Combining Color and Spatial Image Features for Unsupervised Image Segmentation with Mixture Modelling and Spectral Clustering

The demand for accurate and reliable unsupervised image segmentation methods is high. Regardless of whether we are faced with a problem for which we do not have a usable training dataset, or whether it is not possible to obtain one, we still need to be able to extract the desired information from images. In such cases, we are usually gently pushed towards the best possible clustering method, as it is often more robust than simple traditional image processing methods. We investigate the usefulness of combining two clustering methods for unsupervised image segmentation. We use the mixture models to extract the color and spatial image features based on the obtained output segments. Then we construct a similarity matrix (adjacency matrix) based on these features to perform spectral clustering. In between, we propose a label noise correction using Markov random fields. We investigate the usefulness of our method on many hand-crafted images of different objects with different shapes, colorization, and noise. Compared to other clustering methods, our proposal performs better, with 10% higher accuracy. Compared to state-of-the-art supervised image segmentation methods based on deep convolutional neural networks, our proposal proves to be competitive.

Balanced multi-scale target score network for ceramic tile surface defect detection

Ceramic tiles, as a prevalent building material, exhibit a wide variety of types and high demand. Traditional manual inspection methods relying on human visual observation suffer from low efficiency and unreliable accuracy. Current automated detection methods mostly rely on traditional image processing techniques for feature extraction, followed by machine learning-based classification. However, faced with the diversity of tile types and defect categories, fine-tuning and deployment processes require significant human and material resources, while detection efficiency remains limited. In this study, we first construct a high-resolution dataset for studying surface defects in ceramic tiles (CT surface defects dataset), encompassing multiple batches and various patterns of tiles. Subsequently, data analysis is conducted to address the scale and quantity differences in defect distribution. We propose an improved approach by introducing a content-aware feature recombination method and a dynamic attention mechanism to enhance the classical single-stage object detection algorithm YOLOv5. These enhancements aim to reduce information loss in features and enhance the expression of multi-scale features. Furthermore, we design a loss function that mitigates score differences for multi-scale defects. The proposed approach mitigates the discrepancy in contribution among different scale targets caused by imbalanced quantities. It effectively prevents the model from excessively favoring a specific scale target during the learning process. Experimental results demonstrate the superior accuracy and efficiency of our detection method.Compared to the baseline network YOLOv5, our approach achieved improvements of 4.9% in AP (Average Precision), 6% in APs (small-scale objects), and 8% in APl (large-scale objects). Furthermore, we achieved a 3.9% improvement in detecting white point defects, which are most affected by small-scale objects, and a 4.1% improvement in detecting discolored spot defect, which are most affected by class imbalance.

A Comprehensive Review on Multi-Class Recognition of Soybean Leaf Diseases

This paper presents a comprehensive review of the current state-of-the-art methodologies in the multi-class recognition of soybean leaf diseases, addressing the challenges faced by soybean cultivation globally. Focusing on diseases like rust, bacterial blight, anthracnose, and powdery mildew, the review encompasses traditional image processing techniques as well as modern advancements in deep learning, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Topics covered include dataset compilation, preprocessing, feature extraction, and the application of various machine learning algorithms. Special emphasis is placed on exploring the potential of transfer learning, domain adaptation, and the integration of spectral imaging and remote sensing technologies for enhanced disease detection. By providing a thorough comparative analysis, this review aims to guide future research efforts, aiding researchers, agronomists, and practitioners in developing robust and scalable solutions to combat soybean leaf diseases and improve global agricultural productivity.

Trends and hotspots in research on medical images with deep learning: a bibliometric analysis from 2013 to 2023.

With the rapid development of the internet, the improvement of computer capabilities, and the continuous advancement of algorithms, deep learning has developed rapidly in recent years and has been widely applied in many fields. Previous studies have shown that deep learning has an excellent performance in image processing, and deep learning-based medical image processing may help solve the difficulties faced by traditional medical image processing. This technology has attracted the attention of many scholars in the fields of computer science and medicine. This study mainly summarizes the knowledge structure of deep learning-based medical image processing research through bibliometric analysis and explores the research hotspots and possible development trends in this field. Retrieve the Web of Science Core Collection database using the search terms "deep learning," "medical image processing," and their synonyms. Use CiteSpace for visual analysis of authors, institutions, countries, keywords, co-cited references, co-cited authors, and co-cited journals. The analysis was conducted on 562 highly cited papers retrieved from the database. The trend chart of the annual publication volume shows an upward trend. Pheng-Ann Heng, Hao Chen, and Klaus Hermann Maier-Hein are among the active authors in this field. Chinese Academy of Sciences has the highest number of publications, while the institution with the highest centrality is Stanford University. The United States has the highest number of publications, followed by China. The most frequent keyword is "Deep Learning," and the highest centrality keyword is "Algorithm." The most cited author is Kaiming He, and the author with the highest centrality is Yoshua Bengio. The application of deep learning in medical image processing is becoming increasingly common, and there are many active authors, institutions, and countries in this field. Current research in medical image processing mainly focuses on deep learning, convolutional neural networks, classification, diagnosis, segmentation, image, algorithm, and artificial intelligence. The research focus and trends are gradually shifting toward more complex and systematic directions, and deep learning technology will continue to play an important role.

2D Shape Detection for Solving Geometry Word Problems

ABSTRACT The increasing scale of education has decreased the teacher–student ratio, making it difficult to keep track of student's progress individually. Therefore, teachers are leveraging artificial intelligence tools such as smart tutoring digital tools like learning management systems, and automated solvers for various subjects. Mathematics is one of the most fundamental subjects in school education. This work deals with facilitating the automatic solving of geometry math word problems (MWPs) at the elementary school level. To solve such problems, the information from the text as well as the accompanying figure needs to be interpreted. The preliminary task in solving such problems is to first identify which geometric shape is given in the question. Most existing systems or geometry solvers give the shape as one of the inputs to their systems to solve it further. In this work, the focus is on detecting the shape automatedly and proposing a technique for the same using traditional image processing methods. In addition, further, the triangles and quadrilaterals are classified into fine-grained classifications by taking into account their geometrical properties. Performance metrics like precision, recall, accuracy, and F1-score are used to present the results of this approach. The testing is done on various datasets that are readily available. The results show that on Geometry3K, GeoQA, and GeoS datasets the proposed technique attains an accuracy of 77.8%, 93.38%, and 95.02% and an F1 score of 60%, 71%, and 91.97% respectively. Like all other techniques, our system also has a limitation in that it does not detect all inscribed shapes.

Multi-Modal Medical Image Matching Based on Multi-Task Learning and Semantic-Enhanced Cross-Modal Retrieval

With the continuous advancement of medical imaging technology, a vast amount of multi-modal medical image data has been extensively utilized for disease diagnosis, treatment, and research. Effective management and utilization of these data becomes a pivotal challenge, particularly when undertaking image matching and retrieval. Although numerous methods for medical image matching and retrieval exist, they primarily rely on traditional image processing techniques, often limited to manual feature extraction and singular modality handling. To address these limitations, this study introduces an algorithm for medical image matching grounded in multi-task learning, further investigating a semantic-enhanced technique for cross-modal medical image retrieval. By deeply exploring complementary semantic information between different modality medical images, these methods offer novel perspectives and tools for the domain of medical image matching and retrieval.

Advancements in machine learning algorithms for brain tumor detection and segmentation: A comprehensive analysis

Brain tumors pose a substantial health challenge globally. Their accurate detection and segmentation are crucial for effective treatment, and recent advancements in machine learning (ML) present a promising solution to these tasks. This paper provides a comprehensive analysis of traditional and modern ML algorithms for brain tumor detection and segmentation. It highlights the pivotal role of ML in advancing brain tumor analysis and how it can potentially mitigate the impact of malignant tumors. Traditional image processing techniques have shown their value but face limitations in dealing with the complexity of brain tumors. The integration of ML has substantially enhanced the capabilities of traditional detection techniques, with architectures such as convolutional neural networks (CNNs) providing improved results. Moreover, brain tumor segmentation techniques have also seen significant enhancements, with the transition from conventional techniques like Region Growing and Watershed methods to state-of-the-art deep learning methods, such as U-Net. Despite these advancements, great challenges remain. Ongoing researches are necessary to further harness the potential of ML in brain tumor diagnosis and treatment. The findings of this review underscore the significance of ML in brain tumor analysis and its profound potential impact on patient outcomes and the overall landscape of cancer treatment.

An Efficient and Robust Multi Directional Deep Learning Based Licence Plate Recognition

Smart cities must have all the important characteristics to achieve their intended goals. Proper traffic management and controlling, increased surveillance and safety, and enhanced management and avoidance of incidents must be the priorities of smart cities. Meanwhile, license plate recognition (LPR) has become the most debatable topic in the research community due to various real-time applications, such as “law enforcement, toll-free processing, access control, and traffic surveillance.” Automated LPR is a technique based on computer vision to recognize vehicles with their number plates. This study discusses various “Deep Learning based LPR” techniques to detect and identify “alphanumeric characters” in number plate. The projected model works on “license based detection” and “character recognition.” This technique uses “Optical Character Recognition (OCR)” technology to detect and extract the alphanumerical numbers from the license plate.  This study is based on secondary data collected from various studies conducted on Licence Plate Recognition using various Deep Learning models published in databases like Google Scholar, Science Direct, NCBI, etc. Deep learning has been used widely in applications related to computer vision in recent years with great perfection. It is a great solution for modern and traditional image processing, object detection, and feature extraction issues. It has been widely used in different stages of LPR like character segmentation, license plate recognition, and OCR.

Facial wrinkle segmentation using weighted deep supervision and semi-automatic labeling

Facial wrinkles are important indicators of human aging. Recently, a method using deep learning and a semi-automatic labeling was proposed to segment facial wrinkles, which showed much better performance than conventional image-processing-based methods. However, the difficulty of wrinkle segmentation remains challenging due to the thinness of wrinkles and their small proportion in the entire image. Therefore, performance improvement in wrinkle segmentation is still necessary. To address this issue, we propose a novel loss function that takes into account the thickness of wrinkles based on the semi-automatic labeling approach. First, considering the different spatial dimensions of the decoder in the U-Net architecture, we generated weighted wrinkle maps from ground truth. These weighted wrinkle maps were used to calculate the training losses more accurately than the existing deep supervision approach. This new loss computation approach is defined as weighted deep supervision in our study. The proposed method was evaluated using an image dataset obtained from a professional skin analysis device and labeled using semi-automatic labeling. In our experiment, the proposed weighted deep supervision showed higher Jaccard Similarity Index (JSI) performance for wrinkle segmentation compared to conventional deep supervision and traditional image processing methods. Additionally, we conducted experiments on the labeling using a semi-automatic labeling approach, which had not been explored in previous research, and compared it with human labeling. The semi-automatic labeling technology showed more consistent wrinkle labels than human-made labels. Furthermore, to assess the scalability of the proposed method to other domains, we applied it to retinal vessel segmentation. The results demonstrated superior performance of the proposed method compared to existing retinal vessel segmentation approaches. In conclusion, the proposed method offers high performance and can be easily applied to various biomedical domains and U-Net-based architectures. Therefore, the proposed approach will be beneficial for various biomedical imaging approaches. To facilitate this, we have made the source code of the proposed method publicly available at: https://github.com/resemin/WeightedDeepSupervision.

Image Processing Algorithm Design of Agricultural Internet of Things Platform Based on Big Data Analysis

Abstract To solve the problems of low signal to noise ratio (SNR), poor processing effect, and long processing times for traditional image processing algorithms, an image processing algorithm for the agricultural Internet of Things platform based on big data analysis was designed. The big data analysis method was used for agricultural low-quality image recognition, and the Internet of Things platform uses sparse representation and a complete dictionary of low-quality image denoising processing with an improved histogram equalization method to enhance the image denoising result and get a high-quality image. Through this, the image processing algorithm design of the agricultural Internet of Things platform was completed. Experimental results show that the algorithm has high SNR and a good image processing effect, and the image processing time is always below 0.7 s, which meets research expectations.

Uncrewed undersea vehicle embedded autonomous target recognition in synthetic aperture sonar imagery

The opacity of the ocean to electromagnetic signals makes communication with Uncrewed Undersea Vehicles (UUVs) extremely difficult, especially for situations where the vehicle is operating far from its center of command and control, and makes mid-operation data exfiltration for external processing untenable. This leaves operator intervention and control limited necessitating the need for advanced and adaptive autonomy that can create situational awareness from mission data provided by vehicle sensors. Long range high-resolution imagery generated by synthetic aperture sonar (SAS) is a high fidelity data product that can be further processed by both machine learning techniques and traditional image processing techniques to create computer aided detection and classification (CAD/CAC) capabilities to feed that developing autonomy need. This work examines two separate detection and classification pipelines that have been used in parallel to create contact sets that are actionable for onboard decision processes.

Quantitative 3D characterization of chromite ore particles

The main techniques used to characterize raw materials are currently bulk or 2D. This is a consequence of the current lack of standardized and automated methods to characterize particulate materials in 3D. Here, we apply a workflow to characterize a crushed chromite ore with nine particle size classes below 1 mm using X-ray computed tomography. All data processing of all samples follows the same sequence of steps, which means that the analysis can be automated with limited user input as opposed to traditional 3D image processing methods that require user input specific to each particle size fraction. Results of chromite composition, particle size distribution and chromite liberation are obtained for individual particles and compared with the results from x-ray diffraction and 2D-based automated mineralogy. The results shows a consistent accuracy across all size classes down to 75 μm. For the larger particle sizes (>600 μm) the chromite liberation curves are more consistent than those obtained from 2D-based automated mineralogy, possibly due to the stereological bias of 2D sections. The particle size distributions is the property for which the 2D bias causes a larger divergence from 3D results across all particle sizes. In conclusion, the workflow is more automatable (thus, faster and cheaper) and less bias (thus, more accurate and standardisable) than other 3D image analysis methods. Additionally, it stands as complementary to established techniques for particle-based characterization, especially to measure particle properties that 2D-based methods may not measure representatively for larger particle sizes and when sampling is limited. Further testing of the workflow in progressively more complex materials is necessary, but its potential to transform the way mineral particulate materials are characterized is demonstrated.

Obstacle Detection Method of Underground Electric Locomotive Rail Based on Instance Segmentation

Real-time and accurate obstacle detection is a vital technology for electric locomotives, especially as driverless vehicles are introduced. A method of obstacle detection for underground electric locomotive rail based on instance segmentation is developed to solve the problems of misdetection and missing detection, low detection accuracy, and slow detection speed of rail obstacles. The method of locating the track mask, demarcating the effective driving boundary, expanding the track mask, and forming the effective driving area is adopted to verify whether the target is an obstacle based on whether the target is located in the effective driving area, to avoid the problem of misdetection and missing detection of the target obstacle. The YOLACT++ (You Only Look At CoefficienTs) model is improved, and path augmentation and target classification loss function replacement strategies are adopted to enhance the model’s ability to detect target details and increase the accuracy of target segmentation. Compared with traditional image processing, this method can detect both straight rail and turnout. The mean average precision of boundary box mAP0.5(box) and mask mAP0.5(mask) of the improved YOLACT++ model reaches 98.52% and 98.55%, which is higher than that of the YOLACT++ model, and the detection frame rate reaches 21.9 frames per second.

An automatic progressive chromosome segmentation approach using deep learning with traditional image processing.

The fully automatic chromosome analysis system plays an important role in the detection of genetic diseases, which in turn can reduce the diagnosis burden for cytogenetic experts. Chromosome segmentation is a critical step for such a system. However, due to the non-rigid structure of chromosomes, chromosomes may curve in any direction, and two or more chromosomes may touch or overlap to form unpredictable chromosome clusters in metaphase chromosome images, leading to automatic chromosome segmentation as a challenge. In this paper, we propose an automatic progressive segmentation approach to perform the entire metaphase chromosome image segmentation using deep learning with traditional image processing. It follows three stages. In the first stage, thresholding-based and geometric-based methods are employed to divide all chromosomes as single ones and chromosome clusters. To tackle the segmentation for unpredictable chromosome clusters, we first present a new chromosome cluster identification network named CCI-Net to classify all chromosome clusters into different types in the second stage, and then in the third stage, we combine traditional image processing with deep CNNs to accomplish chromosome instance segmentation from different types of clusters. Evaluation results on a clinical dataset of 1148 metaphase chromosome images show that the proposed automatic progressive segmentation method achieves 94.60% chromosome cluster identification accuracy and 99.15% instance segmentation accuracy. The experimental results exhibit that our proposed approach can effectively identify chromosome clusters and successfully perform fully automatic chromosome segmentation.

Comparative study on deep-learning-based leather surface defect identification

Detecting leather surface defects has become an important subject in industrial inspections and attracted significant attention as a challenging task. Traditional image processing techniques struggle with the detection of leather surface defects with a variety of shapes, sizes, backgrounds, and noise. Deep learning is a promising solution to this problem. This work focuses on the comparative study of 26 classical deep learning models in the field of leather surface defect type recognition. That aims to lay a foundation for the design and development of new schemes for leather defect inspection. Based on tanned leather from an enterprise, eight types of leather surface defects (cavities, pinholes, scratches, rotten surfaces, growth lines, healing wounds, folds, and bacterial wounds) were collected using an ultra-high definition whole leather imaging device. Two challenging datasets with various shapes, sizes, and colours were constructed. Extensive experimental evaluations were conducted. The deep learning model can achieve more than 95% accuracy when the defect imaging is ideal and limited. In case that the shapes, sizes, and colour of the above eight defects keep diverse, Densenet169 performed the best with a recognition accuracy of 72.5%, and ShuffleNet model with the worst performance reached 64.3%. Systematic in-depth experimental evaluation shows that deep learning models are promising in the field of leather surface defect detection, however, challenges remain.

Automatic borehole fracture detection and characterization with tailored Faster R-CNN and simplified Hough transform

Understanding the geological information of fractures from borehole images is essential for conducting a thorough geological analysis in engineering. Recently, the convolutional neural network (CNN) has been employed to extract high-level features of fractures from images. We propose an improved Faster R-CNN (region-based CNN) model for fracture detection, which is suitably equipped with a tailored region proposal network f-RPN for borehole fractures. The anchors generated by f-RPN are specifically designed with a fixed width and varying heights, and only the anchors with the maximum intersection-over-union values are labeled as positive. By taking advantage of the cyclic feature of borehole images, the images are expanded in four orientations to increase the amount and diversity of the model input. The detection results of the model can further be utilized to simplify the Hough transform for characterizing dip angles and dip directions of fractures. The proposed method is validated with borehole images from multiple projects in hydraulic engineering. Experiments reveal that the average detection accuracy is approximately 91.5%, which exhibits an improvement of 11.06% compared to the original Faster R-CNN. Further analysis indicates that both f-RPN and the multi-orientation expansion contribute to improving the prediction accuracy. It is also shown that this method demonstrates robustness against blurred image background and outperforms traditional image processing methods in the presence of noise. The fracture characterization by the simplified Hough transform turns out to be computationally efficient and accurate, with the absolute errors less than 5° compared to the manual identification results.

Attention-Guided Multitask Learning for Surface Defect Identification

Surface defect identification is an essential task in the industrial quality control process, in which visual checks are conducted on a manufactured product to ensure that it meets quality standards. Convolutional Neural Network (CNN) based surface defect identification method has proven to outperform traditional image processing techniques. However, the real-world surface defect datasets are limited in size due to the expensive data generation process and the rare occurrence of defects. To address this issue, this paper presents a method for exploiting auxiliary information beyond the primary labels to improve the generalization ability of surface defect identification tasks. Considering the correlation between pixel level segmentation masks, object level bounding boxes and global image level classification labels, we argue that jointly learning features of the related tasks can improve the performance of surface defect identification tasks. This paper proposes a framework named Defect-Aux-Net, based on multi-task learning with attention mechanisms that exploit the rich additional information from related tasks with the goal of simultaneously improving robustness and accuracy of the CNN based surface defect identification. We conducted a series of experiments with the proposed framework. The experimental results showed that the proposed method can significantly improve the performance of state-of-the-art models while achieving an overall accuracy of 97.1%, Dice score of 0.926 and mAP of 0.762 on defect classification, segmentation and detection tasks.

A deep segmentation network for crack detection with progressive and hierarchical context fusion

Accurate detection of crack defects in infrastructure is crucial to ensure their safety and extend their service life. However, the presence of complex backgrounds, various shapes and sizes of crack defects, incomplete discontinuous crack defects, and class imbalance makes this task challenging. Traditional image processing techniques are sensitive to image noise and may miss cracks due to weak textures, complex lighting conditions, and other similar objects on the pavement. In recent years, deep learning-based segmentation networks have been proposed for crack defect detection, but they still face challenges in high-precision crack segmentation due to insufficient local feature processing, information loss caused by pooling operations, and limited receptive fields. To address these issues, we propose an end-to-end deep crack segmentation network, called PHCF-Net, which incorporates progressive and hierarchical context fusion. Firstly, the proposed network consists of progressive context fusion (PCF) and hierarchical context fusion (HCF) blocks for effective aggregation of global and local context information. Secondly, a multi-scale context fusion (MCF) block is proposed for multi-scale context extraction and aggregation. Finally, in order to solve the information loss problem caused by pooling operations, a hierarchical context fusion (HCF) block is proposed for effective aggregation of the deep and shallow features. In addition, a multi-scale input unit is also applied to the proposed segmentation network to obtain more context information. To evaluate the performance of PHCF-Net, we have conducted experiments on two publicly available crack segmentation datasets and compared its performance with mainstream segmentation models. The results demonstrate that proposed PHCF-Net achieves better pixel-level crack detection results and outperforms other advanced segmentation models.