Convolutional Neural Network Image Research Articles

Improving CNN Fish Detection and Classification with Tracking

The regular and consistent monitoring of marine ecosystems and fish communities is becoming more and more crucial due to increasing human pressures. To this end, underwater camera technology has become a major tool to collect an important amount of marine data. As the size of the data collected outgrew the ability to process it, new means of automatic processing have been explored. Convolutional neural networks (CNNs) have been the most popular method for automatic underwater video analysis for the last few years. However, such algorithms are rather image-based and do not exploit the potential of video data. In this paper, we propose a method of coupling video tracking and CNN image analysis to perform a robust and accurate fish classification on deep sea videos and improve automatic classification accuracy. Our method fused CNNs and tracking methods, allowing us to detect 12% more individuals compared to CNN alone.

Automated Cerebrovascular Segmentation and Visualization of Intracranial Time-of-Flight Magnetic Resonance Angiography Based on Deep Learning.

Time-of-flight magnetic resonance angiography (TOF-MRA) is a non-contrast technique used to visualize neurovascular. However, manual reconstruction of the volume render (VR) by radiologists is time-consuming and labor-intensive. Deep learning-based (DL-based) vessel segmentation technology may provide intelligent automation workflow. To evaluate the image quality of DL vessel segmentation for automatically acquiring intracranial arteries in TOF-MRA. A total of 394 TOF-MRA scans were selected, which included cerebral vascular health, aneurysms, or stenoses. Both our proposed method and two state-of-the-art DL methods are evaluated on external datasets for generalization ability. For qualitative assessment, two experienced clinical radiologists evaluated the image quality of cerebrovascular diagnostic and visualization (scoring 0-5 as unacceptable to excellent) obtained by manual VR reconstruction or automatic convolutional neural network (CNN) segmentation. The proposed CNN outperforms the other two DL-based methods in clinical scoring on external datasets, and its visualization was evaluated by readers as having the appearance of the radiologists' manual reconstructions. Scoring of proposed CNN and VR of intracranial arteries demonstrated good to excellent agreement with no significant differences (median, 5.0 and 5.0, P ≥ 12) at healthy-type scans. All proposed CNN image quality were considered to have adequate diagnostic quality (median scores > 2). Quantitative analysis demonstrated a superior dice similarity coefficient of cerebrovascular overlap (training sets and validation sets; 0.947 and 0.927). Automatic cerebrovascular segmentation using DL is feasible and the image quality in terms of vessel integrity, collateral circulation and lesion morphology is comparable to expert manual VR without significant differences.

Enhancing Cataract Detection through Hybrid CNN Approach and Image Quadration: A Solution for Precise Diagnosis and Improved Patient Care

Cataracts, characterized by lens opacity, pose a significant global health concern, leading to blurred vision and potential blindness. Timely detection is crucial, particularly in regions with a shortage of ophthalmologists, where manual diagnosis is time-consuming. While deep learning and convolutional neural networks (CNNs) offer promising solutions, existing models often struggle with diverse datasets. This study introduces a hybrid CNN approach, training on both full retinal fundus images and quadrated parts (i.e., the fundus images divided into four segments). Majority voting is utilized to enhance accuracy, resulting in a superior performance of 97.12%, representing a 1.44% improvement. The hybrid model facilitates early cataract detection, aiding in preventing vision impairment. Integrated into applications, it supports ophthalmologists by providing rapid, cost-efficient predictions. Beyond cataract detection, this research addresses broader computer vision challenges, contributing to various applications. In conclusion, our proposed approach, combining CNNs and image quadration enhances cataract detection’s accuracy, robustness, and generalization. This innovation holds promise for improving patient care and aiding ophthalmologists in precise cataract diagnosis.

Development of Convolutional Neural Network to Segment Ultrasound Images of Histotripsy Ablation.

Histotripsy is a focused ultrasound therapy that ablates tissue via the action of bubble clouds. It is under investigation to treat a number of ailments, including renal tumors. Ultrasound imaging is used to monitor histotripsy, though there remains a lack of definitive imaging metrics to confirm successful treatment outcomes. In this study, a convolutional neural network (CNN) was developed to segment ablation on ultrasound images. A transfer learning approach was used to replace classification layers of the residual network ResNet-18. Inputs to the classification layers were based on ultrasound images of ablated red blood cell phantoms. Digital photographs served as the ground truth. The efficacy of the CNN was compared to subtraction imaging, and manual segmentation of images by two board-certified radiologists. The CNN had a similar performance to manual segmentation, though was improved relative to segmentation with subtraction imaging. Predictions of the network improved over the course of treatment, with the Dice similarity coefficient less than 20% for fewer than 500 applied pulses, but 85% for more than 750 applied pulses. The network was also applied to ultrasound images of ex vivo kidney exposed to histotripsy, which indicated a morphological shift in the treatment profile relative to the phantoms. These findings were consistent with histology that confirmed ablation of the targeted tissue. Overall, the CNN showed promise as a rapid means to assess outcomes of histotripsy and automate treatment. Data collected in this study indicate integration of CNN image segmentation to gauge outcomes for histotripsy ablation holds promise for automating treatment procedures.

Impact of lens autofluorescence and opacification on retinal imaging

BackgroundRetinal imaging, including fundus autofluorescence (FAF), strongly depends on the clearness of the optical media. Lens status is crucial since the ageing lens has both light-blocking and autofluorescence (AF) properties...

Image Enhancement using Convolution Neural Networks

The research presents a comprehensive exploration of the topic of image enhancement using convolutional neural networks (CNN).The research goes deeper into the advanced field of image processing based on the use of neural networks to automatically and efficiently improve the quality and detail of images. The thesis shows that convolutional neural networks are one of the types of deep neural networks, which are specially designed to gain knowledge from big data and extract complex features and patterns found in images. The different layers of the grid are discussed in detail, dealing with images incrementally and extracting different attributes in each layer. The research also highlights CNN's ability to detect, learn and improve important details found in images through convolutions, filtering and data aggregation processes. The proposed CNN image enhancement model was developed and tested on both medical and normal images. The images were optimized using the proposed model and compared with other models. Various quality measures were used to evaluate the results. The results showed that the proposed model can significantly improve the quality of images.

Object detection in optical imaging of the Internet of Things based on deep learning.

This article endeavors to enhance image recognition technology within the context of the Internet of Things (IoT). A dynamic image target detection training model is established through the convolutional neural network (CNN) algorithm within the framework of deep learning (DL). Three distinct model configurations are proposed: a nine-layer convolution model, a seven-layer convolution model, and a residual module convolution model. Subsequently, the simulation model of CNN image target detection based on optical imaging is constructed, and the simulation experiments are conducted in scenarios of simple and salient environments, complex and salient environments, and intricate micro-environment. By determining the optimal training iterations, comparisons are drawn in terms of precision, accuracy, Intersection Over Union (IoU), and frames per second (FPS) among different model configurations. Finally, an attention mechanism is incorporated within the DL framework, leading to the construction of an attention mechanism CNN target detection model that operates at three difficulty levels: simple, intermediate, and challenging. Through comparative analysis against prevalent target detection algorithms, this article delves into the accuracy and detection efficiency of various models for IoT target detection. Key findings include: (1) The seven-layer CNN model exhibits commendable accuracy and confidence in simple and salient environments, although it encounters certain instances of undetected images, indicating scope for improvement. (2) The residual network model, when employing a loss function comprising both mean square error (MSE) and cross entropy, demonstrates superior performance in complex and salient environments, manifesting high precision, IoU, and accuracy metrics, thereby establishing itself as a robust detection model. (3) Within intricate micro-environments, the residual CNN model, utilizing loss functions of MSE and cross entropy, yields substantial results, with precision, IoU, and FPS values amounting to 0.99, 0.83, and 29.9, respectively. (4) The CNN model enriched with an attention mechanism outperforms other models in IoT target image detection, achieving the highest accuracy rates of 24.86%, 17.8%, and 14.77% in the simple, intermediate, and challenging levels, respectively. Although this model entails slightly longer detection times, its overall detection performance is excellent, augmenting the effectiveness of object detection within IoT. This article strives to enhance image target detection accuracy and speed, bolster the recognition capability of IoT systems, and refine dynamic image target detection within IoT settings. The implications encompass reduced manual recognition costs and the provision of a theoretical foundation for optimizing imaging and image target detection technologies in the IoT context.

Wavelet analysis model inspired convolutional neural networks for image denoising

In recent years, many image denoising methods have been proposed based on convolutional neural networks (CNNs). While these methods have shown continuous performance improvement by introducing various mechanisms and structures, their computational cost tends to become increasingly expensive, owing to the resulting complex network architectures. This paper aims at winning the trade-off between computational efficiency and denoising performance for CNN-based image denoisers. Towards this end, we draw inspirations from traditional variational models with wavelet analysis operators for CNN architecture design. A model-inspired CNN is proposed with four key modules: iterative encoding-decoding units inspired by the iterative denoising process, directional convolutions inspired by the separable wavelet filters, inception modules inspired by the multi-scale analysis of wavelets, and stage-wise connections inspired by the adding noise back operation. In experiments, our CNN shows high computational efficiency in both training and test, with competitive results to state-of-the-art approaches.

Merging databases for CNN image recognition, increasing bias or improving results?

Automated microscopy, image processing, and recognition using artificial intelligence is getting a growing interest from the scientific community, as more and more research centres are actively working on building datasets of images for training convolutional neural networks (CNNs) to identify microscopic objects. However, images acquired between institutes might show differences in light and contrast intensity leading to potential bias in identification when using datasets or CNNs from another institute.One might then question if combining datasets acquired in different conditions is likely to improve the efficiency of the resulting CNN by increasing the number of images and integrating lighting variability into the learning process, or on the contrary introduce bias in the CNN training by adding a recurrent noise, common to all classes, through a substantial light and contrast variability.In order to ease collaboration between laboratories, two datasets of middle Eocene radiolarian images, acquired separately at GNS Science (NZ) and the University of Lille (France), were generated to assess the accuracy of CNNs trained on both datasets individually, and also when combined into a third dataset. The performance of the three resulting CNNs was then evaluated on new images acquired at both institutions.Finally, the new radiolarian dataset generated at GNS allowed to easily detect unknown taxa, which are otherwise abundant in the studied material. Seven new species are described: Ceratospyris metroid n. sp., Ceratospyris okazakii n. sp., Desmospyris biloba n. sp., Botryostrobus lagena n. sp., Buryella apiculata n. sp., Lophocyrtis cortesei n. sp., and Cromyosphaera fulgurans n. sp.

A Review of Convolutional Neural Network based Image Denoising Algorithms

Currently, image-denoising algorithms based on convolutional neural networks (CNN)have been widely used and have achieved good results. Compared with traditional image-denoisingmethods, it has powerful learning ability and efficient algorithms. This paper summarizes traditionaldenoising methods and CNN-based image denoising methods, and introduces the basics of imagedenoising in detail, which is helpful for readers who are starting with image denoising processing.In addition, this paper also summarizes some commonly used datasets in the field of imageprocessing, which makes it easier for us to denoise images. Finally, some suggestions for improvingthe performance of CNN image denoising are presented, and possible future research directions arediscussed.

HUMAN GAIT ANALYSIS USING HYBRID CONVOLUTIONAL NEURAL NETWORKS

Human gait analysis is a promising method of researching on human activities like walking or sitting. It reflects the habits of one person and can be observed in any activity that person performs. The patterns in human movements are influenced by many factors, including physiology, social, psychological, and health factors. Differences in limb movements help identify gait patterns, which are often measured using inertial measurement unit sensors (IMU) like gyroscopes and accelerometers placed in various locations throughout the body. This paper analyses the combination of IMU sensors and electromyography sensors (EMG) to improve the identification accuracy of human movements. We propose the hybrid convolutional neural network (CNN) and long short-term memory neuron network (LSTM) for the human gait analysis problem to achieve an accuracy of 0.9418, better than other models including pure CNN models. By using CNN's image classification advancements, we analyse multivariate time series sensor signals by using a sliding window to transform sensor data into image representation and principal component analysis (PCA) to reduce the data dimensionality. To tackle the dataset imbalance issue, we re-weight our model loss by the inverse effective number of samples in each class. We use the human gait HuGaDB dataset with unique characteristics, for gait analysis.

Simulation of unsteady flow around bluff bodies using knowledge-enhanced convolutional neural network

The unsteady flow with massive separation poses challenges to accurately and efficiently simulate wind effects on civil structures, especially in the search for optimal aerodynamic shapes during the preliminary design stage. To this end, an encoding-transformation-decoding convolutional neural network (CNN) architecture is developed in this study to take the pixelated geometry of a bluff body as the input and instantly output the corresponding unsteady aerodynamic flow (represented by a sequence of consecutive flow snapshots). First, a unique (shared) encoding part is used to extract the low-resolution features from the structural shapes and associated initial flow conditions. Then, the transformation part consecutively maps the encoded features (originated from encoder) to a prediction of next flow snapshot in high-level feature space while the separate decoding part unravels the abstract features (from corresponding transformer) to pixelated velocity and pressure fields. Prior domain knowledge is leveraged at various stages of machine learning pipeline to enhance the learning efficiency and accuracy of the proposed convolutional encoder-transformer-decoder. Specifically, the qualitative knowledge that the flow fields vary on a very small length scale within the boundary layer is used to empirically design a nonuniform grid for CNN image inputs and outputs. In addition, the equation-free quantitative knowledge learned from previous flow snapshot (i.e., high-level features from the transformer and filters in the decoder) is employed in the learning of current flow snapshot as the CNN initial conditions based on transfer learning technique. Furthermore, the equation-based quantitative knowledge of both mass and momentum conservation laws is utilized to augment the CNN loss function. Numerical examples suggest good performance of the proposed knowledge-enhanced CNN for unsteady flow simulation of bridge decks. Effects of these three types of knowledge on the performance of the convolutional encoder-transformer-decoder are also examined to highlight their high efficacy to facilitate the training efficiency and simulation accuracy.

Artificial-intelligence-based decision support tools for the differential diagnosis of colitis.

Whereas Artificial Intelligence (AI) based tools have recently been introduced in the field of gastroenterology, application in inflammatory bowel disease (IBD) is in its infancies. We established AI-based algorithms to distinguish IBD from infectious and ischemic colitis using endoscopic images and clinical data. First, we trained and tested a Convolutional Neural Network (CNN) using 1796 real-world images from 494 patients, presenting with three diseases (IBD [n=212], ischemic colitis [n=157], and infectious colitis [n=125]). Moreover, we evaluated a Gradient Boosted Decision Trees (GBDT) algorithm using five clinical parameters as well as a hybrid approach (CNN + GBDT). Patients and images were randomly split into two completely independent datasets. The proposed approaches were benchmarked against each other and three expert endoscopists on the test set. For the image-based CNN, the GBDT algorithm and the hybrid approach global accuracies were .709, .792, and .766, respectively. Positive predictive values were .602, .702, and .657. Global areas under the receiver operating characteristics (ROC) and precision recall (PR) curves were .727/.585, .888/.823, and .838/.733, respectively. Global accuracy did not differ between CNN and endoscopists (.721), but the clinical parameter-based GBDT algorithm outperformed CNN and expert image classification. Decision support systems exclusively based on endoscopic image analysis for the differential diagnosis of colitis, representing a complex clinical challenge, seem not yet to be ready for primetime and more diverse image datasets may be necessary to improve performance in future development. The clinical value of the proposed clinical parameters algorithm should be evaluated in prospective cohorts.

Coherent demodulated underwater wireless optical communication system based on convolutional neural network

The optical vortex beams carrying orbital angular momentum (OAM) is a beam with a special transverse spatial distribution, which has infinite orthogonal properties and can significantly improve the channel capacity of underwater wireless optical communication (UWOC) systems. However, for the underwater wireless optical communication system based on orbital angular momentum shift keying (UWOC-OAM-SK), turbulence in the seawater channel can distort the optical vortex beam and reduce the ability of the system to demodulate the OAM-SK signal, which can reduce the reliability of the system. To improve the performance of UWOC-OAM-SK systems in the channel with different turbulence, a coherent demodulation UWOC-OAM-SK system using an image recognizer based on convolutional neural network (CNN) as demodulator is proposed in this paper. Compared with the incoherent system, the proposed system can obtain a detection image with higher image contrast and more pattern features before the OAM mode is demodulated from the OAM-SK signal at the receiver, therefore has higher reliability. Moreover, the proposed system can recognize mutually conjugate OAM modes, which can greatly save the multiplexing resources in OAM communication. With the excellent image-data processing capability of CNN, the coherent demodulator based on CNN image recognizer has higher demodulation accuracy than the incoherent demodulator in both long distance channels or strongly turbulence channels underwater. The simulation results show that the demodulation accuracy of the proposed system is still higher than 90% in channels with strong turbulence intensity Cn2=1×10−14(K2m−2/3) or transmission distance z=80m. The work done provides some theoretical basis for the design of the UWOC-OAM-SK system.

Identification of tight sandstone reservoir lithofacies based on CNN image recognition technology: A case study of Fuyu reservoir of Sanzhao Sag in Songliao Basin

Currently, lithofacies are automatically classified in petroleum reservoirs based on different algorithm models using well-logging vector datasets. However, well-logging vector datasets are challenging for old oil fields where there are only well-logging images dataset present. Nevertheless, tight sandstone reservoirs, which have been neglected, are becoming exploration targets in old oil fields. This has made it necessary to redefine lithofacies characteristics. Thus, this paper describes a novel, widely applicable method for using a convolutional neural network (CNN) to identify tight sandstone reservoir lithofacies, which can handle well-logging image datasets rather than well-logging vector datasets. We applied this method to the Sanzhao Sag in the Northern Songliao Basin of China using the following steps: (1) dividing typical lithofacies based on core and well-logging characteristics; (2) preparing different lithofacies’ well-logging image datasets; and (3) establishing a CNN model with training, testing, and verification. For lithofacies recognition in a tight sandstone reservoir, compared with the recognized result of a model based on a well-logging vector datasets, such as decision tree analysis (DTA; 83%), support vector machine (SVM; 81%), K-nearest neighbor (KNN; 86%), and random forest (RF; 92%), the CNN model has a better prediction accuracy (92%). It also performs well when recognizing new wells in the Sanzhao Sag (92.16%). This result shows that the CNN model can effectively identify tight sandstone reservoir lithofacies based on a well-logging image dataset. Additionally, the sample intervals, lithofacies, geologic characteristics, and sample numbers affecting the prediction accuracy of the CNN model were also analyzed.

Machine learning for autonomous online exam fraud detection: A concept design

E-learning (EL) has emerged as one of the most valuable means for continuing education around the world, especially in the aftermath of the global pandemic that included a variety of obstacles. Real-time online assessments have become a significant concern for educational organizations. Instances of fraudulent behavior during online exams (OEs) have created considerable challenges for exam invigilators, who are unable to identify and remove such dishonest behavior. In response to this significant issue, educational institutions have used a variety of manual procedures to alleviate the situation, but none of these measures have shown to be particularly innovative or effective. The current study presents a novel strategy for detecting fraudulent actions in real time during OEs that uses convolutional neural network (CNN) algorithms and image processing. The development model will be trained using the CK and CK++ datasets. The training procedure will use 80% of the selected dataset, with the remaining 20% used for model testing to confirm the model’s efficacy and generalization capacity. This project intends to revolutionize the monitoring and prevention of fraudulent actions during online tests by integrating CNN techniques and image processing. The use of CK and CK++ datasets, as well as an 80–20 split for training and testing, contributes to the study’s thorough and rigorous approach. Educational institutions can improve their assessment procedures and maintain the credibility of EL as a credible and equitable way of continuing education by successfully using this unique technique.

Vision Transformers for Vein Biometric Recognition

In October 2020, Google researchers present a promising Deep Learning architecture paradigm for Computer Vision that outperforms the already standard Convolutional Neural Networks (CNNs) on multiple image recognition state-of-the-art datasets: Vision Transformers (ViTs). Based on the self-attention concept inherited from Natural Language Processing (NLP), this new structure surpasses the CNN image classification task on ImageNet, CIFAR-100, and VTAB, among others, when it is fine-tuned (Transfer Leaning) after a previous pre-training on larger datasets. In this work, we confirm this theory and move one step further over the CNN structures applied for Vascular Biometric Recognition (VBR): to the best of our knowledge, we introduce for the first time multiple pure pre-trained and fine-tuned Vision Transformers in this evolving biometric modality to address the challenge of the limited number of samples in VBR datasets. For this purpose, the ViTs have been trained to extract unique image features on the ImageNet-1k and ImageNet-21k and then fine-tuned for the four main existing VBR variants, i.e., finger, palm, hand dorsal, and wrist vein areas. Fourteen existing vascular datasets have been used to perform the vein identification task in the four previously mentioned modalities, based on the True-Positive Identification Rate (TPIR) and 75-25% train-test sets obtaining the following results: HKPU (99.52%), and FV-USM (99.1%); Vera (99.39%), and CASIA (96.00%); Bosphorus (99.86%); PUT-wrist (99.67%), and UC3M-CV1+CV2 (99.67%). Furthermore, we introduce UC3M-CV3: a hygienic contactless wrist database collected on smartphones and consisting of 5400 images from 100 different subjects. The promising results show the Vision Transformer’s versatility in VBR under Transfer Learning and reinforce this new Neural Network architecture paradigm.

Revelation of microcracks as tooth structural element by X-ray tomography and machine learning

Although teeth microcracks (MCs) have long been considered more of an aesthetic problem, their exact role in the structure of a tooth and impact on its functionality is still unknown. The aim of this study was to reveal the possibilities of an X-ray micro-computed tomography (muCT) in combination with convolutional neural network (CNN) assisted voxel classification and volume segmentation for three-dimensional (3D) qualitative analysis of tooth microstructure and verify this approach with four extracted human premolars. Samples were scanned using a muCT instrument (Xradia 520 Versa; ZEISS) and segmented with CNN to identify enamel, dentin, and cracks. A new CNN image segmentation model was trained based on “Multiclass semantic segmentation using DeepLabV3+” example and was implemented with “TensorFlow”. The technique which was used allowed 3D characterization of all MCs of a tooth, regardless of the volume of the tooth in which they begin and extend, and the evaluation of the arrangement of cracks and their structural features. The proposed method revealed an intricate star-shaped network of MCs covering most of the inner tooth, and the main crack planes in all samples were arranged radially in two almost perpendicular directions, suggesting that the cracks could be considered as a planar structure.

Recommender System for the Efficient Treatment of COVID-19 Using a Convolutional Neural Network Model and Image Similarity.

Background: Hospitals face a significant problem meeting patients' medical needs during epidemics, especially when the number of patients increases rapidly, as seen during the recent COVID-19 pandemic. This study designs a treatment recommender system (RS) for the efficient management of human capital and resources such as doctors, medicines, and resources in hospitals. We hypothesize that a deep learning framework, when combined with search paradigms in an image framework, can make the RS very efficient. Methodology: This study uses a Convolutional neural network (CNN) model for the feature extraction of the images and discovers the most similar patients. The input queries patients from the hospital database with similar chest X-ray images. It uses a similarity metric for the similarity computation of the images. Results: This methodology recommends the doctors, medicines, and resources associated with similar patients to a COVID-19 patients being admitted to the hospital. The performance of the proposed RS is verified with five different feature extraction CNN models and four similarity measures. The proposed RS with a ResNet-50 CNN feature extraction model and Maxwell-Boltzmann similarity is found to be a proper framework for treatment recommendation with a mean average precision of more than 0.90 for threshold similarities in the range of 0.7 to 0.9 and an average highest cosine similarity of more than 0.95. Conclusions: Overall, an RS with a CNN model and image similarity is proven as an efficient tool for the proper management of resources during the peak period of pandemics and can be adopted in clinical settings.

Efficiency analysis of artificial vs. Spiking Neural Networks on FPGAs

Artificial Neural Networks (ANNs) incur huge costs in terms of processing power, memory performance, and energy consumption, where in comparison an average human brain operates within a power budget of nearly 20 W. Brain-inspired computing such as Spiking Neural Networks (SNNs) are thus expected to improve efficiency to an unprecedented extent. But apart from the spike coding aspects currently addressed by numerous investigations, research also needs to find solutions for the practical design of future neuromorphic hardware ensuring very low power processing. This paper investigates these questions with a pragmatic comparison of deep Convolutional Neural Networks (CNNs) and their equivalent SNNs based on the implementation and measurement of a set of CNN image classification benchmarks on FPGA devices. Results show that SNNs are clearly less energy efficient than their equivalent CNNs in the general case, further indicating that, on top of ongoing progress in spike modeling theory (e.g. spike encoding, learning), neuromorphic accelerators also have to address important issues in the reality of RTL development and silicon implementation, among which sparsity versus static and idle power consumption, ability to support large levels of parallelism, memory performance, scalability, spiking convolutions.