Input To Convolutional Neural Network Research Articles

Life prediction method of rolling bearing based on CNN-LSTM-AM

Bearing is the key component to determine the health of machinery, and it is of great significance to monitor its working status in real time and predict its remaining useful life. In recent years, the RUL prediction method based on deep learning has been widely used and achieved good prediction results. Here, a bearing life prediction method based on convolution neural network (CNN), long short term memory (LSTM) and attention mechanism (AM) is proposed. First of all, the time domain and frequency domain features of the original vibration signals of rolling bearings are extracted, and the extracted feature set is normalized as the input of CNN. The main function of CNN is to extract spatial features and reduce the dimension of the data. Then, using LSTM to extract the information that may be ignored by CNN, the feature information extracted by CNN-LSTM is input to the attention mechanism for weighting, and the key information is screened. And then more accurately represent the degradation characteristics of the equipment, and finally get the bearing remaining life. The performance of the model is verified by two sets of public data sets, and the experimental results show that it is compared with the CNN-LSTM method. The root mean square error (RMSE) index based on CNN-LSTM-AM method is reduced by 14.6 % and 13.8 % respectively, and the score index is increased by 2.0 % and 2.4 % respectively. The results show that the proposed method has higher accuracy in bearing RUL prediction.

Deep learning-based optimization of field geometry for total marrow irradiation delivered with volumetric modulated arc therapy.

Total marrow (lymphoid) irradiation (TMI/TMLI) is a radiotherapy treatment used to selectively target the bone marrow and lymph nodes in conditioning regimens for allogeneic hematopoietic stem cell transplantation. A complex field geometry is needed to cover the large planning target volume (PTV) of TMI/TMLI with volumetric modulated arc therapy (VMAT). Five isocenters and ten overlapping fields are needed for the upper body, while, for patients with large anatomical conformation, two specific isocenters are placed on the arms. The creation of a field geometry is clinically challenging and is performed by a medical physicist (MP) specialized in TMI/TMLI. To develop convolutional neural networks (CNNs) for automatically generating the field geometry of TMI/TMLI. The dataset comprised 117 patients treated with TMI/TMLI between 2011 and 2023 at our Institute. The CNN input image consisted of three channels, obtained by projecting along the sagittal plane: (1) average CT pixel intensity within the PTV; (2) PTV mask; (3) brain, lungs, liver, bowel, and bladder masks. This "averaged" frontal view combined the information analyzed by the MP when setting the field geometry in the treatment planning system (TPS). Two CNNs were trained to predict the isocenters coordinates and jaws apertures for patients with (CNN-1) and without (CNN-2) isocenters on the arms. Local optimization methods were used to refine the models output based on the anatomy of the patient. Model evaluation was performed on a test set of 15 patients in two ways: (1) by computing the root mean squared error (RMSE) between the CNN output and ground truth; (2) with a qualitative assessment of manual and generated field geometries-scale: 1=not adequate, 4=adequate-carried out in blind mode by three MPs with different expertise in TMI/TMLI. The Wilcoxon signed-rank test was used to evaluate the independence of the given scores between manual and generated configurations (p<0.05 significant). The average and standard deviation values of RMSE for CNN-1 and CNN-2 before/after local optimization were 15±2/13±3mm and 16±2/18±4mm, respectively. The CNNs were integrated into a planning automation software for TMI/TMLI such that the MPs could analyze in detail the proposed field geometries directly in the TPS. The selection of the CNN model to create the field geometry was based on the PTV width to approximate the decision process of an experienced MP and provide a single option of field configuration. We found no significant differences between the manual and generated field geometries for any MP, with median values of 4versus 4 (p=0.92), 3versus 3 (p=0.78), 4versus 3 (p=0.48), respectively. Starting from October 2023, the generated field geometry has been introduced in our clinical practice for prospective patients. The generated field geometries were clinically acceptable and adequate, even for an MP with high level of expertise in TMI/TMLI. Incorporating the knowledge of the MPs into the development cycle was crucial for optimizing the models, especially in this scenario with limited data.

Improving Weather Forecasts for Sailing Events Using a Combination of a Numerical Forecast Model and Machine Learning Postprocessing

Accurate predictions of wind and other weather phenomena are essential for making informed strategic and tactical decisions in sailing. Sailors worldwide utilize current state-of-the-art forecasts, yet such forecasts are often insufficient because they do not offer the high temporal and geographic resolution required by sailors. This paper examines wind forecasting in competitive sailing and demonstrates that traditional wind forecasts can be improved for sailing events by using an integration of traditional numerical modeling and machine learning (ML) methods. Our primary objective is to provide practical and more precise wind forecasts that will give sailors a competitive edge. As a case study, we demonstrate the capabilities of our proposed methods to improve wind forecasting at Lake Kinneret, a popular sailing site. The lake wind pattern is highly influenced by the area’s topographic features and is characterized by unique local and mesoscale phenomena at different times of the day. In this research, we simulate the Kinneret wind during the summers of 2015–2021 in up to one-kilometer resolution using the Weather Research and Forecasting (WRF) atmospheric model. The results are used as input for convolutional neural network (CNN) and multilayer perceptron (MLP) ML models to postprocess and improve the WRF model accuracy. These advanced ML models are trained using training datasets based on the WRF data as well as real data measured by the meteorological service, and subsequently, a validation process of the trained ML model is performed on unseen datasets against site-specific meteorological service observations. Through our experimental analysis, we demonstrate the limitations of the WRF model. It uncovers notable biases in wind direction and velocity, particularly a persistent northern bias in direction and an overestimation of wind strength. Despite its inherent limitations, this study demonstrates that the integration of ML models can potentially improve wind forecasting due to the remarkable prediction accuracy rate achieved by the CNN model, surpassing 95%, while achieving partial success for the MLP model. Furthermore, a successful CNN-based preliminary forecast was effectively generated, suggesting its potential contribution to the future development of a user-friendly tool for sailors.

Condition monitoring of mooring systems for Floating Offshore Wind Turbines using Convolutional Neural Network framework coupled with Autoregressive coefficients

This research presents a novel approach proposed for the monitoring of mooring systems in Floating Offshore Wind Turbines (FOWTs), employing a combination of Convolutional Neural Networks (CNNs) and Auto-Regressive (AR) models. CNN finds broad application in monitoring intricate structures, as they adeptly handle noisy response data without necessitating profound domain expertise. The precision of CNNs relies on the extraction of meaningful features from input data, necessitating meticulous data curation and labeling for optimal computational efficiency and accurate estimation. Emphasis is placed on the preference for feature-rich small datasets over voluminous yet sparse datasets, aiming to enable CNNs to discern crucial patterns more effectively and mitigate issues such as overfitting and extensive preprocessing.The novelty of the proposed approach lies in the integration of AR models, which serve to compress data and enhance damage-sensitive characteristics in the input for CNNs. This integration involves deploying regression models fitted to historical responses, parameterized with AR coefficients sensitive to damage, and further classifying severity using CNNs. The sequential nature of this approach addresses challenges such as vanishing/exploding gradients, particularly for extended historical data, while also attenuating the impact of noise and irrelevant information through data compression.The study explores the effectiveness of the coupled AR-CNN method in monitoring FOWT mooring lines, with a specific focus on two levels of damage identification: detection with classification and damage severity across diverse damage and operational scenarios. The modified methodology exhibits superior outcomes by conducting a performance analysis against traditional CNNs and other machine-learning methods, highlighting the potential of the AR-CNN strategy to improve the precision of FOWT mooring line condition monitoring. These findings underscore the AR-CNN strategy’s potential to enhance the accuracy of FOWT mooring line condition monitoring.

CNN-based model updating for structures by direct use of dynamic structural response measurements

This study presents a convolutional neural network (CNN)-based model updating method for structures using dynamic structural responses. In the presented method, the dynamic structural response is used directly for model updating, thereby omitting the manual process of extracting modal parameters. In the method, the type of stiffness variable in the target structure for model updating is selected, and a number of candidate models are generated by multiple selected stiffness variables in the structures and possible values of variables. Dynamic structural responses are then extracted by time history structural analysis of the candidate models, and a CNN sets dynamic structural response as input, and information on stiffness variables as output is introduced to learn the relationship between the two data. CNNs trained using dynamic displacement response and dynamic acceleration response are presented as Model_XT and Model_XTa, respectively. The presented models can rapidly estimate stiffness of the target structure by entering dynamic responses into the models. These models are applied to model updating for three example structures, and the stiffness estimation performance for model updating is evaluated. Furthermore, the length of data used for CNN training, the stiffness estimation performance according to the CNN input layer settings, and the stiffness estimation applied with limited data to the updated model are analyzed.

SEMANTIC POINT CLOUD SEGMENTATION IN URBAN ENVIRONMENTS WITH 1D CONVOLUTIONAL NEURAL NETWORKS

Abstract. Convolutional Neural Networks (CNNs) have been widely recognized for their efficacy in image analysis tasks. This paper investigates the application of the 1D-CNN variant CNNs for the semantic segmentation of urban point clouds obtained through Mobile Laser Scanning. Ten well-known local geometric features of point clouds were used as input for the 1D CNN. Through an empirical analysis on the Santiago Urban Dataset, the 1D CNN was optimized in terms of numbers of convolution layers, neurons, pooling layers, dropout layers, dense layers, training epochs, and batch size. The performance of the proposed 1D CNN was compared with Support Vector Machine (SVM), Random Forest (RF), and PointNet++. Despite demonstrating a F1-score weighted at 70.3%, outperforming SVM but slightly lagging RF (71.6%) and significantly trailing PointNet++ (90.3%), the proposed 1D-CNN showcases a cost-effective potential for the segmentation of road and building classes. The relative computational requirements of the models were also discussed, highlighting the practical advantages and limitations of each approach.

Automated detection of focal cortical dysplasia based on magnetic resonance imaging and positron emission tomography

Purpose: Focal cortical dysplasia (FCD) is a common etiology of drug-resistant focal epilepsy. Visual identification of FCD is usually time-consuming and depends on personal experience. Herein, we propose an automated type II FCD detection approach utilizing multi-modal data and 3D convolutional neural network (CNN). Methods: MRI and positron emission tomography (PET) data of 82 patients with FCD were collected, including 55 (67.1%) histopathologically, and 27 (32.9%) radiologically diagnosed patients. Three types of morphometric feature maps and three types of tissue maps were extracted from the T1-weighted images. These maps, T1, and PET images formed the inputs for CNN. Five-fold cross-validations were carried out on the training set containing 62 patients, and the model behaving best was chosen to detect FCD on the test set of 20 patients. Furthermore, ablation experiments were performed to estimate the value of PET data and CNN. Results: On the validation set, FCD was detected in 90.3% of the cases, with an average of 1.7 possible lesions per patient. The sensitivity on the test set was 90.0%, with 1.85 possible lesions per patient. Without the PET data, the sensitivity decreased to 80.0%, and the average lesion number increased to 2.05 on the test set. If an artificial neural network replaced the CNN, the sensitivity decreased to 85.0%, and the average lesion number increased to 4.65. Significance: Automated detection of FCD with high sensitivity and few false-positive findings is feasible based on multi-modal data. PET data and CNN could improve the performance of automated detection.

Convolutional neural network intelligent diagnosis method using small samples based on SK-CAM

In order to solve the dependence of convolutional neural networks (CNN) on large samples of training data, an intelligent fault diagnosis method based on spectral kurtosis (SK) and attention mechanism is proposed. Firstly, the SK algorithm is used to obtain two-dimensional fast kurtosis graphs from vibration signals, and the two-dimensional fast spectral kurtosis graphs are converted into one-dimensional kurtosis time-domain samples, which are used as the input of CNN. Then the channel attention module (CAM) is added to CNN, and the weight is increased in the channel domain to eliminate the interference of invalid features. The accuracy of fault identification can reach 99.8 % by applying the proposed method on the fault diagnosis experiment of rolling bearings. Compared with the traditional deep learning (DL) method, the proposed method not only has higher accuracy, but also has lower dependence on the number of samples.

Research on modulation recognition method of electromagnetic signal based on wavelet transform convolutional neural network

The method of electromagnetic signal modulation recognition based on wavelet transform convolutional neural network is studied to improve the effect of electromagnetic signal modulation recognition. By analyzing the electromagnetic signal modulation model, the original electromagnetic signal is preprocessed by wavelet transform to remove the noise of the original electromagnetic signal. The processed electromagnetic signal is used as the input of convolutional neural network, and the electromagnetic signal feature vector is extracted through the convolution layer of convolutional neural network. By using full connection operation, the advanced feature vector of electromagnetic signal is integrated, and the electromagnetic signal is classified by softmax function, and the electromagnetic signal modulation recognition result is output, thus realizing the electromagnetic signal modulation recognition. The experimental results show that when the number of layers of wavelet decomposition is 7 and the wavelet function is Db9, the wavelet transform has the best denoising effect on electromagnetic signal data. At the same time, the network training efficiency of this method is high, and the accuracy of electromagnetic signal modulation recognition is as high as 97.2 %, which improves the effect of electromagnetic signal modulation recognition and is suitable for various types of electromagnetic signal modulation recognition.

Features gradient-based signals selection algorithm of linear complexity for convolutional neural networks

&lt;abstract&gt;&lt;p&gt;Recently, convolutional neural networks (CNNs) for classification by time domain data of multi-signals have been developed. Although some signals are important for correct classification, others are not. The calculation, memory, and data collection costs increase when data that include unimportant signals for classification are taken as the CNN input layer. Therefore, identifying and eliminating non-important signals from the input layer are important. In this study, we proposed a features gradient-based signals selection algorithm (FG-SSA), which can be used for finding and removing non-important signals for classification by utilizing features gradient obtained by the process of gradient-weighted class activation mapping (grad-CAM). When we defined $ n_ \mathrm{s} $ as the number of signals, the computational complexity of FG-SSA is the linear time $ \mathcal{O}(n_ \mathrm{s}) $ (i.e., it has a low calculation cost). We verified the effectiveness of the algorithm using the OPPORTUNITY dataset, which is an open dataset comprising of acceleration signals of human activities. In addition, we checked the average of 6.55 signals from a total of 15 signals (five triaxial sensors) that were removed by FG-SSA while maintaining high generalization scores of classification. Therefore, FG-SSA can find and remove signals that are not important for CNN-based classification. In the process of FG-SSA, the degree of influence of each signal on each class estimation is quantified. Therefore, it is possible to visually determine which signal is effective and which is not for class estimation. FG-SSA is a white-box signal selection algorithm because it can understand why the signal was selected. The existing method, Bayesian optimization, was also able to find superior signal sets, but the computational cost was approximately three times greater than that of FG-SSA. We consider FG-SSA to be a low-computational-cost algorithm.&lt;/p&gt;&lt;/abstract&gt;

Network Robustness Prediction: Influence of Training Data Distributions.

Network robustness refers to the ability of a network to continue its functioning against malicious attacks, which is critical for various natural and industrial networks. Network robustness can be quantitatively measured by a sequence of values that record the remaining functionality after a sequential node-or edge-removal attacks. Robustness evaluations are traditionally determined by attack simulations, which are computationally very time-consuming and sometimes practically infeasible. The convolutional neural network (CNN)-based prediction provides a cost-efficient approach to fast evaluating the network robustness. In this article, the prediction performances of the learning feature representation-based CNN (LFR-CNN) and PATCHY-SAN methods are compared through extensively empirical experiments. Specifically, three distributions of network size in the training data are investigated, including the uniform, Gaussian, and extra distributions. The relationship between the CNN input size and the dimension of the evaluated network is studied. Extensive experimental results reveal that compared to the training data of uniform distribution, the Gaussian and extra distributions can significantly improve both the prediction performance and the generalizability, for both LFR-CNN and PATCHY-SAN, and for various functionality robustness. The extension ability of LFR-CNN is significantly better than PATCHY-SAN, verified by extensive comparisons on predicting the robustness of unseen networks. In general, LFR-CNN outperforms PATCHY-SAN, and thus LFR-CNN is recommended over PATCHY-SAN. However, since both LFR-CNN and PATCHY-SAN have advantages for different scenarios, the optimal settings of the input size of CNN are recommended under different configurations.

Your Input Matters—Comparing Real-Valued PolSAR Data Representations for CNN-Based Segmentation

Inspired by the success of Convolutional Neural Network (CNN)-based deep learning methods for optical image segmentation, there is a growing interest in applying these methods to Polarimetric Synthetic Aperture Radar (PolSAR) data. However, effectively utilizing well-established real-valued CNNs for PolSAR image segmentation requires converting complex-valued data into real-valued representations. This paper presents a systematic comparison of 14 different real-valued representations used as CNN input in the literature. These representations encompass various approaches, including the use of coherency matrix elements, hand-crafted feature vectors, polarimetric features based on target decomposition, and combinations of these methods. The goal is to assess the impact of the choice of PolSAR data representation on segmentation performance and identify the most suitable representation. Four test configurations are employed to achieve this, involving different CNN architectures (U-Net with ResNet-18 or EfficientNet backbone) and PolSAR data acquired in different frequency bands (S- and L-band). The results emphasize the importance of selecting an appropriate real-valued representation for CNN-based PolSAR image segmentation. This study’s findings reveal that combining multiple polarimetric features can potentially enhance segmentation performance but does not consistently improve the results. Therefore, when employing this approach, careful feature selection becomes crucial. In contrast, using coherency matrix elements with amplitude and phase representation consistently achieves high segmentation performance across different test configurations. This representation emerges as one of the most suitable approaches for CNN-based PolSAR image segmentation. Notably, it outperforms the commonly used alternative approach of splitting the coherency matrix elements into real and imaginary parts.

Retracted: 3D Input Convolutional Neural Network for SSVEP Classification in Design of Brain Computer Interface for Patient User.

[This retracts the article DOI: 10.1155/2022/8452002.].

Classification Of Earthquake Observation Stations Using Multiple Input Convolutional Neural Network Method

Analysis of the data quality of earthquake observation stations becomes very important as quality control. However, determining the quality of earthquake observation stations is done manually by analyzing several existing parameters by an expert. This study proposes a deep learning method approach to classify the quality of earthquake observation stations based on the expert’s ability to analyze the quality of earthquake observation station data. The method used is Multiple Input Convolutional Neural Network (MICNN). The proposed MICNN architecture development is a combination of previous research. The dataset used in this study results from recording 411 seismometers for 12 months with a data set containing three horizontal components (East and North) and the vertical component (Z) with a recording length of 30 days. Each component is transformed from the time domain into the frequency domain form to obtain a spectrogram image used as input in the MICNN model. In addition, each component has a CNN block that functions as an extraction feature on the spectrogram image. Classification in this study used three classes: Classified, Usable, and Unusable. Tests and experiments on the model were carried out to obtain the best accuracy by tuning hyperparameters for model validation and the accuracy of this MICNN model was 99%.

Multi-scale EMG classification with spatial-temporal attention for prosthetic hands

A classification framework for hand gestures using Electromyography (EMG) signals in prosthetic hands is presented. Leveraging the multi-scale characteristics and temporal nature of EMG signals, a Convolutional Neural Network (CNN) is used to extract multi-scale features and classify them with spatial-temporal attention. A multi-scale coarse-grained layer introduced into the input of one-dimensional CNN (1D-CNN) facilitates multi-scale feature extraction. The multi-scale features are fed into the attention layer and subsequently given to the fully connected layer to perform classification. The proposed model achieves classification accuracies of 93.4%, 92.8%, 91.3%, and 94.1% for Ninapro DB1, DB2, DB5, and DB7 respectively, thereby enhancing the confidence of prosthetic hand users.

Abstract 17610: A Novel Approach Using Deep Neural Networks on Clinical Cardiac MRIs Can Identify Patients With Clonal Hematopoiesis of Indeterminate Potential With High Accuracy

Background: Clonal hematopoiesis of indeterminate potential (CHIP) has been shown to be an independent risk factor for heart disease and heart failure. DNA sequencing is the gold standard to diagnose CHIP. It is unclear if deep learning techniques using routine clinical imaging can aid in identifying those who are likely to have CHIP and thus likely at increased risk for heart disease and heart failure. Hypothesis: We hypothesized that deep learning models trained on routine cardiac MRI (CMR) would be able to help predict who has CHIP Methods: We enrolled patients into an IRB approved study to evaluate for CHIP and associated imaging findings. Whole exome sequencing and our somatic variant pipeline was used to identify those who had CHIP. Various machine learning techniques were employed for training and testing, including SVM, random forest and convolutional neural networks (CNN) to predict who had CHIP from CMR imaging. Results: We enrolled 98 unique patients with CMR and found that 40 (41%) had CHIP. Majority were female 65 (66%) and there was no significant difference in median age between CHIP vs no CHIP (69, IQR 62-78 vs 65, IQR 54-72). There were no significant differences between CHIP vs no CHIP patients in co-morbidities such as CAD, DM, HTN, HF. Compared to SVM and random forest, the best performing model with high sensitivity, specificity and accuracy was derived from CNN (AUC 0.64 vs 0.71 vs 0.85 respectively). We found that multi-view MRI inputs were more accurate than single view MRI inputs for the CNN model. Using an unsupervised approach with CNN and multi-view input, we were able to predict CHIP with 82% accuracy. Conclusions: CNN is a promising deep learning technique that can accurately classify CHIP and thus those at higher cardiovascular disease risk. Ongoing analyses are underway to validate these findings using the UK Biobank and TOPMED datasets and predict who will develop cardiomyopathy with CHIP.

Generalized global solar radiation forecasting model via cyber-secure deep federated learning

Recently, the increasing prevalence of solar energy in power and energy systems around the world has dramatically increased the importance of accurately predicting solar irradiance. However, the lack of access to data in many regions and the privacy concerns that can arise when collecting and transmitting data from distributed points to a central server pose challenges to current predictive techniques. This study proposes a global solar radiation forecasting approach based on federated learning (FL) and convolutional neural network (CNN). In addition to maintaining input data privacy, the proposed procedure can also be used as a global supermodel. In this paper, data related to eight regions of Iran with different climatic features are considered as CNN input for network training in each client. To test the effectiveness of the global supermodel, data related to three new regions of Iran named Abadeh, Jarqavieh, and Arak are used. It can be seen that the global forecasting supermodel was able to forecast solar radiation for Abadeh, Jarqavieh, and Arak regions with 95%, 92%, and 90% accuracy coefficients, respectively. Finally, in a comparative scenario, various conventional machine learning and deep learning models are employed to forecast solar radiation in each of the study regions. The results of the above approaches are compared and evaluated with the results of the proposed FL-based method. The results show that, since no training data were available from regions of Abadeh, Jarqavieh, and Arak, the conventional methods were not able to forecast solar radiation in these regions. This evaluation confirms the high ability of the presented FL approach to make acceptable predictions while preserving privacy and eliminating model reliance on training data.

Feature extraction and classification of deep-sea mobile underwater acoustic channels

The reliable acoustic path (RAP) is one of the crucial channels for deep-sea sound propagation, which is affected weakly by the interface and has lower transmission loss, enabling long-distance communication. However, RAP-based deep-sea acoustic communication may face channel model mismatch issues. In order to analyze the dynamic characteristics of spatial-temporalvariability channels, deep-sea mobile underwater acoustic channel measurement experiments were conducted. This work proposes a deep learning method based on multi-dimensional properties to classify deep-sea channels. Specifically, the sound ray convergence zone leads to a complex multipath structure and severe delay spread in the RAP channel. The fuzzy c-means (FCM) algorithm is used for multipath clustering to extract accurate channel features, and then the Markov chain (MC) is introduced to track the evolution characteristics of multipath clusters. Finally, the coupling features of channel time-variant impulse response (TVIR) and multi-dimensionalstatistical properties are used as the input of convolutional neural networks (CNNs) to obtain the quantitative evaluation index as the channel classification to build a channel feature dataset for underwater mobile platforms. This dataset can effectively assist in identifying deep-sea mobile channels and promote the development of adaptive underwater acoustic communication systems on mobile platforms.

A framework for immunofluorescence image augmentation and classification based on unsupervised attention mechanism.

Autoimmune encephalitis (AE) is a common neurological disorder. As a standard method for neuroautoantibody detection, pathologists use tissue matrix assays (TBA) for initial disease screening. In this study, microscopic fluorescence imaging was combined with deep learning to improve AE diagnostic accuracy. Due to the inter-class imbalance of medical data, we propose an innovative generative adversarial network supplemented with attention mechanisms to highlight key regions in images to synthesize high-quality fluorescence images. However, securing annotated medical data is both time-consuming and costly. To circumvent this problem, we employ a self-supervised learning approach that utilizes unlabeled fluorescence data to support downstream classification tasks. To better understand the fluorescence properties in the data, we introduce a multichannel input convolutional neural network that adds additional channels of fluorescence intensity. This study builds an AE immunofluorescence dataset and obtains the classification accuracy of 88.5% using our method, thus confirming the effectiveness of the proposed method.

Gray-to-color image conversion in the classification of breast lesions on ultrasound using pre-trained deep neural networks.

Breast ultrasound (BUS) image classification in benign and malignant classes is often based on pre-trained convolutional neural networks (CNNs) to cope with small-sized training data. Nevertheless, BUS images are single-channel gray-level images, whereas pre-trained CNNs learned from color images with red, green, and blue (RGB) components. Thus, a gray-to-color conversion method is applied to fit the BUS image to the CNN's input layer size. This paper evaluates 13 gray-to-color conversion methods proposed in the literature that follow three strategies: replicating the gray-level image to all RGB channels, decomposing the image to enhance inherent information like the lesion's texture and morphology, and learning a matching layer. Besides, we introduce an image decomposition method based on the lesion's structural information to describe its inner and outer complexity. These gray-to-color conversion methods are evaluated under the same experimental framework using a pre-trained CNN architecture named ResNet-18 and a BUS dataset with more than 3000 images. In addition, the Matthews correlation coefficient (MCC), sensitivity (SEN), and specificity (SPE) measure the classification performance. The experimental results show that decomposition methods outperform replication and learning-based methods when using information from the lesion's binary mask (obtained from a segmentation method), reaching an MCC value greater than0.70 and specificity up to0.92, although the sensitivity is about0.80. On the other hand, regarding the proposed method, the trade-off between sensitivity and specificity is better balanced, obtaining about0.88 for both indices and an MCC of0.73. This study contributes to the objective assessment of different gray-to-color conversion approaches in classifying breast lesions, revealing that mask-based decomposition methods improve classification performance. Besides, the proposed method based on structural information improves the sensitivity, obtaining more reliable classification results on malignant cases and potentially benefiting clinical practice.