Convolutional Neural Network Design Research Articles

An Automatic Architecture Designing Approach of Convolutional Neural Networks for Road Surface Conditions Image Recognition: Tradeoff between Accuracy and Efficiency

Convolutional neural network (CNN) is a promising image recognition technique for winter road surface condition (RSC), a measure that is crucial for winter maintenance operations. In the past, researchers have designed RSC CNN models that displayed acceptable results but did so focusing solely on obtaining high classification accuracy without any consideration for efficiency. Furthermore, when it comes to model development itself, architecture design requires expertise in CNN as well as rich knowledge in the investigated problem itself. To rectify these issues, this paper proposes an innovative approach to automatically design RSC CNN architecture without compromising classification accuracy. The proposed approach uses a weighted sum method, which provides the freedom of choosing relative importance level between accuracy and efficiency. Once the relative importance has been set, one of the most successful and widely adopted heuristics, namely, simulated annealing (SA), is employed to generate (sub)optimal solutions. Results show that both accuracy and efficiency of the automatically generated CNNs are better or at least comparable to the two selected state-of-the-art CNN models, ResNet50 and MobileNet, achieving as high as 93.44% classification accuracy. Ultimately, the outcome of this study fills the gap in existing CNN design methods that do not consider the tradeoff between accuracy and efficiency while providing insight into the effect varying architectures have on CNN model performance.

Image Target Recognition Based on Improved Convolutional Neural Network

Convolutional neural network (CNN) algorithm is a very important branch of deep learning research, which has been widely applied in many fields and achieved excellent results, especially in computer vision, where convolutional neural network has made breakthroughs in image classification and object detection. Convolutional neural network architecture can realize more efficient network training through the final combination of different modules, and the convolutional neural network training does not need to actively extract image features and can directly carry out end-to-end training and prediction. At first, this paper analyzed some problems of the current image recognition and expounds the progress of convolution neural network in image recognition and then studied the traditional algorithm of target recognition, including traditional recognition algorithm framework of target, the target orientation, feature extraction, classifier classification, etc., and the traditional target recognition algorithm is compared with those of the target recognition algorithm of deep learning. On the basis of the above research, an improved model of CNN is proposed, which focuses on the structural design and network optimization of convolutional neural network and designs a more efficient convolutional neural network. Test experiments verify the effectiveness of the proposed model, which not only achieves lower error rate, but also greatly reduces the number of network parameters and has stronger learning ability.

Bayesian Depth-Wise Convolutional Neural Network Design for Brain Tumor MRI Classification.

In recent years, deep learning has been applied to many medical imaging fields, including medical image processing, bioinformatics, medical image classification, segmentation, and prediction tasks. Computer-aided detection systems have been widely adopted in brain tumor classification, prediction, detection, diagnosis, and segmentation tasks. This work proposes a novel model that combines the Bayesian algorithm with depth-wise separable convolutions for accurate classification and predictions of brain tumors. We combine Bayesian modeling learning and Convolutional Neural Network learning methods for accurate prediction results to provide the radiologists the means to classify the Magnetic Resonance Imaging (MRI) images rapidly. After thorough experimental analysis, our proposed model outperforms other state-of-the-art models in terms of validation accuracy, training accuracy, F1-score, recall, and precision. Our model obtained high performances of 99.03% training accuracy and 94.32% validation accuracy, F1-score, precision, and recall values of 0.94, 0.95, and 0.94, respectively. To the best of our knowledge, the proposed work is the first neural network model that combines the hybrid effect of depth-wise separable convolutions with the Bayesian algorithm using encoders.

An Artificial Intelligence Approach for Gene Editing Off-Target Quantification: Convolutional Self-attention Neural Network Designs and Considerations

An Artificial Intelligence Approach for Gene Editing Off-Target Quantification: Convolutional Self-attention Neural Network Designs and Considerations

Landweber Iteration-Inspired Network for Image Super-Resolution

Image super-resolution (SR) is one of the classical ill-posed image processing issues to generate high-resolution (HR) images from given low-resolution (LR) instances. Recent SR works aim to find an elaborate convolutional neural network (CNN) design and regard it as an end-to-end filter to map the image from LR space to HR space. However, seldom of them concentrate on the mathematical proof of network design or consider the problem from an optimization perspective. In this paper, we investigate the image SR based on the Landweber iteration method, which is an effective optimization method to find a feasible solution for the ill-posed problem. By considering the issue from the optimization perspective, we design a corresponding Landweber iteration-inspired network to adaptively learn the parameters and find the HR results. Experimental results show the proposed network achieves competitive or better subjective and objective performance than other state-of-the-art methods with fewer parameters and computational costs.

Toward Reflective Spiking Neural Networks Exploiting Memristive Devices.

The design of modern convolutional artificial neural networks (ANNs) composed of formal neurons copies the architecture of the visual cortex. Signals proceed through a hierarchy, where receptive fields become increasingly more complex and coding sparse. Nowadays, ANNs outperform humans in controlled pattern recognition tasks yet remain far behind in cognition. In part, it happens due to limited knowledge about the higher echelons of the brain hierarchy, where neurons actively generate predictions about what will happen next, i.e., the information processing jumps from reflex to reflection. In this study, we forecast that spiking neural networks (SNNs) can achieve the next qualitative leap. Reflective SNNs may take advantage of their intrinsic dynamics and mimic complex, not reflex-based, brain actions. They also enable a significant reduction in energy consumption. However, the training of SNNs is a challenging problem, strongly limiting their deployment. We then briefly overview new insights provided by the concept of a high-dimensional brain, which has been put forward to explain the potential power of single neurons in higher brain stations and deep SNN layers. Finally, we discuss the prospect of implementing neural networks in memristive systems. Such systems can densely pack on a chip 2D or 3D arrays of plastic synaptic contacts directly processing analog information. Thus, memristive devices are a good candidate for implementing in-memory and in-sensor computing. Then, memristive SNNs can diverge from the development of ANNs and build their niche, cognitive, or reflective computations.

LPLB: An approach for the design of a lightweight convolutional neural network

SUMMARYConvolutional neural network (CNN) is one of the widely used deep neural network architecture for data analytics in the Internet of Things (IoT). However, due to its severe resource requirements, deploying CNN on resource‐constrained edge devices is quite challenging. Moreover, IoT services demand fast data analytics in order to be useful in their context. Hence, ensuring the deployment of CNN models on IoT edge devices is crucial. To this purpose, this article proposes a framework LPLB (less parameters and less bits) for the design of a lightweight CNN with reduced number of parameters and lesser storage requirements while preserving the model accuracy. LPLB consists of three steps: CNN training, iterative parameter drop and post‐training quantization. The performance of LPLB is evaluated on three datasets: German traffic sign recognition dataset, Kaggle hand gesture recognition dataset, and CIFAR‐10 dataset. Experiments conducted in the study reveal that the proposed framework is able to reduce the number of parameters by 11× in case of German traffic sign recognition dataset, 12.5× in case of Kaggle hand gesture recognition dataset, and 14× in case of CIFAR‐10 dataset. Moreover, the storage requirements are reduced by 44.42× in case of German traffic sign recognition dataset, 50× in case of Kaggle hand gesture recognition dataset, and 56.7× in case of CIFAR‐10 dataset. This is achieved without a significant drop in the accuracy. The accuracy drop in case of German traffic sign recognition dataset is 0.215%, 0.18% in case of Kaggle hand gesture recognition dataset, and 0.148% in case of CIFAR‐10 dataset. The proposed framework is generic and can be used for the design of lightweight CNN models corresponding to other use‐cases as well. This will make real‐time classification/recognition possible in delay‐sensitive applications like self‐driving cars, advanced driver assistance systems, health monitoring, elderly posture recognition and so forth.

Deep Learning Image Classification for Fashion Design

Fashion has always been an essential feature in our daily routine. It also plays a significant role in everyone’s lives. In this research, convolutional neural networks (CNN) were used to train images of different fashion styles, which were attempted to be predicted with a high success rate. Deep learning has been widely applied in a variety of fields recently. A CNN is a deep neural network that delivers the most accurate answers when tackling real-world situations. Apparel manufacturers have employed CNN to tackle various difficulties on their e-commerce sites, including clothing recognition, search, and suggestion. A set of photos from the Fashion-MNIST dataset is used to train a series of CNN-based deep learning architectures to distinguish between photographs. CNN design, batch normalization, and residual skip connections reduce the time it takes to learn. The CNN model’s findings are evaluated using the Fashion-MNIST datasets. In this paper, classification is done with a convolutional layer, filter size, and ultimately connected layers. Experiments are run with different activation functions, optimizers, learning rates, dropout rates, and batch sizes. The results showed that the choice of activation function, optimizer, and dropout rate impacts the correctness of the results.

COVID-Net CXR-2: An Enhanced Deep Convolutional Neural Network Design for Detection of COVID-19 Cases From Chest X-ray Images.

As the COVID-19 pandemic devastates globally, the use of chest X-ray (CXR) imaging as a complimentary screening strategy to RT-PCR testing continues to grow given its routine clinical use for respiratory complaint. As part of the COVID-Net open source initiative, we introduce COVID-Net CXR-2, an enhanced deep convolutional neural network design for COVID-19 detection from CXR images built using a greater quantity and diversity of patients than the original COVID-Net. We also introduce a new benchmark dataset composed of 19,203 CXR images from a multinational cohort of 16,656 patients from at least 51 countries, making it the largest, most diverse COVID-19 CXR dataset in open access form. The COVID-Net CXR-2 network achieves sensitivity and positive predictive value of 95.5 and 97.0%, respectively, and was audited in a transparent and responsible manner. Explainability-driven performance validation was used during auditing to gain deeper insights in its decision-making behavior and to ensure clinically relevant factors are leveraged for improving trust in its usage. Radiologist validation was also conducted, where select cases were reviewed and reported on by two board-certified radiologists with over 10 and 19 years of experience, respectively, and showed that the critical factors leveraged by COVID-Net CXR-2 are consistent with radiologist interpretations.

Convolutional neural network and clustering-based codebook design method for massive MIMO systems

In this paper, we propose a convolutional neural network (CNN) and clustering-based codebook design method. Specifically, we train two different CNNs, i.e., CNN1 and CNN2, to compress the channel state information (CSI) matrices into the channel vectors and recover the channel vectors back into the CSI matrices, respectively. After that, the clustering algorithm clusters the output of CNN1, i.e., the channel vectors into several clusters and outputs a centroid for each cluster. The sum distance between each centroid and the channel vectors in the corresponding cluster is the smallest, which can lead to the maximum sum rate of massive MIMO codebook design. Then, the centroids are recovered into matrices by CNN2. The output of CNN2 is our proposed codebook for massive multiple-input multiple-output (MIMO) systems. In the simulation, we compare the performance of different clustering algorithms. We also compare the proposed codebook with the traditional discrete Fourier transform (DFT) codebook. Simulation results show the superiority of the proposed algorithm.

Joint design and compression of convolutional neural networks as a Bi-level optimization problem.

Over the last decade, deep neural networks have shown great success in the fields of machine learning and computer vision. Currently, the CNN (convolutional neural network) is one of the most successful networks, having been applied in a wide variety of application domains, including pattern recognition, medical diagnosis and signal processing. Despite CNNs’ impressive performance, their architectural design remains a significant challenge for researchers and practitioners. The problem of selecting hyperparameters is extremely important for these networks. The reason for this is that the search space grows exponentially in size as the number of layers increases. In fact, all existing classical and evolutionary pruning methods take as input an already pre-trained or designed architecture. None of them take pruning into account during the design process. However, to evaluate the quality and possible compactness of any generated architecture, filter pruning should be applied before the communication with the data set to compute the classification error. For instance, a medium-quality architecture in terms of classification could become a very light and accurate architecture after pruning, and vice versa. Many cases are possible, and the number of possibilities is huge. This motivated us to frame the whole process as a bi-level optimization problem where: (1) architecture generation is done at the upper level (with minimum NB and NNB) while (2) its filter pruning optimization is done at the lower level. Motivated by evolutionary algorithms’ (EAs) success in bi-level optimization, we use the newly suggested co-evolutionary migration-based algorithm (CEMBA) as a search engine in this research to address our bi-level architectural optimization problem. The performance of our suggested technique, called Bi-CNN-D-C (Bi-level convolution neural network design and compression), is evaluated using the widely used benchmark data sets for image classification, called CIFAR-10, CIFAR-100 and ImageNet. Our proposed approach is validated by means of a set of comparative experiments with respect to relevant state-of-the-art architectures.

Evolving Optimised Convolutional Neural Networks for Lung Cancer Classification

Detecting pulmonary nodules early significantly contributes to the treatment success of lung cancer. Several deep learning models for medical image analysis have been developed to help classify pulmonary nodules. The design of convolutional neural network (CNN) architectures, however, is still heavily reliant on human domain knowledge. Manually designing CNN design solutions has been shown to limit the data’s utility by creating a co-dependency on the creator’s cognitive bias, which urges the development of smart CNN architecture design solutions. In this paper, an evolutionary algorithm is used to optimise the classification of pulmonary nodules with CNNs. The implementation of a genetic algorithm (GA) for CNN architectures design and hyperparameter optimisation is proposed, which approximates optimal solutions by implementing a range of bio-inspired mechanisms of natural selection and Darwinism. For comparison purposes, two manually designed deep learning models, FractalNet and Deep Local-Global Network, were trained. The results show an outstanding classification accuracy of the fittest GA-CNN (91.3%), which outperformed both manually designed models. The findings indicate that GAs pose advantageous solutions for diagnostic challenges, the development of which may to be fully automated in the future using GAs to design and optimise CNN architectures for various clinical applications.

Convolutional Neural Network Analysis to Predict Expiratory Flow Limitation During Exercise in Adults with Asthma

OBJECTIVE and HYPOTHESISThe objective of this study was to assess the potential for using well‐known convolutional neural network (CNN) designs to accurately identify the presence of expiratory flow‐limitation (EFL) during exercise in adult asthmatics. We hypothesized that a CNN model would accurately identify spontaneous exercise tidal flow‐volume loops (eFVL) that exceed maximal airflow from the maximal forced expiration (i.e., EFL).METHODS and RESULTSThe expiratory portion of spontaneous eFVLs were placed within the pre‐exercise maximal volitional flow‐volume envelope. Inspiratory capacity maneuvers were performed to determine the correct lung volume for placement of the eFVLs. We assembled a collection of spontaneous eFVLs from one asthmatic subject consisting of an equal number of eFVLs that were flow‐limited and non‐flow‐limited. Subsequently, our overall approach followed standard deep learning practices. The eFVLs were manually labelled as EFL and non‐EFL. The sci‐kit learn train_test_split() method was used to split the eFVLs randomly into training and testing sets using a 67%/33% split. We trained both a one‐ (expired airflow only) and a two‐channel (expired volume and airflow) CNN model to predict the presence or absence of EFL as a response to an eFVL. The models were instantiated as sequential models using the TensorFlow 2.6.0 python API. After 100 training epochs on the training cases, the 1‐channel model was challenged to predict the labels of the previously unseen testing cases. The 1‐channel model achieved 93.3% accuracy at predicting the EFL/non‐EFL labels as a response to the eFVLs despite having not been trained on these data. Additionally, the 2‐channel model was 100% accurate on the unseen test cases. We then challenged the 2‐channel model on 45 unseen eFVLs from a different asthmatic subject; the eFVLs consisted of a balance of EFL and non‐EFL. The model performed poorly, accurately identifying only 57% of the eFVLs as EFL or non‐EFL. We then trained an additional 2‐channel model on a random sample of the combined data from the two subjects and challenged the model on the unseen eFVLs from the same two subjects. At 25 epochs and 100 epochs, the CNN was challenged with unseen testing subjects and achieved 88 and 92% accuracy, respectively, at distinguishing between EFL and non‐EFL.CONCLUSIONSThese proof‐of‐concept findings provide strong evidence that neural network analyses hold promise as a novel method for identifying either the presence or absence of EFL. Given the current, tedious and nuanced methods for determining EFL, such artificial intelligence analyses hold promise for automating the analysis and identification of exercise ventilatory constraints.

Automatic Cancer Cell Taxonomy Using an Ensemble of Deep Neural Networks.

Simple SummaryAs of recently, cancer is considered a major cause of death in developed and developing countries. Therefore, there is an urgent need for improvements in existing diagnostic methods for effective early diagnosis. However, cross-contamination of cancer cell lines results in the development of inappropriate treatments that cannot be administered to patients. To address this issue, we propose an automatic cancer cell taxonomy with high accuracy using optical images of cells obtained through low-scale benchtop optical microscopy. Specifically, we built a deep-learning-based framework to classify cervical, hepatocellular, breast, and lung cancer cells. The experimental results demonstrated that the proposed deep-learning-based approach facilitates the automatic identification of cancer cells. Moreover, our findings provide important insights into the design of convolutional neural networks for various clinical tasks that utilize microscopic images.Microscopic image-based analysis has been intensively performed for pathological studies and diagnosis of diseases. However, mis-authentication of cell lines due to misjudgments by pathologists has been recognized as a serious problem. To address this problem, we propose a deep-learning-based approach for the automatic taxonomy of cancer cell types. A total of 889 bright-field microscopic images of four cancer cell lines were acquired using a benchtop microscope. Individual cells were further segmented and augmented to increase the image dataset. Afterward, deep transfer learning was adopted to accelerate the classification of cancer types. Experiments revealed that the deep-learning-based methods outperformed traditional machine-learning-based methods. Moreover, the Wilcoxon signed-rank test showed that deep ensemble approaches outperformed individual deep-learning-based models (p < 0.001) and were in effect to achieve the classification accuracy up to 97.735%. Additional investigation with the Wilcoxon signed-rank test was conducted to consider various network design choices, such as the type of optimizer, type of learning rate scheduler, degree of fine-tuning, and use of data augmentation. Finally, it was found that the using data augmentation and updating all the weights of a network during fine-tuning improve the overall performance of individual convolutional neural network models.

Multiobjective Tree-Structured Parzen Estimator

Practitioners often encounter challenging real-world problems that involve a simultaneous optimization of multiple objectives in a complex search space. To address these problems, we propose a practical multiobjective Bayesian optimization algorithm. It is an extension of the widely used Tree-structured Parzen Estimator (TPE) algorithm, called Multiobjective Tree-structured Parzen Estimator (MOTPE). We demonstrate that MOTPE approximates the Pareto fronts of a variety of benchmark problems and a convolutional neural network design problem better than existing methods through the numerical results. We also investigate how the configuration of MOTPE affects the behavior and the performance of the method and the effectiveness of asynchronous parallelization of the method based on the empirical results.

TB-Net: A Tailored, Self-Attention Deep Convolutional Neural Network Design for Detection of Tuberculosis Cases From Chest X-Ray Images.

Tuberculosis (TB) remains a global health problem, and is the leading cause of death from an infectious disease. A crucial step in the treatment of tuberculosis is screening high risk populations and the early detection of the disease, with chest x-ray (CXR) imaging being the most widely-used imaging modality. As such, there has been significant recent interest in artificial intelligence-based TB screening solutions for use in resource-limited scenarios where there is a lack of trained healthcare workers with expertise in CXR interpretation. Motivated by this pressing need and the recent recommendation by the World Health Organization (WHO) for the use of computer-aided diagnosis of TB in place of a human reader, we introduce TB-Net, a self-attention deep convolutional neural network tailored for TB case screening. We used CXR data from a multi-national patient cohort to train and test our models. A machine-driven design exploration approach leveraging generative synthesis was used to build a highly customized deep neural network architecture with attention condensers. We conducted an explainability-driven performance validation process to validate TB-Net's decision-making behavior. Experiments on CXR data from a multi-national patient cohort showed that the proposed TB-Net is able to achieve accuracy/sensitivity/specificity of 99.86/100.0/99.71%. Radiologist validation was conducted on select cases by two board-certified radiologists with over 10 and 19 years of experience, respectively, and showed consistency between radiologist interpretation and critical factors leveraged by TB-Net for TB case detection for the case where radiologists identified anomalies. The proposed TB-Net not only achieves high tuberculosis case detection performance in terms of sensitivity and specificity, but also leverages clinically relevant critical factors in its decision making process. While not a production-ready solution, we hope that the open-source release of TB-Net as part of the COVID-Net initiative will support researchers, clinicians, and citizen data scientists in advancing this field in the fight against this global public health crisis.

A novel CAPTCHA solver framework using deep skipping Convolutional Neural Networks.

A Completely Automated Public Turing Test to tell Computers and Humans Apart (CAPTCHA) is used in web systems to secure authentication purposes; it may break using Optical Character Recognition (OCR) type methods. CAPTCHA breakers make web systems highly insecure. However, several techniques to break CAPTCHA suggest CAPTCHA designers about their designed CAPTCHA’s need improvement to prevent computer vision-based malicious attacks. This research primarily used deep learning methods to break state-of-the-art CAPTCHA codes; however, the validation scheme and conventional Convolutional Neural Network (CNN) design still need more confident validation and multi-aspect covering feature schemes. Several public datasets are available of text-based CAPTCHa, including Kaggle and other dataset repositories where self-generation of CAPTCHA datasets are available. The previous studies are dataset-specific only and cannot perform well on other CAPTCHA’s. Therefore, the proposed study uses two publicly available datasets of 4- and 5-character text-based CAPTCHA images to propose a CAPTCHA solver. Furthermore, the proposed study used a skip-connection-based CNN model to solve a CAPTCHA. The proposed research employed 5-folds on data that delivers 10 different CNN models on two datasets with promising results compared to the other studies.

Design of Convolutional Neural Network Based on FPGA

Recently with the rapid development of artificial intelligence AI, various deep learning algorithms represented by Convolutional Neural Networks (CNN) have been widely utilized in various fields, showing their unique advantages; especially in Skin Cancer (SC) imaging Neural networks (NN) are methods for performing machine learning (ML) and reside in what's called deep learning (DL). DL refers to the utilization of multiple layers during a neural network to perform the training and classification of data. The Convolutional Neural Networks (CNNs), a kind of neural network and a prominent machine learning algorithm go through multiple phases before they get implemented in hardware to perform particular tasks for a specific application. State-of-the-art CNNs are computationally intensive, yet their parallel and modular nature make platforms like Field Programmable Gate Arrays (FPGAs) compatible with the acceleration process. The objective of this paper is to implement a hardware architecture capable of running on an FPGA platform of a convolutional neural network CNN, for that, a study was made by describing the operation of the concerned modules, we detail them then we propose a hardware architecture with RTL scheme for each of these modules using the software ISE (Xilinx). The main objective is to show the efficiency of such a realization compared to a GPU based execution. An experimental study is accomplished for the PH2 database set of benchmark images. The proposed FPGA-based CNN design gives competitive results and shows well its efficiency.

Low-Cost and High-Efficiency Electromechanical Integration for Smart Factories of IoT with CNN and FOPID Controller Design under the Impact of COVID-19

This study proposes a design for unmanned chemical factories and implementation based on ultra-low-cost Internet of Things technology, to combat the impact of COVID-19 on industrial factories. A safety and private blockchain network architecture was established, including a three-layer network structure comprising edge, fog, and cloud calculators. Edge computing uses a programmable logic controller and a single-chip microcomputer to transmit and control the motion path of a four-axis robotic arm motor. The fog computing architecture is implemented using Python software. The structure is integrated and applied using a convolutional neural network (CNN) and a fractional-order proportional-integral-derivative controller (FOPID). In addition, edge computing and fog computing signals are transmitted through the blockchain, and can be directly uploaded to the cloud computing controller for signal integration. The integrated application of the production line sensor and image recognition based on the network layer was addressed. We verified the image recognition of the CNN and the robot motor signal control of the FOPID. This study proposes that a CNN + FOPID method can improve the efficiency of the factory by more than 50% compared with traditional manual operators. The low-cost, high-efficiency equipment of the new method has substantial contribution and application potential.

Genetic U-Net: Automatically Designed Deep Networks for Retinal Vessel Segmentation Using a Genetic Algorithm

Recently, many methods based on hand-designed convolutional neural networks (CNNs) have achieved promising results in automatic retinal vessel segmentation. However, these CNNs remain constrained in capturing retinal vessels in complex fundus images. To improve their segmentation performance, these CNNs tend to have many parameters, which may lead to overfitting and high computational complexity. Moreover, the manual design of competitive CNNs is time-consuming and requires extensive empirical knowledge. Herein, a novel automated design method, called Genetic U-Net, is proposed to generate a U-shaped CNN that can achieve better retinal vessel segmentation but with fewer architecture-based parameters, thereby addressing the above issues. First, we devised a condensed but flexible search space based on a U-shaped encoder-decoder. Then, we used an improved genetic algorithm to identify better-performing architectures in the search space and investigated the possibility of finding a superior network architecture with fewer parameters. The experimental results show that the architecture obtained using the proposed method offered a superior performance with less than 1% of the number of the original U-Net parameters in particular and with significantly fewer parameters than other state-of-the-art models. Furthermore, through in-depth investigation of the experimental results, several effective operations and patterns of networks to generate superior retinal vessel segmentations were identified. The codes of this work are available at https://github.com/96jhwei/Genetic-U-Net.