Training Deep Neural Networks Research Articles

Reducing the Reality Gap Using Hybrid Data for Real-Time Autonomous Operations

This paper presents an ablation study aimed at investigating the impact of a hybrid dataset, domain randomisation, and custom-designed neural network architecture on the performance of object localisation. In this regard, real images were gathered from the Boeing 737-400 aircraft while synthetic images were generated using the domain randomisation technique involved randomising various parameters of the simulation environment in a photo-realistic manner. The study results indicated that the use of the hybrid dataset, domain randomisation, and the custom-designed neural network architecture yielded a significant enhancement in object localisation performance. Furthermore, the study demonstrated that domain randomisation facilitated the reduction of the reality gap between the real-world and simulation environments, leading to a better generalisation of the neural network architecture on real-world data. Additionally, the ablation study delved into the impact of each randomisation parameter on the neural network architecture’s performance. The insights gleaned from this investigation shed light on the importance of each constituent component of the proposed methodology and how they interact to enhance object localisation performance. The study affirms that deploying a hybrid dataset, domain randomisation, and custom-designed neural network architecture is an effective approach to training deep neural networks for object localisation tasks. The findings of this study can be applied to a wide range of computer vision applications, particularly in scenarios where collecting large amounts of labelled real-world data is challenging. The study employed a custom-designed neural network architecture that achieved 99.19% accuracy, 98.26% precision, 99.58% recall, and 97.92% mAP@.95 trained using a hybrid dataset comprising synthetic and real images.

Approximate Scenario-Based Economic Model Predictive Control With Application to Wind Energy Conversion System

This paper considers the effective handling of uncertainty for economic model predictive control with feasibility and stability guarantees. Firstly, a stable scenario-based economic model predictive control strategy is proposed based on Lyapunov techniques. This control strategy optimizes over a sequence of control policies instead of a sequence of control inputs, so as to take feedback into account to reduce the conservativeness. More uncertainty information over the prediction horizon is incorporated by employing an augmented prediction model with a scenario tree describing the evolution of the uncertainty. Secondly, since the scenario tree structure inevitably increases the optimization problem size, a trained deep neural network, as an approximation function, is resorted to modeling the scenario-based economic model predictive control feedback control law to make online implementation tractable. The effectiveness of this approximate controller is verified through the probabilistic validation technique. Finally, the feasibility and stability of this approximate scenario-based economic model predictive control are addressed theoretically. An application of this proposed controller on wind energy conversion systems demonstrates its effectiveness.

A Short Overview of the Main Concepts of Artificial Neural Networks

A significant increase in computer performance, the accumulation of a large amount of data necessary for training deep neural networks, the development of training methods for neural networks that allow you to quickly and efficiently train networks consisting of a hundred or more layers, has led to significant progress in training deep neural networks. This allowed deep neural networks to take a leading position among machine learning methods. In this work, neural network paradigms (and their methods of training and functioning) considers, such as Rosenblatt perceptron, multilayer perceptrons, radial basis function network, Kohonen network, Hopfield network, Boltzmann machine, and deep neural networks. As a result of comparative consideration of these paradigms, it can be concluded that they all successfully solve the tasks set before them, but now, deep neural networks are the most effective mechanism for solving intellectual practical tasks.

Unpaired image-to-image translation of structural damage

Condition assessment of civil infrastructure from manual inspections can be time consuming, subjective, and unsafe. Advances in computer vision and Deep Neural Networks (DNNs) provide methods for automating important condition assessment tasks such as damage and context identification. One critical challenge towards the training of robust and generalizable DNNs for damage identification is the difficulty in obtaining large and diverse datasets. To maximally leverage available data, researchers have investigated using synthetic images of damaged structures from Generative Adversarial Networks (GANs) for data augmentation. However, GANs are limited in the diversity of data they can produce as they are only able to interpolate between samples of damaged structures in a dataset. Unpaired image-to-image translation using Cycle Consistent Adversarial Networks (CCAN) provide one means of extending the diversity and control in generated images, but have not been investigated for applications in condition assessment. We present EIGAN, a novel CCAN architecture for generating realistic synthetic images of a damaged structure, given an image of its undamaged state. EIGAN has the capability to translate undamaged images to damaged representations and vice-versa while retaining the geometric structure of the infrastructure (e.g, building shape, layout, color, size etc). We create a new unpaired dataset of damaged and undamaged building images taken after the 2017 Puebla Earthquake. Using this dataset, we demonstrate how EIGAN is able to address shortcomings of three other established CCAN architectures specifically for damage translation with both qualitative and quantitative measures. Additionally, we introduce a new methodology to explore the latent space of EIGAN allowing for some control over the properties of the generated damage (e.g., the damage severity). The results demonstrate that unpaired image-to-image translation of undamaged to damaged structures is an effective means of data augmentation to improve network performance.

Cross-Modal Supervision-Based Multitask Learning With Automotive Radar Raw Data

With the rapid development of autonomous driving technology, radar sensors play a vital role in the perception system due to their robustness under harsh environmental conditions, exact range and velocity perception capability. However, the state-of-the-art performance of algorithms solely based on radar to achieve various perception tasks, such as classifying road users and infrastructures, still lags far behind expectation. Their failure can mainly be accounted for the extreme sparseness of radar point cloud for objects, low angular resolution, and the issue of ghost targets. In this work, we propose a novel network that employs the complex range-Doppler matrix as input to achieve radar-tailored panoptic segmentation (i.e., <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">free-space segmentation</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">object detection</i> ). Our network surpasses previous works in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">free-space segmentation</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">object detection</i> tasks, and the improvement in the former task is especially notable. During training, the segmented camera image with radar customized adaption is utilized as the ground truth. Through such a cross-modal supervision method, the labeling expense is alleviated considerably. Based on it, we further design an innovative camera-radar system concept that is able to automatically train deep neural networks with radar measurement.

102 AI-Based Molecular Classification of Diffuse Gliomas using Rapid, Label-Free Optical Imaging

Molecular classification has transformed the management of brain tumors by enabling more accurate prognostication and personalized treatment. Access to timely molecular diagnostic testing for brain tumor patients is limited, complicating surgical and adjuvant treatment and obstructing clinical trial enrollment. By combining stimulated Raman histology (SRH), a rapid, label-free, non-consumptive, optical imaging method, and deep learning-based image classification, we are able to predict the molecular genetic features used by the World Health Organization (WHO) to define the adult-type diffuse glioma taxonomy, including IDH-1/2, 1p19q-codeletion, and ATRX loss. We developed a multimodal deep neural network training strategy that uses both SRH images and large-scale, public diffuse glioma genomic data (i.e. TCGA, CGGA, etc.) in order to achieve optimal molecular classification performance. One institution was used for model training (University of Michigan) and four institutions (NYU, UCSF, Medical University of Vienna, and University Hospital Cologne) were included for patient enrollment in the prospective testing cohort. Using our system, called DeepGlioma, we achieved an average molecular genetic classification accuracy of 93.2% and identified the correct diffuse glioma molecular subgroup with 91.5% accuracy within 2 minutes in the operating room. DeepGlioma outperformed conventional IDH1-R132H immunohistochemistry (94.2% versus 91.4% accuracy) as a first-line molecular diagnostic screening method for diffuse gliomas and can detect canonical and non-canonical IDH mutations. Our results demonstrate how artificial intelligence and optical histology can be used to provide a rapid and scalable alternative to wet lab methods for the molecular diagnosis of brain tumor patients during surgery.

Information Entropy Measures for Evaluation of Reliability of Deep Neural Network Results.

Deep neural networks (DNN) try to analyze given data, to come up with decisions regarding the inputs. The decision-making process of the DNN model is not entirely transparent. The confidence of the model predictions on new data fed into the network can vary. We address the question of certainty of decision making and adequacy of information capturing by DNN models during this process of decision-making. We introduce a measure called certainty index, which is based on the outputs in the most penultimate layer of DNN. In this approach, we employed iEEG (intracranial electroencephalogram) data to train and test DNN. When arriving at model predictions, the contribution of the entire information content of the input may be important. We explored the relationship between the certainty of DNN predictions and information content of the signal by estimating the sample entropy and using a heatmap of the signal. While it can be assumed that the entire sample must be utilized for arriving at the most appropriate decisions, an evaluation of DNNs from this standpoint has not been reported. We demonstrate that the robustness of the relationship between certainty index with the sample entropy, demonstrated through sample entropy-heatmap correlation, is higher than that with the original signal, indicating that the DNN focuses on information rich regions of the signal to arrive at decisions. Therefore, it can be concluded that the certainty of a decision is related to the DNN's ability to capture the information in the original signal. Our results indicate that, within its limitations, the certainty index can be used as useful tool in estimating the confidence of predictions. The certainty index appears to be related to how effectively DNN heatmaps captured the information content in the signal.

Deep neural network training method based on vectorgraphs for designing of metamaterial broadband polarization converters

In this work, we proposed a method of extracting feature parameters for deep neural network prediction based on the vectorgraph storage format, which can be applied to the design of electromagnetic metamaterials with sandwich structures. Compared to current methods of manually extracting feature parameters, this method can automatically and precisely extract the feature parameters of arbitrary two-dimensional surface patterns of the sandwich structure. The position and size of surface patterns can be freely defined, and the surface patterns can be easily scaled, rotated, translated, or transformed in other ways. Compared to the pixel graph feature extraction method, this method can adapt to very complex surface pattern design in a more efficient way. And the response band can be easily shifted by scaling the designed surface pattern. To illustrate and verify the method, a 7-layer deep neural network was built to design a metamaterial broadband polarization converter. Prototype samples were fabricated and tested to verify the accuracy of the prediction results. In general, the method is potentially applicable to the design of different kinds of sandwich-structure metamaterials, with different functions and in different frequency bands.

Spatial gradient consistency for unsupervised learning of hyperspectral demosaicking: application to surgical imaging

PurposeHyperspectral imaging has the potential to improve intraoperative decision making if tissue characterisation is performed in real-time and with high-resolution. Hyperspectral snapshot mosaic sensors offer a promising approach due to their fast acquisition speed and compact size. However, a demosaicking algorithm is required to fully recover the spatial and spectral information of the snapshot images. Most state-of-the-art demosaicking algorithms require ground-truth training data with paired snapshot and high-resolution hyperspectral images, but such imagery pairs with the exact same scene are physically impossible to acquire in intraoperative settings. In this work, we present a fully unsupervised hyperspectral image demosaicking algorithm which only requires exemplar snapshot images for training purposes.MethodsWe regard hyperspectral demosaicking as an ill-posed linear inverse problem which we solve using a deep neural network. We take advantage of the spectral correlation occurring in natural scenes to design a novel inter spectral band regularisation term based on spatial gradient consistency. By combining our proposed term with standard regularisation techniques and exploiting a standard data fidelity term, we obtain an unsupervised loss function for training deep neural networks, which allows us to achieve real-time hyperspectral image demosaicking.ResultsQuantitative results on hyperspetral image datasets show that our unsupervised demosaicking approach can achieve similar performance to its supervised counter-part, and significantly outperform linear demosaicking. A qualitative user study on real snapshot hyperspectral surgical images confirms the results from the quantitative analysis.ConclusionOur results suggest that the proposed unsupervised algorithm can achieve promising hyperspectral demosaicking in real-time thus advancing the suitability of the modality for intraoperative use.

Artificial-intelligence-assisted mass fabrication of nanocantilevers from randomly positioned single carbon nanotubes

Nanoscale cantilevers (nanocantilevers) made from carbon nanotubes (CNTs) provide tremendous benefits in sensing and electromagnetic applications. This nanoscale structure is generally fabricated using chemical vapor deposition and/or dielectrophoresis, which contain manual, time-consuming processes such as the placing of additional electrodes and careful observation of single-grown CNTs. Here, we demonstrate a simple and Artificial Intelligence (AI)-assisted method for the efficient fabrication of a massive CNT-based nanocantilever. We used randomly positioned single CNTs on the substrate. The trained deep neural network recognizes the CNTs, measures their positions, and determines the edge of the CNT on which an electrode should be clamped to form a nanocantilever. Our experiments demonstrate that the recognition and measurement processes are automatically completed in 2 s, whereas comparable manual processing requires 12 h. Notwithstanding the small measurement error by the trained network (within 200 nm for 90% of the recognized CNTs), more than 34 nanocantilevers were successfully fabricated in one process. Such high accuracy contributes to the development of a massive field emitter using the CNT-based nanocantilever, in which the output current is obtained with a low applied voltage. We further showed the benefit of fabricating massive CNT-nanocantilever-based field emitters for neuromorphic computing. The activation function, which is a key function in a neural network, was physically realized using an individual CNT-based field emitter. The introduced neural network with the CNT-based field emitters recognized handwritten images successfully. We believe that our method can accelerate the research and development of CNT-based nanocantilevers for realizing promising future applications.

Optimized deep neural network based DDoS attack detection and bait mitigation process in software defined network

SummaryThe software‐defined network (SDN) is a new network design with an operating system that allows better network quality control. The controller's primary role in an SDN network is to divide the control and forwarding planes in order to provide essential network power. The backup controller in an SDN may confront a variety of obstacles during a DDoS attack. It disrupts the flow of the network by attacking the service nodes hence obstructing the legitimate users from getting service. To overcome these issues, this work introduces a DDoS attack detection model that will aid in removing the attacker from the network. Initially, the input data is sent into the detection phase, which identifies the existence and kind of attacks. The presence of an attack is determined based on the data flow, and statistical features. With the reference of the extracted features, the optimized deep neural network (DNN) will decide the existence of attacker in the network. To make the decision more precise, the training of DNN will be carried out by self‐improved moth flame optimization (SIMFO) Algorithm via tuning the optimal weights. Once an attacker's presence has been identified, it is critical to remove the attacker's node from the network.

Nonstationary Time Series Prediction Based on Deep Echo State Network Tuned by Bayesian Optimization

The predictions from time series data can help us sense development trends and make scientific decisions in advance. The commonly used forecasting methods with backpropagation consume a lot of computational resources. The deep echo state network (DeepESN) is an advanced prediction method with a deep neural network structure and training algorithm without backpropagation. In this paper, a Bayesian optimization algorithm (BOA) is proposed to optimize DeepESN to address the problem of increasing parameter scale. Firstly, the DeepESN was studied and constructed as the basic prediction model for the time series data. Secondly, the BOA was reconstructed, based on the DeepESN, for optimal parameter searching. The algorithm is proposed within the framework of the DeepESN. Thirdly, an experiment was conducted to verify the DeepESN with a BOA within three datasets: simulation data generated from computer programs, a real humidity dataset collected from Beijing, and a power load dataset obtained from America. Compared with the models of BP (backpropagation), LSTM (long short-term memory), GRU (gated recurrent unit), and ESN (echo state network), DeepESN obtained optimal results, which were 0.0719, 18.6707, and 764.5281 using RMSE evaluation. While getting better accuracy, the BOA optimization time was only 323.4 s, 563.2 s, and 9854 s for the three datasets. It is more efficient than grid search and grey wolf optimizer.

MediNet: transfer learning approach with MediNet medical visual database.

The rapid development of machine learning has increased interest in the use of deep learning methods in medical research. Deep learning in the medical field is used in disease detection and classification problems in the clinical decision-making process. Large amounts of labeled datasets are often required to train deep neural networks; however, in the medical field, the lack of a sufficient number of images in datasets and the difficulties encountered during data collection are among the main problems. In this study, we propose MediNet, a new 10-class visual dataset consisting of Rontgen (X-ray), Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound, and Histopathological images such as calcaneal normal, calcaneal tumor, colon benign colon adenocarcinoma, brain normal, brain tumor, breast benign, breast malignant, chest normal, chest pneumonia. AlexNet, VGG19-BN, Inception V3, DenseNet 121, ResNet 101, EfficientNet B0, Nested-LSTM + CNN, and proposed RdiNet deep learning algorithms are used in the transfer learning for pre-training and classification application. Transfer learning aims to apply previously learned knowledge in a new task. Seven algorithms were trained with the MediNet dataset, and the models obtained from these algorithms, namely feature vectors, were recorded. Pre-training models were used for classification studies on chest X-ray images, diabetic retinopathy, and Covid-19 datasets with the transfer learning technique. In performance measurement, an accuracy of 94.84% was obtained in the traditional classification study for the InceptionV3 model in the classification study performed on the Chest X-Ray Images dataset, and the accuracy was increased 98.71% after the transfer learning technique was applied. In the Covid-19 dataset, the classification success of the DenseNet121 model before pre-trained was 88%, while the performance after the transfer application with MediNet was 92%. In the Diabetic retinopathy dataset, the classification success of the Nested-LSTM + CNN model before pre-trained was 79.35%, while the classification success was 81.52% after the transfer application with MediNet. The comparison of results obtained from experimental studies observed that the proposed method produced more successful results.

Extraction of Interconnect Parasitic Capacitance Matrix Based on Deep Neural Network

Interconnect parasitic capacitance extraction is crucial in analyzing VLSI circuits’ delay and crosstalk. This paper uses the deep neural network (DNN) to predict the parasitic capacitance matrix of a two-dimensional pattern. To save the DNN training time, the neural network’s output includes only coupling capacitances in the matrix, and total capacitances are obtained by summing corresponding predicted coupling capacitances. In this way, we can obtain coupling and total capacitances simultaneously using a single neural network. Moreover, we introduce a mirror flip method to augment the datasets computed by the finite element method (FEM), which doubles the dataset size and reduces data preparation efforts. Then, we compare the prediction accuracy of DNN with another neural network ResNet. The result shows that DNN performs better in this case. Moreover, to verify our method’s efficiency, the total capacitances calculated from the trained DNN are compared with the network (named DNN-2) that takes the total capacitance as an extra output. The results show that the prediction accuracy of the two methods is very close, indicating that our method is reliable and can save the training workload for the total capacitance. Finally, a solving efficiency comparison shows that the average computation time of the trained DNN for one case is not more than 2% of that of FEM.

A rule-based deep fuzzy system with nonlinear fuzzy feature transform for data classification

In today's fuzzy community, the blending of fuzzy model and deep learning has become one hot topic for the development of more sophisticated and high-powered fuzzy systems. In this study, a rule-based deep fuzzy system with nonlinear fuzzy feature transform for data classification named by NFFT-DFSC is proposed, borrowing the “hierarchically stacked thought” originated from deep learning. Firstly, two fuzzy models with nonlinear fuzzy feature transform function and with decision-making function, and their neuro-fuzzy network implementations are proposed. Subsequently, we stack multiple fuzzy models with nonlinear fuzzy feature transform function and the fuzzy model with decision-making function together in a cascade way, which results in the formation of NFFT-DFSC. Here the stacked fuzzy models with nonlinear fuzzy feature transform function snoops and transforms features of raw data continuously to finally obtain the high-level latent fuzzy features, while the fuzzy model with decision-making function completes classification by exploring the multiple prototypes used to represent the high-level latent fuzzy features of each class. Besides, batch normalization operation and mini-batch gradient descent optimization rooted from the training of deep neural networks are also used to enhance the generalization of NFFT-DFSC and learn its parameters in end-to-end manner, respectively. Experiments between NFFT-DFSCs with three nonlinear mapping functions and 15 benchmark classifiers involving gaussian kernel support vector machine (GSVM), Takagi-Sugeno-Kang classifier (TSK), deep neural network (DNN) and state-of-the-art deep TSK ones on synthetic datasets and nineteen diverse real-world datasets demonstrate that the proposed NFFT-DFSC is a more competitive classifier on classification accuracy.

Deep learning for the screening of primary ciliary dyskinesia based on cranial computed tomography.

Objective: To analyze the cranial computed tomography (CT) imaging features of patients with primary ciliary dyskinesia (PCD) who have exudative otitis media (OME) and sinusitis using a deep learning model for early intervention in PCD. Methods: Thirty-two children with PCD diagnosed at the Children's Hospital of Fudan University, Shanghai, China, between January 2010 and January 2021 who had undergone cranial CT were retrospectively analyzed. Thirty-two children with OME and sinusitis diagnosed using cranial CT formed the control group. Multiple deep learning neural network training models based on PyTorch were built, and the optimal model was trained and selected to observe the differences between the cranial CT images of patients with PCD and those of general patients and to screen patients with PCD. Results: The Swin-Transformer, ConvNeXt, and GoogLeNet training models had optimal results, with an accuracy of approximately 0.94; VGG11, VGG16, VGG19, ResNet 34, and ResNet 50, which are neural network models with fewer layers, achieved relatively strong results; and Transformer and other neural networks with more layers or neural network models with larger receptive fields exhibited a relatively weak performance. A heat map revealed the differences in the sinus, middle ear mastoid, and fourth ventricle between the patients with PCD and the control group. Transfer learning can improve the modeling effect of neural networks. Conclusion: Deep learning-based CT imaging models can accurately screen for PCD and identify differences between the cranial CT images.

Data-driven stress and strain curves of the unidirectional composites by deep neural networks with principal component analysis and selective-data augmentation

This study proposes a data-driven approach using a deep neural network (DNN) to efficiently predict the stress–strain (S-S) curves for unidirectional composites. Firstly, a representative volume element (RVE), including arbitrary fiber distribution of circular fibers, was generated. Then, the S-S curves for each RVE were obtained from finite element simulation considering the interfacial debonding phenomenon. Next, input and output features were chosen in terms of the center positions of fibers and S-S curves, respectively, for DNN training. In addition, to learn the S-S curves efficiently in a lower-dimensional space, principal component analysis (PCA) was employed. Subsequently, the data-driven model combining PCA and DNN was developed; this quickly and accurately predicted the S-S curves with a relative error for the toughness of about 2 %. Furthermore, selective-data augmentation is proposed to improve the prediction accuracy in insufficient and nongeneralized datasets, leading to the higher prediction accuracy of ultimate tensile strength and toughness.

GenSMILES: An enhanced validity conscious representation for inverse design of molecules

Deep neural networks have become increasingly important in recent years for creating molecules with desirable properties. In general, SMILES strings are used to train deep neural network based models. The trained model is then used to generate the desired molecules. Unfortunately, due to syntactical and semantic flaws in the representation, the SMILES string generates a substantial number of invalid molecules. SMILES representation fails to efficiently handle rings, branch and bonds between atoms. Lack of robustness in dealing with the cited aspects results in abundance of invalid strings. To overcome this limitation, this paper proposes a SMILES like representation, called GenSMILES. GenSMILES tackles syntactical and semantic issues by relying upon derivative rules to apply constraints. This causes a generative model produces more valid SMILES strings. By substituting a single notation for the pair representation of branches and rings in SMILES with a ) and ^, respectively, GenSMILES corrects the syntactical issues. The mismatching of atom’s bonds is the main cause of semantic errors. GenSMILES addresses such issues by employing derivation rules during string conversion from GenSMILES to SMILES. Every SMILES string can be represented with an equivalent GenSMILES. When used for designing drug molecule, GenSMILES increases molecule’s validity when compared with SMILES and DeepSMILES on two popular architectures i.e., Recurrent Neural Network and Variational Autoencoder. The main benefit of GenSMILES is that it can be applied directly to generative algorithms without adapting the model environment. GenSMILES is beneficial not only to generative approaches of DL but also to the approaches that use SMILES string-like representation. On most of the datasets, GenSMILES is effective in improving validity above 90% and diversity score 15. GenSMILES results in more diversity in the properties of generated molecules and allows exploration of larger portion of undiscovered chemical space as compared to SMILES and DeepSMILES. GenSMILES guarantee that the generative model does not need to remember any long dependencies and is principal contribution of this work.

On exploring weakly supervised domain adaptation strategies for semantic segmentation using synthetic data

Pixel-wise image segmentation is key for many Computer Vision applications. The training of deep neural networks for this task has expensive pixel-level annotation requirements, thus, motivating a growing interest on synthetic data to provide unlimited data and its annotations. In this paper, we focus on the generation and application of synthetic data as representative training corpuses for semantic segmentation of urban scenes. First, we propose a synthetic data generation protocol, which identifies key features affecting performance and provides datasets with variable complexity. Second, we adapt two popular weakly supervised domain adaptation approaches (combined training, fine-tuning) to employ synthetic and real data. Moreover, we analyze several backbone models, real/synthetic datasets and their proportions when combined. Third, we propose a new curriculum learning strategy to employ several synthetic and real datasets. Our major findings suggest the high performance impact of pace and order of synthetic and real data presentation, achieving state of the art results for well-known models. The results by training with the proposed dataset outperform popular alternatives, thus demonstrating the effectiveness of the proposed protocol. Our code and dataset are available at http://www-vpu.eps.uam.es/publications/WSDA_semantic/

An Efficient Optimization Technique for Training Deep Neural Networks

Deep learning is a sub-branch of artificial intelligence that acquires knowledge by training a neural network. It has many applications in the field of banking, automobile industry, agriculture, and healthcare industry. Deep learning has played a significant role in solving complex tasks related to computer vision, such as image classification, natural language processing, and object detection. On the other hand, optimizers also play an intrinsic role in training the deep learning model. Recent studies have proposed many deep learning models, such as VGG, ResNet, DenseNet, and ImageNet. In addition, there are many optimizers such as stochastic gradient descent (SGD), Adam, AdaDelta, Adabelief, and AdaMax. In this study, we have selected those models that require lower hardware requirements and shorter training times, which facilitates the overall training process. We have modified the Adam based optimizers and minimized the cyclic path. We have removed an additional hyper-parameter from RMSProp and observed that the optimizer works with various models. The learning rate is set to minimum and constant. The initial weights are updated after each epoch, which helps to improve the accuracy of the model. We also changed the position of the epsilon in the default Adam optimizer. By changing the position of the epsilon, it accumulates the updating process. We used various models with SGD, Adam, RMSProp, and the proposed optimization technique. The results indicate that the proposed method is effective in achieving the accuracy and works well with the state-of-the-art architectures.