Random Initialization Research Articles

Encryption Modes Identification of Block Ciphers based on Machine Learning

Encryption modes affect the security of block ciphers. This paper proposes a new approach for identifying encryption modes based on machine learning and feature engineering. In the conditions of random keys and initialization vectors, five encryption modes are used for identification. Each mode is encrypted by several block ciphers. By comparing with previous work, we have overcome the shortcomings and improved the accuracy of identification by about 30% to 40%. The experiments improve the existing results and can effectively help cryptanalysts recover the keys.

ECKM: An improved K-means clustering based on computational geometry

A modified version of traditional k-means clustering algorithm applying computational geometry for initialization of cluster centers has been presented in this paper. It is well known that the quality of k-means clustering depends on its random initialization of centers. This paper shows that when the initial cluster centers are selected from the list of circumference points of the empty circles sorted in non-increasing order of their corresponding radii, it can significantly improve the performance. The proposed algorithm is named as Empty Circles based k-means (ECKM) Clustering, which was successfully implemented with PYTHON. Extensive experimentation was carried out on various benchmark data sets that includes both artificial and real data sets having different shapes, sizes and features. In order to minimize the size of this paper, we presented the results for seven of such benchmark datasets, viz., iris, wine, hepta, flame compound, breast_cancer and DS577 although in reality we experimented for more than fifteen data sets in order to validate our ECKM. Results establish the superiority of the ECKM clustering over standard techniques.

A Spatial Temporal Classification Analysis And Visualization Of Tropical Cyclone Tracks In Bay Of Bengal Using GIS

Tropical cyclones (TC) are among the most devastating forms of natural hazards and the east coast of India is more prone to TC landfall causing significant socio-economic impacts. The Bay of Bengal (BoB) which forms the eastern sub basin of North Indian Ocean experiences the seasonally reversing monsoon, depression and TCs. In this study TC best track dataset of NIO basin over the period 1960 - 2016 from the IBTrACKs archive maintained by NOAA are used. In this work Firefly optimization is coupled with FCM for TC tracks classification. The classical FCM uses random initialization of cluster centroid often gets trapped in local optimal problem. The firefly algorithm is applied on the FCM for the cluster centroid computation, in this way improving the efficiency of FCM algorithm. The obtained classes are then projected in the visualization space. Visualizations are generated using the GIS environment to gain insight into the spatial distribution of TC tracks over decades. This study aims to develop a comprehensive assessment of variability in tropical cyclones with respect to ENSO modulated events, inter decadal variability and track sinuosity. In this paper we attempt to convey the cognitive results of comparative visualizations of TC tracks over Arabian Sea and Bay of Bengal sub basin during the strong, very strong El Niño and La Niña events. Finally we use Parallel Coordinate Plot (PCP) a visualization technique to demonstrate the correlation patterns of the TC parameters.

Sparse Gaussian processes for multi-step motion prediction of space tumbling objects

Since the robotic capturing of a space tumbling object requires a rapidly accurate prediction of the object motion, this paper addresses the nonlinear multi-step motion prediction issue of the object based on machine learning theory and proposes a novel prediction approach, in which the prediction efficiency and the prediction accuracy are main focuses. To improve the prediction efficiency, sparse Gaussian processes are employed for the multi-step motion prediction, only providing the historical data of the object motion states. The sparse pseudo training data continuously optimized from real historical data makes it feasible for real-time predictions. To improve the prediction accuracy, a heuristic optimizer, Markov chain Monte Carlo, is integrated within sparse Gaussian processes to overcome the sensitivity problem caused by random initializations during the training process, thus avoids the local optimal training results. Then, a high-quality sparse pseudo training data is obtained to construct the predictive distributions during the testing process. The benchmark example and comparisons of several motion prediction applications illustrate the correctness and effectiveness of the proposed approach, and the experimental validation demonstrates its real-world potential.

A convergence analysis of Nesterov’s accelerated gradient method in training deep linear neural networks

As the training process of deep neural networks involves expensive computational cost, speeding up the convergence is of great importance. Nesterov’s accelerated gradient (NAG) is one of the most popular accelerated optimizers in the deep learning community, which often exhibits improved convergence performance over gradient descent (GD) in practice. However, theoretical investigations of NAG mainly focus on the convex setting. Since the optimization landscape of the neural network is non-convex, little is known about the convergence and acceleration of NAG. Nowadays, some works make progress towards understanding the convergence of NAG in training over-parameterized neural networks, where the number of the parameters exceeds that of the training instances. Nonetheless, previous studies are limited to the two-layer neural network, which are far from explaining the remarkable success of NAG in optimizing deep neural networks. In this paper, we investigate the convergence of NAG in training two architectures of deep linear networks: deep fully-connected linear neural networks and deep linear ResNets. Based on the over-parameterization regime, we first analyze the residual dynamics induced by the training trajectory of NAG for a deep fully-connected linear neural network under random Gaussian initialization. Our results show that NAG can converge to the global minimum at a (1-O(1/κ))t rate when the width is near-linear in the depth of the network, where t is the number of iterations and κ>1 is a constant depending on the condition number of the feature matrix. Compared to the (1-O(1/κ))t rate of GD, NAG achieves an acceleration over GD. For deep linear ResNets, we utilize the same analytical approach and obtain a similar convergence result, while the width requirement is independent of the depth. To the best of our knowledge, these are the first theoretical guarantees for the convergence and acceleration of NAG in training deep neural networks. Numerical results show the acceleration of NAG compared to GD in terms of iterations. In addition, we conduct experiments to evaluate the effect of the depth on the convergence rate of NAG, which validate our derived conditions of the width. We hope our results may shed light on understanding the optimization behavior of NAG for modern deep neural networks.

NDPD: an improved initial centroid method of partitional clustering for big data mining

PurposeThe K-means (KM) clustering algorithm is extremely responsive to the selection of initial centroids since the initial centroid of clusters determines computational effectiveness, efficiency and local optima issues. Numerous initialization strategies are to overcome these problems through the random and deterministic selection of initial centroids. The random initialization strategy suffers from local optimization issues with the worst clustering performance, while the deterministic initialization strategy achieves high computational cost. Big data clustering aims to reduce computation costs and improve cluster efficiency. The objective of this study is to achieve a better initial centroid for big data clustering on business management data without using random and deterministic initialization that avoids local optima and improves clustering efficiency with effectiveness in terms of cluster quality, computation cost, data comparisons and iterations on a single machine.Design/methodology/approachThis study presents the Normal Distribution Probability Density (NDPD) algorithm for big data clustering on a single machine to solve business management-related clustering issues. The NDPDKM algorithm resolves the KM clustering problem by probability density of each data point. The NDPDKM algorithm first identifies the most probable density data points by using the mean and standard deviation of the datasets through normal probability density. Thereafter, the NDPDKM determines K initial centroid by using sorting and linear systematic sampling heuristics.FindingsThe performance of the proposed algorithm is compared with KM, KM++, Var-Part, Murat-KM, Mean-KM and Sort-KM algorithms through Davies Bouldin score, Silhouette coefficient, SD Validity, S_Dbw Validity, Number of Iterations and CPU time validation indices on eight real business datasets. The experimental evaluation demonstrates that the NDPDKM algorithm reduces iterations, local optima, computing costs, and improves cluster performance, effectiveness, efficiency with stable convergence as compared to other algorithms. The NDPDKM algorithm minimizes the average computing time up to 34.83%, 90.28%, 71.83%, 92.67%, 69.53% and 76.03%, and reduces the average iterations up to 40.32%, 44.06%, 32.02%, 62.78%, 19.07% and 36.74% with reference to KM, KM++, Var-Part, Murat-KM, Mean-KM and Sort-KM algorithms.Originality/valueThe KM algorithm is the most widely used partitional clustering approach in data mining techniques that extract hidden knowledge, patterns and trends for decision-making strategies in business data. Business analytics is one of the applications of big data clustering where KM clustering is useful for the various subcategories of business analytics such as customer segmentation analysis, employee salary and performance analysis, document searching, delivery optimization, discount and offer analysis, chaplain management, manufacturing analysis, productivity analysis, specialized employee and investor searching and other decision-making strategies in business.

Pneumonia Detection in Chest X-Ray Images Using Enhanced Restricted Boltzmann Machine.

The process of pneumonia detection has been the focus of researchers as it has proved itself to be one of the most dangerous and life-threatening disorders. In recent years, many machine learning and deep learning algorithms have been applied in an attempt to automate this process but none of them has been successful significantly to achieve the highest possible accuracy. In a similar attempt, we propose an enhanced approach of a deep learning model called restricted Boltzmann machine (RBM) which is named enhanced RBM (ERBM). One of the major drawbacks associated with the standard format of RBM is its random weight initialization which leads to improper feature learning of the model during the training phase, resulting in poor performance of the machine. This problem has been tried to eliminate in this work by finding the differences between the means of a specific feature vector and the means of all features given as inputs to the machine. By performing this process, the reconstruction of the actual features is increased which ultimately reduces the error generated during the training phase of the model. The developed model has been applied to three different datasets of pneumonia diseases and the results have been compared with other state of the art techniques using different performance evaluation parameters. The proposed model gave highest accuracy of 98.56% followed by standard RBM, SVM, KNN, and decision tree which gave accuracies of 97.53%, 92.62%, 91.64%, and 88.77%, respectively, for dataset named dataset 2. Similarly, for the dataset 1, the highest accuracy of 96.66 has been observed for the eRBM followed by srRBM, KNN, decision tree, and SVM which gave accuracies of 90.22%, 89.34%, 87.65%, and 86.55%, respectively. In the same way, the accuracies observed for the dataset 3 by eRBM, standard RBM, KNN, decision tree, and SVM are 92.45%, 90.98%, 87.54%, 85.49%, and 84.54%, respectively. Similar observations can also be seen for other performance parameters showing the efficiency of the proposed model. As revealed in the results obtained, a significant improvement has been observed in the working of the RBM by introducing a new method of weight initialization during the training phase. The results show that the improved model outperforms other models in terms of different performance evaluation parameters, namely, accuracy, sensitivity, specificity, F1-score, and ROC curve.

A Dynamic Colored Traveling Salesman Problem With Varying Edge Weights

A colored traveling salesman problem (CTSP) is a generalization of the well-known multiple traveling salesman problem. In it, each city has one to multiple colors and allows a salesman in the same color to visit exactly once. This work presents for the first time a CTSP whose edge weights among the cities change over time. It can be applied to dynamic routing problems arising in logistic distribution systems with various goods accessibilities to different types of vehicles. A non-linear integer mathematical program is constructed. A Variable Neighborhood Search (VNS) algorithm with a direct-route encoding and random initialization is then presented to solve it. To increase the convergence rate and population diversity of VNS, it is further combined with two-stage greedy initialization and an appropriate population immigrant scheme to perform the population search in a dynamic environment. Then, off-line performance evaluation and Wilcoxon test of the proposed algorithms are performed. The results show that their solution quality is improved by 30% ~ 60% over the basic VNS’s. They can track the environmental changes of CTSP-VEW more rapidly and effectively. In the end, a case study with a practical scenario is conducted.

Principal coefficient encoding for subject-independent human activity analysis

Tracking human physical activity using smartphones is an emerging trend in healthcare monitoring and healthy lifestyle management. Neural networks are broadly used to analyze the inertial data of activity recognition. Inspired by the autoencoder neural networks, we propose a layer-wise network, namely principal coefficient encoder model (PCEM). Unlike the vanilla neural networks which apply random weight initialization andback-propagation for parameter updating, an optimized weight initialization is implemented in PCEM via principal coefficient learning. This principal coefficient encoding allows rapid data learning with no back-propagation intervention and no gigantic hyperparameter tuning. In PCEM, the most principal coefficients of the training data are determined to be the network weights. Two hidden layers with principal coefficient encoding are stacked in PCEM for the sake of deep architecture design. The performance of PCEM is evaluated based on a subject-independent protocol where training and testing samples are from different users, with no overlapping subjects in between the training and testing sets. This subject-independent protocol can better assess the generalization of the model to new data. Experimental results exhibit that PCEM outperforms certain state-of-the-art machine learning and deep learning models, including convolutional neural network, and deep belief network. PCEM can achieve ~97% accuracy in subject-independent human activity analysis.

Research on Human Resource Management Performance Evaluation Method Based on Chaos Optimization Algorithm

All enterprise components and business activities are centered on the customer. Whether human resource management is reasonable and effective not only is a matter of human resource management, but also directly influences whether other resources can be used reasonably and effectively, and determines the efficiency of business operations. In human resource management, perfect performance evaluation criteria are developed based on the actual situation in order to evaluate employee productivity and work ethic. Nonetheless, in the process of actual enterprise HR management performance evaluation work, there are not only imperfect performance evaluation standards, but also relatively objective evaluation standards, and HR cannot conduct a comprehensive analysis of each position, which has a negative impact on the quality of enterprise HR management work. This paper improves the performance evaluation model for the original data mining and proposes a human resource management performance evaluation method based on chaotic optimization algorithm, which generates the initial values of evaluation data through chaotic logistic mapping, bringing the initial data closer to the optimal value, reducing the impact of random initialization on the algorithm’s performance, and when the model is trained, it can make the algorithm’s performance more stable. Capability thereby addresses the flaws in the original evaluation method. Experiments demonstrate that our method significantly improves the precision of model training and prediction.

A faster dynamic convergency approach for self-organizing maps

This paper proposes a novel variable learning rate to address two main challenges of the conventional Self-Organizing Maps (SOM) termed VLRSOM: high accuracy with fast convergence and low topological error. We empirically showed that the proposed method exhibits faster convergence behavior. It is also more robust in topology preservation as it maintains an optimal topology until the end of the maximum iterations. Since the learning rate adaption and the misadjustment parameter depends on the calculated error, the VLRSOM will avoid the undesired results by exploiting the error response during the weight updation. Then the learning rate is updated adaptively after the random initialization at the beginning of the training process. Experimental results show that it eliminates the tradeoff between the rate of convergence and accuracy and maintains the data's topological relationship. Extensive experiments were conducted on different types of datasets to evaluate the performance of the proposed method. First, we experimented with synthetic data and handwritten digits. For each data set, two experiments with a different number of iterations (200 and 500) were performed to test the stability of the network. The proposed method was further evaluated using four benchmark data sets. These datasets include Balance, Wisconsin Breast, Dermatology, and Ionosphere. In addition, a comprehensive comparative analysis was performed between the proposed method and three other SOM techniques: conventional SOM, parameter-less self-organizing map (PLSOM2), and RA-SOM in terms of accuracy, quantization error (QE), and topology error (TE). The results indicated the proposed approach produced superior results to the other three methods.

RadImageNet: An Open Radiologic Deep Learning Research Dataset for Effective Transfer Learning.

PurposeTo demonstrate the value of pretraining with millions of radiologic images compared with ImageNet photographic images on downstream medical applications when using transfer learning.Materials and MethodsThis retrospective study included patients who underwent a radiologic study between 2005 and 2020 at an outpatient imaging facility. Key images and associated labels from the studies were retrospectively extracted from the original study interpretation. These images were used for RadImageNet model training with random weight initiation. The RadImageNet models were compared with ImageNet models using the area under the receiver operating characteristic curve (AUC) for eight classification tasks and using Dice scores for two segmentation problems.ResultsThe RadImageNet database consists of 1.35 million annotated medical images in 131 872 patients who underwent CT, MRI, and US for musculoskeletal, neurologic, oncologic, gastrointestinal, endocrine, abdominal, and pulmonary pathologic conditions. For transfer learning tasks on small datasets—thyroid nodules (US), breast masses (US), anterior cruciate ligament injuries (MRI), and meniscal tears (MRI)—the RadImageNet models demonstrated a significant advantage (P < .001) to ImageNet models (9.4%, 4.0%, 4.8%, and 4.5% AUC improvements, respectively). For larger datasets—pneumonia (chest radiography), COVID-19 (CT), SARS-CoV-2 (CT), and intracranial hemorrhage (CT)—the RadImageNet models also illustrated improved AUC (P < .001) by 1.9%, 6.1%, 1.7%, and 0.9%, respectively. Additionally, lesion localizations of the RadImageNet models were improved by 64.6% and 16.4% on thyroid and breast US datasets, respectively.ConclusionRadImageNet pretrained models demonstrated better interpretability compared with ImageNet models, especially for smaller radiologic datasets.Keywords: CT, MR Imaging, US, Head/Neck, Thorax, Brain/Brain Stem, Evidence-based Medicine, Computer Applications–General (Informatics) Supplemental material is available for this article. Published under a CC BY 4.0 license.See also the commentary by Cadrin-Chênevert in this issue.

SELF-ORGANIZING RESERVOIR NETWORK FOR ACTION RECOGNITION

Current research in human action recognition (HAR) focuses on efficient and effective modelling of the temporal features of human actions in 3-dimensional space. Echo State Networks (ESNs) are one suitable method for encoding the temporal context due to its short-term memory property. However, the random initialization of the ESN's input and reservoir weights may increase instability and variance in generalization. Inspired by the notion that input-dependent self-organization is decisive for the cortex to adjust the neurons according to the distribution of the inputs, a Self-Organizing Reservoir Network (SORN) is developed based on Adaptive Resonance Theory (ART) and Instantaneous Topological Mapping (ITM) as the clustering process to cater deterministic initialization of the ESN reservoirs in a Convolutional Echo State Network (ConvESN) and yield a Self-Organizing Convolutional Echo State Network (SO-ConvESN). SORN ensures that the activation of ESN’s internal echo state representations reflects similar topological qualities of the input signal which should yield a self-organizing reservoir. In the context of HAR task, human actions encoded as a multivariate time series signals are clustered into clustered node centroids and interconnectivity matrices by SORN for initializing the SO-ConvESN reservoirs. By using several publicly available 3D-skeleton-based action recognition datasets, the impact of vigilance threshold and reservoir perturbation of SORN in performing clustering, the SORN reservoir dynamics and the capability of SO-ConvESN on HAR task have been empirically evaluated and analyzed to produce competitive experimental results.

To Estimate Performance of Artificial Neural Network Model Based on Terahertz Spectrum: Gelatin Identification as an Example.

It is a necessity to determine significant food or traditional Chinese medicine (TCM) with low cost, which is more likely to achieve high accurate identification by THz-TDS. In this study, feedforward neural networks based on terahertz spectra are employed to predict the animal origin of gelatins, whose adaption to the mission is examined by parallel models built by random sample partition and initialization. It is found that the generalization performance of feedforward ANNs in original data is not satisfactory although prediction on trained samples can be accurate. A multivariate scattering correction is conducted to enhance prediction accuracy, and 20 additional models verify the effectiveness of such dispose. A special partition of total dataset is conducted based on statistics of parallel models, whose influence on ANN performance is investigated with another 20 models. The performance of the models is unsatisfactory because of notable differences in training and test sets according to principal component analysis. By comparing the distribution of the first two principal components before and after multivariate scattering correction, we found that the reciprocal of the minimum number of line segments required for error-free classification in 2-D feature space can be viewed as an index to describe linear separability of data. The rise of proposed linear separability would have a lower requirement for harsh parameter tuning of ANN models and tolerate random initialization. The difference in principal components of samples between a training set and a data set determines whether partition is acceptable or whether a model would have generality. A rapid way to estimate the performance of an ANN before sufficient tuning on a classification mission is to compare differences between groups and differences within groups. Given that a representative peak missing curve is discussed in this article, an analysis based on gelatin THz spectra may be helpful for studies on some other feature-less species.

Accelerating and improving radiochromic film calibration by utilizing the dose ratio in photon and proton beams.

PurposeRadiochromic films are versatile 2D dosimeters with high‐resolution and near tissue equivalence. To assure high precision and accuracy, a time‐consuming calibration process is required. To improve the time efficiency, a novel calibration method utilizing the ratio of the same dose profile measured at different monitor units (MUs) is introduced and tested in a proton and photon beam.MethodsThe calibration procedure employs the dose ratio of film measurements of the same relative profile for different absolute dose values. Hence, the ratio of the dose is constant at any point of the profile, but the ratio of the net optical densities is not constant. The key idea of the method is to optimize the calibration function until the ratio of the calculated doses is constant. The proposed method was tested in the dose range between 0.25–12 and 1–6 Gy in a proton and photon beam, respectively. A radial symmetric profile and a rectangular profile were created, both having a central plateau region of about 3 cm diameter and a dose falloff of about 1.5 cm at larger distances. The dose falloff region was used as input for the optimization method and the central plateau region served as dose reference points. Only the plateau region of the highest dose entered the optimization as an additional objective. The measured data were randomly split into differently sized training and test sets. The optimization was repeated 1000 times with random start value initialization using the same start values for the standard and the gradient method. Finally, a proton plan with four dose levels was created, which were separated spatially, to test the possibility of a full calibration within a single measurement.ResultsParameter estimation was possible with as low as one dose ratio used for optimization in both the photon and the proton case, yet exhibiting a high sensitivity on the dose level. The root mean squared deviation (RMSD) of the dose was less than 1% when the dose ratio was in the order of 20, whereas the median RMSD of all optimizations was 1.7%. Using four dose levels for optimization resulted in a median RMSD of 1% when randomly selecting the dose levels. Having at least one dose ratio of about 20 included in the optimization considerably improved the RMSD of the calibration function. Using six or eight dose levels reduced the sensitivity on the dose level selection and the median RMSD was 0.8%. A full calibration was possible in a single measurement having four dose levels in one plan but spatially separated.ConclusionsThe number of measurements required to obtain an EBT3 film calibration function could be reduced using the proposed dose ratio method while maintaining the same accuracy as with the standard method.

Accelerating and improving deep reinforcement learning-based active flow control: Transfer training of policy network

Deep reinforcement learning (DRL) has gradually emerged as an effective and novel method to achieve active flow control with outstanding performance. This paper focuses on exploring the strategy of improving learning efficiency and control performance of a new task using existing control experience. More specifically, the proximal policy optimization algorithm is used to control the flow past a circular cylinder using jets. The DRL controllers trained from the initialized parameter are able to obtain drag reductions of 8%, 18.7%, 18.4%, and 25.2%, at Re = 100, 200, 300, and 1000, respectively, and it takes more episodes to converge for the cases with higher Reynolds number, due to the increased flow complexity. Furthermore, the agent trained at high Reynolds number shows satisfied control performance when it is applied to the lower Reynolds number cases, which proves a strong correlation between the control policy and the flow patterns between the flows under different conditions. To better utilize the experience of the control policy of the trained agent, the flow control tasks with Re = 200, 300, and 1000 are retrained, based on the trained agent at Re = 100, 200, and 300, respectively. Our results show that a dramatic enhancement of the learning efficiency can be achieved; that is, the number of the training episodes reduces to be less than 20% of the agents trained with random initialization. Moreover, for each flow condition drag reduction approaches a significant level of 20.9%, 27.7%, and 36.1%, respectively. The great performance of the transfer training method of the DRL agent shows its potential on economizing the training cost and improving control effectiveness, especially for complex control tasks.

Activation function design for deep networks: linearity and effective initialisation

The activation function deployed in a deep neural network has great influence on the performance of the network at initialisation, which in turn has implications for training. In this paper we study how to avoid two problems at initialisation identified in prior works: rapid convergence of pairwise input correlations, and vanishing and exploding gradients. We prove that both these problems can be avoided by choosing an activation function possessing a sufficiently large linear region around the origin, relative to the bias variance σ b 2 of the network's random initialisation. We demonstrate empirically that using such activation functions leads to tangible benefits in practice, both in terms of test and training accuracy and in terms of training time. Furthermore, we observe that the shape of the nonlinear activation outside the linear region appears to have a relatively limited impact on training. Finally, our results also allow us to train networks in a new hyperparameter regime, with a much larger bias variance than has previously been possible.

A Link Prediction Algorithm Based on GAN

Link prediction, as an important research direction in complicated network analysis, has broad application prospects. However, traditional link prediction algorithms are generally designed by the sparse expression of the adjacency matrix, which is computationally expensive and inefficient, being also unable to run on large-scale networks and to preserve their higher order structural features. To fill this gap, we propose a GAN (generative adversarial network)-based link prediction algorithm. The algorithm layers the network graph, preserving the local features and higher-level structural features of the original network graph, and uses a generative adversarial model to recursively and backwardly obtain the low-dimensional vector form of the vertices in each layer of the network graph as the initialization of the network graph in the previous layer. It then obtains the low-dimensional vector form of all the vertices in the original network graph for link prediction, and the problem of local minima that can be generated by random initialization is solved. The experimental results show that our method is superior to many state-of-the-art algorithms.

Evaluation for Development Effect of Enterprise Innovation with Neural Network from Low-Carbon Economy

China has resolved to pay more and more attention to the sustainability of ecological construction and social development. The country has proposed a low-carbon economy policy; development and management of enterprises are also actively responding to the call for a low-carbon economy. Through a series of innovations and changes, it intends to realize its own low-carbon production and business development, thereby promoting transformation and optimization of industrial institutions. Innovation is a significant driving force of national economic development. In order to build China into an innovative country, there is a need of national innovation strategy and innovation system. The state vigorously promotes the implementation of innovation-driven development strategies. Enterprises are not only an important carrier for the state to implement mass entrepreneurship and innovation but also a powerful driving force for development of economy. There is a dire need and importance to identify techniques to evaluate impact of corporate innovation on development effects with low-carbon economy. Considering this aforementioned problem, random initialization parameters in back propagation network are used. This evaluation process can easily lead to the functioning of the model falling in the local optimum. The work designs an impact evaluation model (IGWO-BP) with improved gray wolf algorithm (IGWO) in order to optimize BP network. To improve optimization ability for GWO, the study uses the chaotic map to initialize the population. The nonlinear convergence factor strategy as well as dynamic weight strategy is used to promote GWO, so as to optimize initial weight as well as threshold point for the BP network. The IGWO-BP is applied to perform evaluation of the impact in case of enterprise innovation relevant to development effects with low-carbon economy.

Efficient Robust Training via Backward Smoothing

Adversarial training is so far the most effective strategy in defending against adversarial examples. However, it suffers from high computational costs due to the iterative adversarial attacks in each training step. Recent studies show that it is possible to achieve fast Adversarial Training by performing a single-step attack with random initialization. However, such an approach still lags behind state-of-the-art adversarial training algorithms on both stability and model robustness. In this work, we develop a new understanding towards Fast Adversarial Training, by viewing random initialization as performing randomized smoothing for better optimization of the inner maximization problem. Following this new perspective, we also propose a new initialization strategy, backward smoothing, to further improve the stability and model robustness over single-step robust training methods. Experiments on multiple benchmarks demonstrate that our method achieves similar model robustness as the original TRADES method while using much less training time (~3x improvement with the same training schedule).